Pods were not being deleted when rbd-provisioner or cephfs-provisioner
charts are disabled and then reapplied the app.
We needed to add new value in helm.sh/hook annotation on deployment
resource to delete pods when chart is disabled and then reapplied
platform-integ-apps.
Test Plan:
PASS: Apply platform-integ-apps, disable rbd-provisioner chart,
then reapply App and Check if chart related pods, FluxCD helmrelease
and helm release are all removed.
PASS: Apply platform-integ-apps, disable cephfs-provisioner chart,
then reapply App and Check if chart related pods, FluxCD helmrelease
and helm release are all removed.
Closes-Bug: #2029496
Change-Id: Ie4353cab3cf93abf362dc1e55265d4eb6a9d017e
Signed-off-by: Gustavo Ornaghi Antunes <gustavo.ornaghiantunes@windriver.com>
Test Plan:
PASSED: The platform-integ-apps debian package was built succesfully
Story: 2010769
Task: 48235
Depends-On: https://review.opendev.org/c/starlingx/app-node-feature-discovery/+/885097
Change-Id: I14b739127ac1da9c794a7eb253a52690cf3da93f
Signed-off-by: Marcos Paulo Oliveira Silva <Marcos.PauloOliveiraSilva@windriver.com>
Updating the rsa ssh host key based on:
https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/
Note: In the future, StarlingX should have a zuul job and
secret setup for all repos so we do not need to do this
for every repo.
Needed to rename the secret, because zuul fails if like-named
secrets have diffent values in different branches of the same
repo.
Partial-Bug: #2015246
Change-Id: I1ebaccd444d78478d99450bf5a06a92ebae768f8
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
According to Jim's comments in [1], the ceph-fs and rbd init_storage
scripts needed improvement. In this way, the return codes of some
commands were checked, indentation problems were corrected,
in addition to making the log more readable.
Test Plan:
PASS: Update platform-integ-apps
PASS: Check logs with
"kubectl logs -n kube-system rbd-storage-init-XXXXX"
PASS: Check logs with
"kubectl logs -n kube-system cephfs-storage-init-XXXXX"
Story: 2010688
Task: 47806
[1] https://review.opendev.org/c/starlingx/platform-armada-app/+/871789
Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
Change-Id: I4191af4e5af541443744a20e43ace8d4e9e7b847
The platform-integ-apps docker images were pointing to the
public registry. This was caused because the path for each
image was defined in values.yml file for cephfs and rbd provisioners,
but not defined in the static overrides files.
This change adds the image paths to the static overrides files, so
Conductor can change the prefix to registry.local:9001 in the
deployments.
Test-Plan:
PASS: Fresh install AIO-SX and apply platform-integ-apps
application. Verify if the deployments have all images
pointing to registry.local:9001 registry.
Closes-bug: 2015788
Change-Id: Icb04d4fb9c9073eeb8de4bd9507a8460f0fbda14
Signed-off-by: Felipe Sanches Zanoni <Felipe.SanchesZanoni@windriver.com>
This change is part of solution to add userID and userKey
to rbd/cephfs secrets. Updating helmrelease and appvesion for
stx-rbd-provisioner and stx-cephfs-provisioner allows changes made
to patches related to rbd-storage-init and cephfs-storage-init jobs
can be applied via application-update avoiding an extra manual step.
This is needed to trigger a new revision on stx-rbd-provisioner and
stx-cephfs-provisioner during helm upgrade.
Test Plan:
PASS: build platform-helm, python3-k8sapp-platform and stx-platform-helm
packages successfully.
PASS: secrets updated after apply using application-update.
PASS: new revision was released.
Partial-bug: 2003907
Depends-On: https://review.opendev.org/c/starlingx/platform-armada-app/+/871789
Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
Change-Id: Ie4427bdc801c07c170e87dddb35018c9edc64efa
The batch/v1beta1 API version of CronJob is no longer served as of
v1.25.
Migrate API clients to use the batch/v1 API version instead which is
available since k8s 1.21.
Test Plan:
PASS: Install ISO with k8s 1.24 on AIO-SX
PASS: Install ISO with k8s 1.25 on AIO-SX
PASS: Apply platform-armada-app and verify if ceph-pools-audit is
running
PASS: Verify that batch/v1 API version is being used
Story: 2010368
Task: 47291
Signed-off-by: Sachin Gopala Krishna <saching.krishna@windriver.com>
Change-Id: I29fb40a46ff8a8e9fb076667499fa795e07970b6
The Debian packaging has been changed to reflect all the
git commits under the directory, and not just the commits
to the metadata folder.
This ensures that any new code submissions under those
directories will increment the versions.
Test Plan:
PASS: build-pkgs -p platform-helm
Story: 2010550
Task: 47547
Signed-by: Pedro Vinícius Silva da Cruz
<pedro.silvadacruz@windriver.com>
Change-Id: I0925205e0368acd08431c697682da7832544497c
This allows platform-integ-apps to be automatically updated.
This is helpful when providing patches, removing the need for manual
intervention.
Test Plan:
PASS: Rebuild platform-integ-apps containing "auto_update: true" then
manually update. Created another designer tarball with version
bumped and observed the auto update process be triggered
automatically and finished successfully.
Story: 2010628
Task: 47576
Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
Change-Id: I51143231d07c770113600d01b72052e8c2460882
Updated a couple variable names to better
represent their contents.
Note this change has no impact on running apps
or build process, it's just for clarity.
Tests:
pass build-pkgs
Story: 2010542
Task: 47574
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I5e0c9a350fdb3a30f000cef10d88f386842d82ad
Fixed versioning and adjusted some build files
to bring them as close to a standard as possible.
- Removed centos files
- Added version tracking via GITREVCOUNT
- Fixed mismatch in plugin name, set to python3-k8sapp-<app>
- Standardized plugin debian files (rules, *.install)
- Plugin wheels saved to /plugin instead of /plugin/<app>
Test Plan:
PASS - Build-pkgs -a
PASS - Build-image
PASS - Install, bootstrap, unlock
PASS - app tarball contains wheel file
PASS - wheel versioning updated properly
Story: 2010542
Task: 47193
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I601800969134661ff40d6fb6277ffad3ce403eb7
tox changes:
- remove python2.7 and python3.6 unit test targets
from tox.ini since only python 3.9 is supported
- tox jobs are now setup for python 3.9
- Added additional suppressions for new pylint codes.
Zuul changes:
- remove py27 job from zuul
- set the parent zuul jobs to be the jobs defined under
openstack-zuul-jobs rather than just zuul-jobs
- Specify debian-bullseye (python3.9) as the nodeset.
ubuntu-jammy is python3.10, ubuntu-focal is python3.8
- override tox constraints file to use the STX constraints
This means starlingx/root needs to be a required project
Notes:
- these changes do not affect runtime.
Test Plan:
pass tox
pass build-pkgs
Story: 2010531
Task: 47511
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I4649c60e669b546ff8b31df832e6e16b348f2fee
To create a new RBD volume snapshot the userID and userKey fields
are required in the RBD secret. These fields are required by
csi-snapshotter container in the rbd-provisioner pod.
Official examples show that the RBD secrets need userID and userKey
fields, while cephfs needs adminID and adminKey required
for dynamically provisioned volumes and userID and userKey required
for statically provisioned volumes. That's why these fields were also
added in cephfs secret.
Test Plan:
PASS: Create RBD volume snapshot and check if it's ready to use.
PASS: Restore RBD volume snapshot.
PASS: Create Cephfs volume snapshot and check if it's ready to use.
PASS: Restore Cephfs volume snapshot.
Closes-bug: 2003907
Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
Change-Id: If526d8412f71da936a66ab783c5919bba0c6611d
Add kube-upgrade-complete trigger for reevaluating app reaplies
The kube-upgrade-complete trigger was introduced to resume the
app reapply evaluation process after a Kubernetes upgrade is completed.
Test Plan:
PASS: Full image build
PASS: AIO-SX deployment
Relates-to: https://review.opendev.org/c/starlingx/config/+/870990
Closes-Bug: 2003260
Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
Change-Id: I85973b076aa1d05ca60cc58d0d26dd93c84d7092
It was observed when updating the app part of platform upgrade that
it loses user overrides. I suspect a day 0 slip. All the other apps
already have a flag enabled to keep the user overrides during
app updates.
Enable 'maintain_user_overrides' flag, if present on N+1 version then it
will keep the N version user overrides.
If there are any kind of user overrides that need transformation
between N->N+1 update, these must be handled in the upgrade scripts,
helm charts or in the lifecycle hooks(maybe create new lifecycle
hook for this purpose).
Tests:
PASS: tested stx.5.0 to stx.6.0 upgrade, with patched
platform-integ-apps, user overrides are kept
Closes-Bug: 2002301
Signed-off-by: Dan Voiculeasa <dan.voiculeasa@windriver.com>
Change-Id: I8d9563519129372e35d328942813256f1a3c1e38
This change will allow this repo to pass zuul now
that this has merged:
https://review.opendev.org/c/zuul/zuul-jobs/+/866943
Tox 4 deprecated whitelist_externals.
Replace whitelist_externals with allowlist_externals
Zuul is pinned to use tox < 4 in order for the py39
unit tests to pass. Additional changes for skipsdist
may be required in order to work properly with tox 4.
Partial-Bug: #2000399
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Iab988c50b18d683a0d0967cbcc31bbad621005f8
Ceph-csi, part of platform-integ-apps application, was not able
to create multiple StorageClasses like the older provisioner.
Code has been changed to accept multiple StorageClasses from
helm-override command.
Test Plan:
PASS: On AIO-SX, create rbd pvc, create pod
and verify pod is running with no errors
PASS: On AIO-SX, create cephfs pvc, create pod
and verify pod is running with no errors
PASS: On AIO-SX, upgrade platform activation complete
PASS: On Storage setup, create new storage tier,
add new storage backend, add disk to storage-0 for this new
backend. Unlock storage-0 and verify StorageClass is created
for the new storage backend.
PASS: Add helm override with multiple StorageClasses,
reapply platform-integ-apps and verify creation of
all StorageClasses. Add new pvc and pod to use the
new StorageClasses and verify pod is running with no errors.
Closes-bug: 1999169
Signed-off-by: Felipe Sanches Zanoni <Felipe.SanchesZanoni@windriver.com>
Change-Id: I521450cf92fa914aec4c1b7fabd7840a3d984954
Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
Set reconciliation interval for all flux helm resources to 1m
to allow it to manage resources by itself in a reasonable time
interval.
Test Plan (tested as part of [1]):
PASS: bootstrap
PASS: unlocked enabled available
PASS: apps applied
PASS: inspect flux pod logs for errors
PASS: re-test known trigger for 1996747 and 1995748
PASS: re-test known trigger 1997368
[1] https://review.opendev.org/c/starlingx/config/+/866862
Partial-Bug: 1999032
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I4b7b663107a96427b87162fbf1c68b26e4e8fc01
One of the build configurations is listing '1.0' as the specific
version to use for one of its files, but this restriction is
unnecessary and causes the build to fail when the app version
is not 1.0.
Test Plan:
PASS build app using a version different than '1.0'
(modify meta_data.yaml and changelog)
Closes-Bug: 1998637
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I9e77b570262658dbed4570fa378a4ceeb4634aa4
Move the packages of "platform-armada-app" from stx-std.lst
to debian_iso_image.inc
A subsequent commit will be sent to clean up stx-std.lst.
Test Plan:
Pass: build-pkgs -c -a
Pass: build-image
Pass: boot
Story: 2008862
Task: 46923
Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: I95c74db378073714ffbc26638d44c571128fa9d2
Remove old RBD/CephFS provisioners and replace with a currently
supported and evolving set of provisioners based on
https://github.com/ceph/ceph-csi version 3.6.2.
Test Plan:
PASS: AIO-SX app upload/apply/remove/delete/update
PASS: AIO-DX app upload/apply/remove/delete
PASS: Storage 2+2+2 app upload/apply/remove/delete
PASS: Create pvc using storageclass general (rbd) on SX/DX/Storage
PASS: Create pod using rbd pvc on SX/DX/Storage
PASS: Create pvc using storageclass cephfs on SX/DX/Storage
PASS: Create pod using cephfs pvc on SX/DX/Storage
Story: 2009987
Task: 45050
Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
Change-Id: Iffcd56f689aa70788c4c2abbbf2c9a02b5a797cf
The ubuntu-jammy nodeset gets selected by default
and is causing problems during setup.
Collecting cffi>=1.1
Failed to build cffi
ubuntu-focal seem to work fine.
Will specify the nodeset to be focal to resolve this.
Need to update a file under python-k8sapp-platform in
order to trigger the failing zuul jobs.
Partial-Bug: 1994843
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I18235473c04fbcf97dd5df676f054f34cd18b8b8
After the introduction of FluxCD, there have been issues on
platform-integ-apps auto re-apply on SX to DX migration.
This change adds the following helm hooks that were missing from
StorageClass:
- "helm.sh/hook": "pre-upgrade, pre-install"
- "helm.sh/hook-delete-policy": "before-hook-creation"
Test Plan:
PASS: CentOS - Create platform-integ-apps tgz with these changes and
test the migration.
PASS: Debian - Create platform-integ-apps tgz with these changes and
test the migration.
Closes-bug: 1970443
Signed-off-by: Enzo Candotti <enzo.candotti@windriver.com>
Change-Id: I3aa2e8f6cbb7504f9a4bf368d90f130dd5131106
As part of Armada deprecation we need to remove all Armada application
builds for all applications that have been migrated to FluxCD.
This patch removes the armada app build from centos and debian.
TEST PLAN:
PASS: Build centos
PASS: Build debian
PASS: rpm package has no armada tarball
PASS: deb package has no armada tarball
PASS: fluxcd tarball is unchanged
Story: 2009138
Task: 46046
Signed-off-by: Lucas Cavalcante <lucasmedeiros.cavalcante@windriver.com>
Change-Id: Ic1d31c3aba2e6c43c436f679f0a3f4c8479757a5
This commit makes the platform-app complies with the FluxCD capability
to disable HelmReleases instead of charts.
TEST PLAN
PASS Install stx-platform-helm rpm, update the app and
enable/disable the charts via helm-chart-attribute-modify
PASS Verify resources created with app enabled and disabled
LOGS: https://paste.opendev.org/show/b2rRUhTWY6mESmiav6UO/
Story: 2009138
Task: 45615
Depends-On: https://review.opendev.org/c/starlingx/config/+/845155
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I282737ccc6139b36baccb097933fa9885e1b8d52
On [1] we switched the platform app to use the FluxCD version of the app
by default and keep packaging the armada app as an option to allow
testing, but we missed renaming it on the centos_iso_image.inc. This
commit fixes it.
[1] https://review.opendev.org/c/starlingx/platform-armada-app/+/843543
Story: 2009138
Task: 45482
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: Ief9667716269981566ad00396e08d481eb978440
This commit switches platform-app to use the fluxcd app by default on
Debian and also preserves the armada app on the build for future
tests.
TEST PLAN
PASS Debian: build ISO and install, verify that FluxCD app is installed
PASS CentOS: build ISO and install, verify that FluxCD app is installed
Story: 2009138
Task: 45482
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: If0f1cd6f3ea64f2af408ec0b43045baeb3db48cb
Add the fluxcd app for platform-integ-apps to the debian build.
Some code repetition is intentional to ease removing armada in the
future.
TEST PLAN
PASS Install new .deb and install FluxCD app
PASS Verify created resources by FluxCD
PASS Install armada app
PASS Verify created resources by armada
Logs: https://paste.opendev.org/show/bRuKugVnX3VRRCvqeY3j/
Story: 2009138
Task: 45119
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I49b6477ce5eb72ece4a46a218988bde0f47e5717
Add an example of how to integrate a FluxCDKustomizeOperator plugin that
supports enabling/disabling helm charts dynamically based on the
platform configuration.
This is the FluxCD equivalent of the PlatformArmadaManifestOperator.
Change-Id: I4f35c0ab74e9b2b9599b50aef63a899829b6355e
Story: 2009138
Task: 45413
Depends-On: https://review.opendev.org/c/starlingx/config/+/842466
Signed-off-by: Robert Church <robert.church@windriver.com>
Add new manifest files for the FluxCD version of platform-integ-apps
This creates two tarballs, one for aramada and one for FluxCD
* job-ceph-pools-audit.yaml change to support helmv3
Differences from armada-app:
* Helm-toolkit release is disabled as it is included in other charts
Test (AIO-SX):
PASS: Application is applied
Story: 2009138
Task: 45118
Depends-on: https://review.opendev.org/c/starlingx/config/+/838321
Signed-off-by: Lucas Cavalcante <lucasmedeiros.cavalcante@windriver.com>
Change-Id: I417d6b7cf6a12968f36a1045dc98c2c26579a9b8
Use docker.io/openstackhelm/ceph-config-helper:ubuntu_bionic-20201223
instead of docker.io/starlingx/ceph-config-helper:v1.15.0. With this,
we align the ceph client version with the cluster and also go back to
the upstream versions.
Test plan:
Build armada app
Clean all pools and apply platform-integ-apps
After applying, verify that there are 3 new pools created
Check if kube-cephfs is configured
Inspect pods yaml to check images used
Check if ceph-pools-audit job is running successfully
Story: 2009867
Task: 44555
Signed-off-by: Delfino Curado <delfinogomes.curadofilho@windriver.com>
Change-Id: If73ab5666d1c6694f718f498e72ad84c522d1ab6
Replace dl_hook with "src_path" and "src_files"
Test Plan:
Pass: successfully build stx-platform-helm
Pass: No difference comparing with the result of dl_hook
Story: 2009101
Task: 43792
Signed-off-by: Yue Tao <Yue.Tao@windriver.com>
Change-Id: I624cfb5ab184a8aadf0bbcfb17ed0fb69da09e2e
Add a toleration for the node-role.kubernetes.io/master:NoSchedule
taint. This taint will be restored to all standard (non-AIO) master
nodes to prevent user pods from being scheduled and run. These workloads
will be scheduled and run on a worker node.
This change will ensure that the rbd/cephfs provisioner and ceph audit
pods will continue run on the master nodes (as designed).
The following tests were executed:
- Using an the existing app, remove the application, add the
node-role.kubernetes.io/master:NoSchedule taint and confirmed that
upon application-apply that the application will not apply as the pods
will be in a Pending state.
- With the existing application applied, add the
node-role.kubernetes.io/master:NoSchedule taint and execute an
application-update to a new application with these changes. Confirmed
that the application updates successfully. Confirmed that toleration
is present when describing the pod(s).
Change-Id: I0a6368c717d336ac6c024bda596c283d2943285b
Depends-On: https://review.opendev.org/c/starlingx/config/+/812629
Story: 2009232
Task: 43346
Signed-off-by: Robert Church <robert.church@windriver.com>
Removing redundant py36 Zuul jobs since we now have py39 Zuul jobs in
place with the debian nodeset
Story: 2006796
Task: 43493
Signed-off-by: Bernardo Decco <bernardo.deccodesiqueira@windriver.com>
Change-Id: I7cf62d671c48d45f6ddff3ecad7fa013744001a5
We are updating openstack-helm-infra for stx-openstack
and all dependencies need to be updated accordingly.
Depends-On: https://review.opendev.org/c/starlingx/openstack-armada-app/+/808800/
Story: 2009161
Task: 43151
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I769b743f46ae472f1e47df9681b4badf89a2567b
Due to a recent change in fm-api's directory structure, unit tests would
fail since the virtualenv would not be able to find fm-api/setup.py.
Adjust the tox.ini to point to the correct directory. Tested locally
by running tox.ini.
Depends-On: https://review.opendev.org/c/starlingx/fault/+/806046
Story: 2009101
Task: 43091
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I0960205bedb04afcc2411bb1abe892af946a7a39
Gustavo Ornaghi Antunes