A new command "system application-update" is introduced in this
commit to support updating an applied application to a new version
with a new versioned app tarfile.
The application update leverages the existing application upload
workflow to first validating/uploading the new app tarfile, then
invokes Armada apply or rollback to deploy the charts for the new
versioned application. If the version has ever applied before,
Armada rollback will be performed, otherwise, Armada apply will be
performed.
After apply/rollback to the new version is done, the files for the
old application version will be cleaned up as well as the releases
which are not in the new application version. Once the update is
completed successfully, the status will be set to "applied" so that
user can continue applying app with user overrides.
If there has any failure during updating, application recover will be
triggered to recover the app to the old version. If application recover
fails, the application status will be populated to "apply-failed" so
that user can re-apply app.
In order to use Armada rollback, a new sysinv table "kube_app_releases"
is created to record deployed helm releases versions. After each app
apply, if any helm release version changed, the corresponding release
needs to be updated in sysinv db as well.
The application overrides have been changed to tie to a specific
application in commit https://review.opendev.org/#/c/660498/. Therefore,
the user overrides is preserved when updating.
Note: On the AIO-SX, always use Armada apply even it was applied issue
on AIO-SX(replicas is 1) to leverage rollback, Armada/helm
rollback --wait does not wait for pods to be ready before it
returns.
Related helm issue,
https://github.com/helm/helm/issues/4210https://github.com/helm/helm/issues/2006
Tests conducted(AIO-SX, DX, Standard):
- functional tests (both stx-openstack and simple custom app)
- upload stx-openstack-1.0-13-centos-stable-latest tarfile
which uses latest docker images
- apply stx-openstack
- update to stx-openstack-1.0-13-centos-stable-versioned
which uses versioned docker images
- update back to stx-openstack-1.0-13-centos-stable-latest
- update to a version that has less/more charts compared to
the old version
- remove stx-openstack
- delete stx-openstack
- failure tests
- application-update rejected
(app not found, update to a same version,
operation not permitted etc...)
- application-update fails that trigger recover
- upload failure
ie. invalid tarfile, manifest file validation failed ...
- apply/rollback failure
ie. download images failure, Armada apply/rollback fails
Change-Id: I4e094427e673639e2bdafd8c476b897b7b4327a3
Story: 2005350
Task: 33568
Signed-off-by: Angie Wang <angie.wang@windriver.com>
28d069bccc