starlingx
/
config
Code Issues Proposed changes
StarlingX System Configuration Management
4,298 Commits 22 Branches 10 Tags 79 MiB
Python 97.3%
Shell 2.3%
CSS 0.2%
Go to file
Dan Voiculeasa 00c2129a16 AppFwk: Recover apply from helm operation in progress 
It is observed that when a helm release is in pending state, another
helm release can't be started by FluxCD. FluxCD will not try to
do steps to apply the newer helm release, but will just error.

This prevents us from applying a new helm release over a release with
pods stuck in Pending state (just an example).

When the specific message for helm operation in progress is detected,
attempt to recover by moving the older releases to failed state.
Move inspired by [1].
To do so, patch the helm secret for the specific release.
As an optimization, trigger the FluxCD HelmRelease reconciliation right
after.
One future optimization we can do is run an audit to delete the helm
releases for which metadata status is a pending operation, but release
data is failed (resource that we patched in this commit).

Refactor HelmRelease resource reconciliation trigger, smaller size.

There are upstream references related to this bug, see [2] and [3].

Tests on Debian AIO-SX:
PASS: unlocked enabled available
PASS: platform-integ-apps applied
after reproducing error:
PASS: inspect sysinv logs, see recovery is attemped
PASS: inspect fluxcd logs, see that HelmRelease reconciliation is
triggered part of recovery

[1]: https://github.com/porter-dev/porter/pull/1685/files
[2]: https://github.com/helm/helm/issues/8987
[3]: https://github.com/helm/helm/issues/4558
Closes-Bug: 1997368
Signed-off-by: Dan Voiculeasa <dan.voiculeasa@windriver.com>
Change-Id: I36116ce8d298cc97194062b75db64541661ce84d
 		2022-11-24 12:21:48 +02:00
api-ref/source Add runtime reconfiguration of kubelet 2022-06-09 17:59:35 -04:00
config-gate debian: Remove package preset install for config 2022-09-27 08:20:41 +00:00
controllerconfig Merge "Skip 67 activation script for DC subcloud upgrade" 2022-11-16 21:40:45 +00:00
devstack Deprecate old policy engine and restrict access 2022-08-10 11:18:38 -03:00
doc Fix tox-docs failing sphinx 2022-05-31 13:56:30 +00:00
releasenotes Remove host hardware sysinv profile 2021-10-18 18:01:40 -03:00
storageconfig Add storageconfig.service file 2022-10-13 10:27:23 -04:00
sysinv AppFwk: Recover apply from helm operation in progress 2022-11-24 12:21:48 +02:00
tmp/patch-scripts/EXAMPLE_SYSINV/scripts StarlingX open source release updates 2018-05-31 07:35:52 -07:00
tools/docker/images Enable kubernetes SCTPSupport feature 2019-09-03 19:23:05 +00:00
tsconfig Removed unused code in upgrade. 2022-09-28 22:08:34 -04:00
workerconfig Debian: Reorganize controller&worker config related packages 2022-08-04 19:19:25 +03:00
.gitignore Minor zuul and tox file cleanup after manifest re-org 2019-09-06 15:40:37 -05:00
.gitreview OpenDev Migration Patch 2019-04-19 19:52:42 +00:00
.yamllint clear yamllint errors under stx-config 2018-09-12 21:11:57 +08:00
.zuul.yaml Cleanup tox for python3.9 jobs 2022-03-25 20:32:09 +00:00
CONTRIBUTORS.wrs StarlingX open source release updates 2018-05-31 07:35:52 -07:00
LICENSE StarlingX open source release updates 2018-05-31 07:35:52 -07:00
README.rst StarlingX open source release updates 2018-05-31 07:35:52 -07:00
bindep.txt py3: Add py39 gate for sysinv 2021-08-27 08:39:06 -04:00
centos_build_layer.cfg Build layering, add layer build config file 2019-10-15 12:29:05 +08:00
centos_dev_wheels.inc Config file changes to add 'tsconfig' after relocation from 'update' 2019-09-05 11:51:05 -04:00
centos_iso_image.inc Merge sysinv_fpga_agent with sysinv_agent 2022-10-03 14:12:28 -04:00
centos_pkg_dirs Merge sysinv_fpga_agent with sysinv_agent 2022-10-03 14:12:28 -04:00
centos_pkg_dirs_containers Config file changes for packages relocated to repo 'openstack-armada-app' 2019-09-05 10:42:00 -04:00
centos_stable_wheels.inc Config file changes to add 'tsconfig' after relocation from 'update' 2019-09-05 11:51:05 -04:00
debian_build_layer.cfg Add debian_build_layer.cfg file 2021-10-05 14:50:08 -04:00
debian_iso_image.inc Debian: config: update debian_iso_image.inc 2022-11-08 15:48:04 +08:00
debian_pkg_dirs Merge sysinv_fpga_agent with sysinv_agent 2022-10-03 14:12:28 -04:00
debian_stable_wheels.inc debian: add cgts-client wheel to the build 2022-10-27 18:09:15 +00:00
test-requirements.txt Calling an additional shell lint command from zuul 2021-06-03 17:35:50 -05:00
tox.ini Calling an additional shell lint command from zuul 2021-06-03 17:35:50 -05:00

README.rst

stx-config

StarlingX Configuration Management