Auto-increment chart versions

Enable auto-versioning of helm charts to ensure the FluxCD helm
controller recognizes chart changes.

To guarantee the helm chart version is incremented when a helm chart
change is submitted, the following is implemented:
 - Provide a top level hierarchy for helm charts to differentiate
   between upstream and custom charts: helm-charts/{custom,upstream}
 - Move platform-helm package to helm-charts/upstream
 - Move ceph-pools-audit to helm-charts/custom.
 - stx-APP-helm now contains only manifests and final application
   packaging rules. No custom helm charts should be delivered here.
 - Use StarlingX debian git revcount packaging mechanisms to derive the
   semver PATCH version for custom helm charts. Chart
   version (MAJOR.MINOR.PATCH) is passed to 'helm package' command to
   force the version, where PATCH == 'git revcount'
 - Use StarlingX debian git revcount packaging mechanisms to derive the
   semver BUILD version for upstream helm charts which maintains the
   upstream chart version and adds a versioned BUILD extension.

     <version core> ::= <major> "." <minor> "." <patch>
     <valid semver> ::= <version core> "+" <build>

   Chart version (MAJOR.MINOR.PATCH+STX.REV) is passed to 'helm package'
   command to force the version, where REV == 'git revcount'
 - Establish a new package for the custom 'ceph-pools-audit' chart
   called 'ceph-pools-audit-helm'
 - Enforce a baseline version for both 'platform-helm' and
   'ceph-pools-audit-helm' based on existing pre-move chart changes.

   $ git rev-list --count HEAD platform-helm/
   16

   $ git rev-list --count HEAD \
       stx-platform-helm/stx-platform-helm/helm-charts/ceph-pools-audit
   19
 - Update 'stx-platform-helm' to:
   - Update the build dependencies to include the new helm chart package
     and remove dependency on helm
   - Update the rules to remove building the ceph-pools-audit chart and
     automatically update the chart versions in the fluxCD
     helmrelease.yaml files.
 - Fix ceph-csi-rbd storage-init.yaml Job chart label as it was
   generating an invalid label based on the new auto versioning scheme.

Additional changes and cleanup:
 - Remove 'helm dep' commands from chart Makefile as no chart
   dependencies are present
 - Fix formatting in 'platform-helm' and 'stx-platform-helm' changelogs
   to avoid dpkg-genchanges errors.

Test Plan:
PASS - Build all packages generating an application tarball verifying
       all versions on the charts and application make sense.
PASS - Introduce temporary chart changes and ensure that the versions
       increment as expected.
PASS - Validate basic application lifecycle operations:
       upload/apply/remove/delete.

Change-Id: Ie5d5b9c7bb080210be18b0d886631638e705657f
Story: 2010929
Task: 49363
Signed-off-by: Robert Church <robert.church@windriver.com>
This commit is contained in:
Robert Church 2023-12-30 09:56:42 -06:00
parent 9195bfb887
commit 829555e0d9
49 changed files with 185 additions and 59 deletions

View File

@ -1,3 +1,4 @@
platform-helm
helm-charts/custom/ceph-pools-audit-helm
helm-charts/upstream/platform-helm
python3-k8sapp-platform
stx-platform-helm

View File

@ -25,20 +25,16 @@ $(CHARTS):
init-%:
if [ -f $*/Makefile ]; then make -C $*; fi
if [ -f $*/requirements.yaml ]; then helm dep up $*; fi
lint-%: init-%
if [ -d $* ]; then helm lint $*; fi
@echo "Clobber dependencies from packaging"
rm -v -f $*/requirements.lock $*/requirements.yaml
build-%: lint-%
if [ -d $* ]; then helm package $*; fi
if [ -d $* ]; then helm package --version $(CHART_VERSION) $*; fi
clean:
@echo "Clean all build artifacts"
rm -f */templates/_partials.tpl */templates/_globals.tpl
rm -f *tgz */charts/*tgz */requirements.lock
rm -rf */charts */tmpcharts
%:

View File

@ -0,0 +1,5 @@
ceph-pools-audit-helm (0.1-0) unstable; urgency=medium
* Initial release.
-- Robert Church <robert.church@windriver.com> Fri, 29 Dec 2023 20:07:42 +0000

View File

@ -0,0 +1,15 @@
Source: ceph-pools-audit-helm
Section: libs
Priority: optional
Maintainer: StarlingX Developers <starlingx-discuss@lists.starlingx.io>
Build-Depends: debhelper-compat (= 13),
helm,
Standards-Version: 4.5.1
Homepage: https://www.starlingx.io
Package: ceph-pools-audit-helm
Section: libs
Architecture: any
Depends: ${misc:Depends}
Description: StarlingX Platform Ceph pool auditor helm chart
This package contains a helm chart to audit the replication factor of Ceph pools.

View File

@ -0,0 +1,41 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: ceph-pools-audit-helm
Source: https://opendev.org/starlingx/platform-armada-app/
Files: *
Copyright: (c) 2019-2023 Wind River Systems, Inc
License: Apache-2
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
https://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian-based systems the full text of the Apache version 2.0 license
can be found in `/usr/share/common-licenses/Apache-2.0'.
# If you want to use GPL v2 or later for the /debian/* files use
# the following clauses, or change it to suit. Delete these two lines
Files: debian/*
Copyright: 2023 Wind River Systems, Inc
License: Apache-2
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
https://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian-based systems the full text of the Apache version 2.0 license
can be found in `/usr/share/common-licenses/Apache-2.0'.

View File

@ -0,0 +1,27 @@
#!/usr/bin/make -f
# export DH_VERBOSE = 1
export ROOT = debian/tmp
export APP_FOLDER = $(ROOT)/usr/lib/helm
export DEB_VERSION = $(shell dpkg-parsechangelog | egrep '^Version:' | cut -f 2 -d ' ')
export RELEASE = $(shell echo $(DEB_VERSION) | cut -f 1 -d '-')
export REVISION = $(shell echo $(DEB_VERSION) | cut -f 4 -d '.')
export CHART_VERSION = $(RELEASE).$(REVISION)
%:
dh $@
override_dh_auto_build:
# Stage the chart for building
mkdir -p build
mv Makefile ceph-pools-audit build
# Build the chart
cd build && make CHART_VERSION=$(CHART_VERSION) ceph-pools-audit
override_dh_auto_install:
install -d -m 755 $(APP_FOLDER)
install -p -D -m 755 build/ceph-pools-audit*.tgz $(APP_FOLDER)
override_dh_auto_test:

View File

@ -0,0 +1,10 @@
---
debname: ceph-pools-audit-helm
debver: 0.1-0
src_path: ceph-pools-audit-helm
revision:
dist: $STX_DIST
stx_patch: 19
GITREVCOUNT:
SRC_DIR: ${MY_REPO}/stx/platform-armada-app/helm-charts/custom/ceph-pools-audit-helm/ceph-pools-audit-helm/ceph-pools-audit
BASE_SRCREV: 301e310b67f608c309a34c7431848f680e2ea827

View File

@ -2,13 +2,13 @@ platform-helm (3.9-0) unstable; urgency=medium
* New upstream release.
-- Gabriel de Araújo Cabral <gabriel.cabral@windriver.com> Fri, 21 Jul 2023 08:30:00 +0000
-- Gabriel de Araújo Cabral <gabriel.cabral@windriver.com> Fri, 21 Jul 2023 08:30:00 +0000
platform-helm (3.8-0) unstable; urgency=medium
* New upstream release.
-- Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com> Wed, 17 May 2023 08:35:00 +0000
-- Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com> Wed, 17 May 2023 08:35:00 +0000
platform-helm (1.0-1) unstable; urgency=medium

View File

@ -0,0 +1,30 @@
From: Robert Church <robert.church@windriver.com>
Date: Sun, 31 Dec 2023 12:49:27 -0600
Subject: Fix chart label for rbd-storage-init Job
Update the chart label on the rbd-storage-init Job to align with all the
other chart labels used throughout the chart.
This one-off implementation resulted in an invalid
label (ceph-csi-rbd-3.9.0+STX.14) based on the new chart auto-versioning
scheme. Using the common include mechanism provides a valid
label (ceph-csi-rbd-3.9.0_STX.14).
Signed-off-by: Robert Church <robert.church@windriver.com>
---
charts/ceph-csi-rbd/templates/storage-init.yaml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/charts/ceph-csi-rbd/templates/storage-init.yaml b/charts/ceph-csi-rbd/templates/storage-init.yaml
index 6abd8ec..0b36c58 100644
--- a/charts/ceph-csi-rbd/templates/storage-init.yaml
+++ b/charts/ceph-csi-rbd/templates/storage-init.yaml
@@ -344,7 +344,7 @@ spec:
labels:
heritage: {{ .Release.Service | quote }}
release: {{ .Release.Name | quote }}
- chart: "{{ .Chart.Name }}-{{- .Chart.Version }}"
+ chart: {{ include "ceph-csi-rbd.chart" . }}
spec:
serviceAccountName: {{ include "ceph-csi-rbd.serviceAccountName.provisioner" . }}
restartPolicy: OnFailure

View File

@ -15,3 +15,4 @@
0015-ceph-csi-cephfs-add-snapshotclass.patch
0016-ceph-csi-rbd-add-snapshotclass.patch
0017-ceph-csi-cephfs-remove-mountoptions-debug-on-pvs.patch
0018-Fix-chart-label-for-rbd-storage-init-Job.patch

View File

@ -0,0 +1 @@
usr/lib/helm/*

View File

@ -1,5 +1,10 @@
#!/usr/bin/make -f
export DH_VERBOSE = 1
# export DH_VERBOSE = 1
export DEB_VERSION = $(shell dpkg-parsechangelog | egrep '^Version:' | cut -f 2 -d ' ')
export PATCH_VERSION = $(shell echo $(DEB_VERSION) | cut -f 4 -d '.')
export CHART_BASE_VERSION = "3.9.0"
export CHART_VERSION = $(CHART_BASE_VERSION)+STX.$(PATCH_VERSION)
export ROOT = debian/tmp
export APP_FOLDER = $(ROOT)/usr/lib/helm
@ -8,7 +13,6 @@ export APP_FOLDER = $(ROOT)/usr/lib/helm
dh $@
override_dh_auto_build:
mkdir -p ceph-csi
# Copy ceph-csi charts
@ -16,8 +20,8 @@ override_dh_auto_build:
cp Makefile ceph-csi
cd ceph-csi && make ceph-csi-rbd
cd ceph-csi && make ceph-csi-cephfs
cd ceph-csi && make CHART_VERSION=$(CHART_VERSION) ceph-csi-rbd
cd ceph-csi && make CHART_VERSION=$(CHART_VERSION) ceph-csi-cephfs
override_dh_auto_install:
# Install the app tar file.

View File

@ -0,0 +1 @@
3.0 (quilt)

View File

@ -9,6 +9,7 @@ src_files:
- platform-helm/files/Makefile
revision:
dist: $STX_DIST
stx_patch: 16
GITREVCOUNT:
BASE_SRCREV: 69c37e9978d45dd928537523eda905e34e034b73
SRC_DIR: ${MY_REPO}/stx/platform-armada-app/platform-helm
SRC_DIR: ${MY_REPO}/stx/platform-armada-app/helm-charts/upstream/platform-helm

View File

@ -25,20 +25,16 @@ $(CHARTS):
init-%:
if [ -f $*/Makefile ]; then make -C $*; fi
if [ -f $*/requirements.yaml ]; then helm dep up $*; fi
lint-%: init-%
if [ -d $* ]; then helm lint $*; fi
@echo "Clobber dependencies from packaging"
rm -v -f $*/requirements.lock $*/requirements.yaml
build-%: lint-%
if [ -d $* ]; then helm package $*; fi
if [ -d $* ]; then helm package --version $(CHART_VERSION) $*; fi
clean:
@echo "Clean all build artifacts"
rm -f */templates/_partials.tpl */templates/_globals.tpl
rm -f *tgz */charts/*tgz */requirements.lock
rm -rf */charts */tmpcharts
%:

View File

@ -1,5 +0,0 @@
This directory contains all StarlingX charts that need to be built for this
application. Some charts are common across applications. These common charts
reside in the stx-config/kubernetes/helm-charts directory. To include these in
this application update the build_srpm.data file and use the COPY_LIST_TO_TAR
mechanism to populate these common charts.

View File

@ -2,13 +2,13 @@ stx-platform-helm (1.2-0) unstable; urgency=medium
* New upstream release.
-- Gabriel de Araújo Cabral <gabriel.cabral@windriver.com> Fri, 21 Jul 2023 08:40:00 +0000
-- Gabriel de Araújo Cabral <gabriel.cabral@windriver.com> Fri, 21 Jul 2023 08:40:00 +0000
stx-platform-helm (1.1-0) unstable; urgency=medium
* New upstream release.
-- Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com> Wed, 17 May 2023 08:45:00 +0000
-- Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com> Wed, 17 May 2023 08:45:00 +0000
stx-platform-helm (1.0-1) unstable; urgency=medium

View File

@ -3,11 +3,11 @@ Section: libs
Priority: optional
Maintainer: StarlingX Developers <starlingx-discuss@lists.starlingx.io>
Build-Depends: debhelper-compat (= 13),
helm,
python3-k8sapp-platform,
python3-k8sapp-platform-wheels,
build-info,
ceph-pools-audit-helm,
platform-helm,
build-info
python3-k8sapp-platform,
python3-k8sapp-platform-wheels
Standards-Version: 4.5.1
Homepage: https://www.starlingx.io
@ -16,4 +16,4 @@ Section: libs
Architecture: any
Depends: ${misc:Depends}
Description: StarlingX Platform FluxCD Helm Charts
This package contains FluxCD helm charts for the platform applications.
This package contains FluxCD manifests for the platform applications.

View File

@ -10,46 +10,54 @@ export REVISION = $(shell echo $(DEB_VERSION) | cut -f 4 -d '.')
export APP_NAME = platform-integ-apps
export APP_VERSION = $(RELEASE)-$(REVISION)
export APP_TARBALL_FLUXCD = $(APP_NAME)-$(APP_VERSION).tgz
export APP_TARBALL = $(APP_NAME)-$(APP_VERSION).tgz
export HELM_FOLDER = /usr/lib/helm
export HELM_REPO = stx-platform
export STAGING_FLUXCD = staging-fluxcd
export STAGING = staging
%:
dh $@
override_dh_auto_build:
# Create the TGZ file.
cd helm-charts && make ceph-pools-audit
# Setup the staging directory.
mkdir -p $(STAGING_FLUXCD)
cp files/metadata.yaml $(STAGING_FLUXCD)
cp -Rv fluxcd-manifests $(STAGING_FLUXCD)
mkdir -p $(STAGING_FLUXCD)/charts
cp helm-charts/*.tgz $(STAGING_FLUXCD)/charts
cp /usr/lib/helm/ceph-csi-*.tgz $(STAGING_FLUXCD)/charts
mkdir -p $(STAGING)
cp files/metadata.yaml $(STAGING)
cp -Rv fluxcd-manifests $(STAGING)
mkdir -p $(STAGING)/charts
cp /usr/lib/helm/*.tgz $(STAGING)/charts
# Adjust the helmrelease yamls based on the chart versions
for c in $(STAGING)/charts/*; do \
chart=$$(basename $$c .tgz); \
chart_name=$${chart%-*}; \
chart_version=$${chart##*-}; \
echo "Found $$chart; name: $$chart_name, version: $$chart_version"; \
chart_manifest=$$(find $(STAGING)/fluxcd-manifests -name helmrelease.yaml -exec grep -q $$chart_name {} \; -print); \
echo "Updating manifest: $$chart_manifest"; \
sed -i "s/REPLACE_HELM_CHART_VERSION/$$chart_version/g" $$chart_manifest; \
grep version $$chart_manifest; \
done
# Populate metadata.
sed -i 's/APP_REPLACE_NAME/$(APP_NAME)/g' $(STAGING_FLUXCD)/metadata.yaml
sed -i 's/APP_REPLACE_VERSION/$(APP_VERSION)/g' $(STAGING_FLUXCD)/metadata.yaml
sed -i 's/HELM_REPLACE_REPO/$(HELM_REPO)/g' $(STAGING_FLUXCD)/metadata.yaml
sed -i 's/APP_REPLACE_NAME/$(APP_NAME)/g' $(STAGING)/metadata.yaml
sed -i 's/APP_REPLACE_VERSION/$(APP_VERSION)/g' $(STAGING)/metadata.yaml
sed -i 's/HELM_REPLACE_REPO/$(HELM_REPO)/g' $(STAGING)/metadata.yaml
# Copy the plugins: installed in the buildroot
mkdir -p $(STAGING_FLUXCD)/plugins
cp /plugins/*.whl $(STAGING_FLUXCD)/plugins
mkdir -p $(STAGING)/plugins
cp /plugins/*.whl $(STAGING)/plugins
# Create the app package.
cd $(STAGING_FLUXCD) && find . -type f ! -name '*.md5' -print0 | xargs -0 md5sum > checksum.md5
tar cfz $(APP_TARBALL_FLUXCD) -C $(STAGING_FLUXCD)/ .
cd $(STAGING) && find . -type f ! -name '*.md5' -print0 | xargs -0 md5sum > checksum.md5
tar cfz $(APP_TARBALL) -C $(STAGING)/ .
# Cleanup staging
rm -rf $(STAGING_FLUXCD)
rm -rf $(STAGING)
override_dh_auto_install:
# Install the app tar file.
install -d -m 755 $(APP_FOLDER)
install -p -D -m 755 $(APP_TARBALL_FLUXCD) $(APP_FOLDER)
install -p -D -m 755 $(APP_TARBALL) $(APP_FOLDER)
override_dh_auto_test:

View File

@ -1,6 +0,0 @@
This directory contains all StarlingX charts that need to be built to support
platform integration immediately after installation. Some charts are common
across applications. These common charts reside in the
stx-config/kubernetes/helm-charts directory. To include these in this
application update the build_srpm.data file and use the COPY_LIST_TO_TAR
mechanism to populate these commom charts.

View File

@ -15,7 +15,7 @@ spec:
chart:
spec:
chart: ceph-pools-audit
version: 0.1.0
version: REPLACE_HELM_CHART_VERSION
sourceRef:
kind: HelmRepository
name: stx-platform

View File

@ -15,7 +15,7 @@ spec:
chart:
spec:
chart: ceph-csi-cephfs
version: 3.9.0
version: REPLACE_HELM_CHART_VERSION
sourceRef:
kind: HelmRepository
name: stx-platform

View File

@ -15,7 +15,7 @@ spec:
chart:
spec:
chart: ceph-csi-rbd
version: 3.9.0
version: REPLACE_HELM_CHART_VERSION
sourceRef:
kind: HelmRepository
name: stx-platform