The original legacy restore steps encountered issues when OAM IP was
issued with a short lease duration. This should prevent that from
happening by using the static IPs from back up.
TEST PLAN
PASS: AIO-DX B&R bootstrap
PASS: AIO-DX B&R, IPv4 with static OAM
Closes-Bug: https://bugs.launchpad.net/starlingx/+bug/2100786
Change-Id: I3031cf9c70b26bee2b7bec733aa0e776627c258e
Signed-off-by: Joshua Kraitberg <joshua.kraitberg@windriver.com>
Kubernetes 1.30.6 will be the new default version for new
StarlingX installs while we continue to support K8s 1.29.2
for upgrades.
Test Plan:
PASS: Fresh install on AIO-SX and AIO-DX. Check if active
K8s version is 1.30.6 and all pods are running as
expected.
PASS: Basic operations like lock, unlock and swact were
performed on both simplex and duplex installs.
PASS: K8s upgrade from v1.29.2 to v1.30.6.
Story: 2011047
Task: 51729
Change-Id: I7fc3e7cdc551b596365fe3cd2cbd18eda55793ff
Signed-off-by: Ramesh Kumar Sivanandam <rameshkumar.sivanandam@windriver.com>
Create custom keystone roles operator,configurator during the
bootstrap so that users can be assigned to these roles and also
access control policies can be implemented based on these roles.
Also add these roles to manage_local_ldap_account playbook to
simplify the user creation with these roles.
Test Cases:
PASS: Deploy standalone system, after bootstrap verify that
operator,configurator roles list under "openstack role list"
PASS: Deploy a DC system, after systemcontroller is bootstrapped
verify that operator,configurator roles list under
"openstack role list"
PASS: In DC, bootstrap a subcloud,verify that operator,
configurator roles list under "openstack role list"
PASS: Create the user with 'operator' role using
manage_local_ldap_account playbook
PASS: Delete the user with 'operator' role using
manage_local_ldap_account playbook
PASS: Create the user with 'configurator' role using
manage_local_ldap_account playbook
PASS: Delete the user with 'configurator' role using
manage_local_ldap_account playbook
Story: 2011348
Task: 51701
Change-Id: I29bd6e36ace55730ef8439a88326fb2f4e13899e
Signed-off-by: amantri <ayyappa.mantri@windriver.com>
This reverts commit 7b1c94793cb3dc2a4f60532be9263795321f7a27.
Reason for revert: reverting this change as keystone usage on the platform is only restricted to the admin project. There is no support for multiple projects / multi-tenancy
Change-Id: I02d0400396200a2224284f300c7fbfbdfc005a50
Restoring software deployments was not working for RR patches.
This was because the reboot was in the wrong place.
I've taken this opportunity to clean up the entire role, improving it's
overall structure and error messaging.
TEST PLAN
PASS: B&R with RR patch
PASS: B&R with RR patches
PASS: B&R with NRR patch
PASS: B&R with RR patch but manually patched
* Should fail because ostree doesn't match
PASS: B&R with RR patch (exclude_sw_deployments=true) manually patched
* sw deployments section should be skipped, no error
PASS: B&R with patches and different ISO
* Should fail because ostree doesn't match
Closes-Bug: https://bugs.launchpad.net/starlingx/+bug/2100014
Change-Id: Idf0cfa118aea08e7649fd84fc080838638aa5d93
Signed-off-by: Joshua Kraitberg <joshua.kraitberg@windriver.com>
An issue was noticed that during during legacy restore, if the
registries URLs contained in the backup matched the default registries
used by ansible, ansible would use it's defaults instead of the values
from the backup. This issue was also reproducible during bootstrap.
This change forces the restore playbook to always
use the values from the backup.
Another thing, no_log was added and some debug steps were removed to
stop password leaks.
TEST PLAN
PASS: AIO-DX bootstrap/deploy with default registries
PASS: AIO-DX bootstrap/deploy with private registries
PASS: AIO-DX legacy restore with default registries
PASS: AIO-DX legacy restore with private registries
PASS: AIO-DX legacy restore with default registries
* Change some registry values with restore_overrides
* Values in restore_overrides should take precedence
PASS: AIO-DX legacy restore with private registries
* Change some registry values with restore_overrides
* Values in restore_overrides should take precedence
PASS: Verify no password are leaked in logs
* Bootstrap
* Restore
* Subcloud enroll
Closes-Bug: https://bugs.launchpad.net/starlingx/+bug/2100002
Change-Id: I82b85c0d2e92bf7d2e74c60068e565583c2c64e2
Signed-off-by: Joshua Kraitberg <joshua.kraitberg@windriver.com>
Move FluxCD deployment from a pure Ansible approach to a Helm-based
method. This will ensure tighter control over StarlingX required
customizations via Helm overrides and make FluxCD uprev tasks simpler.
This leverages the new flux2-charts package introduced under the
starlingx/integ repository.
Test plan:
PASS: SX fresh install
PASS: DX fresh install
PASS: SX platform upgrade from previous version
PASS: DX platform upgrade from previous version
PASS: SX backup and restore
PASS: DX backup and restore
Story: 2011354
Task: 51690
DependsOn: https://review.opendev.org/c/starlingx/integ/+/941731
Change-Id: Ib7bbe723cfb76bc8f2695e16bbfedd2f95232c30
Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
In this commit, added the code to set permissions to 600
for all .crt files in /etc/kubernetes/pki directory.
TEST CASES:
PASSED: Run full build, system install, bootstrap and unlock(SX)
PASSED: System install, bootstrap, unlock and swact (DX)
PASSED: Checked permission using below command
"ls -al /etc/kubernetes/pki/*.crt"
PASSED: Checked whether certificates are accessible and readable
Example:
openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text
PASSED: Checked status of kubernetes cluster and pods.
PASSED: No alarms when ran "fm alarm-list"
Story: 2011334
Task: 51618
Change-Id: Id6c3f8e09f6c41fd9a2ecf7ad772955b2d58818b
Signed-off-by: sshaikh1 <sirin.shaikh@windriver.com>
When rook-ceph recovery is performed on a system that uses
an IPv6 network, the clusterIP is not parsed correctly,
causing communication failures and affecting the cluster's
recovery.
To resolve this, the IP family (IPv4 or IPv6) of the
clusterIP is checked, and when it is IPv6, the necessary
adjustments are made in relation to the brackets.
Additionally, a final ceph status check has also been added.
Because depending on the system configuration and the status
of the pods during recovery, MDS failures may occur, which
will prevent ceph from becoming HEALTH_OK. Therefore, it is
checked whether the ceph status is 'mds daemon damaged' or
'filesystem is degraded'. If so, the cephfs recovery process
is run again to fix this.
A task has also been added to the playbook to check
health of Kubernetes before starting rook-ceph recovery.
Test Plan:
- PASS: optimized SX B&R with IPv4
- PASS: legacy DX B&R with IPv4
- PASS: legacy DX B&R with IPv6
Closes-Bug: 2098747
Change-Id: Icea44a66e01e2994d14baca135f9e0115f5f6f7e
Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
Ensure dnsmasq restart completes before proceeding with subsequent
playbook tasks during rehoming execution. The OAM network creation
restarts dnsmasq asynchronously when the runtime manifest is applied.
Depending on the timing, the next task in the rehoming playbook may
execute before, during, or after the dnsmasq restart. This commit
ensures that dnsmasq has fully restarted and the system is ready to
execute commands before continuing with the next steps.
Test Plan
=========
[PASS] Execute rehoming to a different system controller using the
rehoming playbook. It should not fail. Check if the entries for
system-controller-oam-subnet and system-controller-subnet have been
correctly updated in the addrpool table of the subcloud.
Closes-Bug: 2098676
Change-Id: Ied57652f507a97b36e934acccec1f2fc06f182d1
For manage_local_ldap_account playbook, there is no option to
specify the project name, the user is always created in the
"admin" project, this change adds an option "project_name"
to the playbook if specified otherwise "admin" is set as
project name
Test Cases:
PASS: Run playbook to create user with option "project_name" and
verify the user is created under the given project
PASS: Run playbook to create user without option "project_name" and
verify that user is created under "admin"
PASS: Run playbook to create user with option "project_name"
without that project existing on the system and expect it to
fail.
Closes-Bug: 2098116
Change-Id: I5f529e3784c3c9e584fe00cd8c0b9d9a1419fbfb
Signed-off-by: amantri <ayyappa.mantri@windriver.com>
When the prestage is executed regardless of whether
it is for-install or for-sw-deploy, upon completion
it should display the list of prestaged versions.
Currently, when prestage is for-install, the list
is displayed correctly. However, when the prestage
is for-sw-deploy, the list comes empty, not
providing the correct information.
This commit adds code to support both for-install
and for-sw-deploy cases to display the list of
prestaged versions correctly.
Test Plan:
PASS: - Bring up a DC with 24.09 with at least 1 subcloud
- Upload and deploy patch 24.09.1 System Controller
- Prestage the subcloud for-install
- Verify that the release is prestaged on the subcloud
- Verify that the prestaged version is shown in
dcmanager subcloud show output
PASS: - Prestage the subcloud for-sw-deploy
- Verify that the release is prestaged on the subcloud
- Verify that the prestaged version is shown in
dcmanager subcloud show output
Story: 2010676
Task: 51513
Change-Id: I6568a3437d151ce16e9f1b195ddc08b135a6ec00
Signed-off-by: Cristian Mondo <cristian.mondo@windriver.com>
This commit compares the system-local-ca issuer certificates
installed in the system controller and in the subcloud during
init enroll operation and fails the playbook if one of the
certificates are not the same between system controller and
subcloud.
Test Plan:
PASS: Factory install a subcloud without certificates and enroll
the subcloud in the system controller. Verify that the enroll init
playbook fails.
PASS: Factory install a subcloud without certificates. Remove any
previous stored keys in root user 'known_hosts'. Verify that the
subcloud key is added and the enroll fails due to a mismatch
between the certificates.
PASS: Factory install a subcloud with the same certificates as the
system controller. Remove any previous stored keys in root user
'known_hosts' and enroll the subcloud. Verify that the subcloud
enrollment completes without errors.
Closes-bug: 2094925
Depends-on: https://review.opendev.org/c/starlingx/distcloud/+/939503
Signed-off-by: Gustavo Pereira <gustavo.lyrapereira@windriver.com>
Change-Id: Ie6de5fe9272a47a3a8ae41e733d5437642c8368b
In some scenarios, the rook-ceph recovery job may get stuck
when recovering cephfs, this happens because there is no mgr
active at that time. Furthermore, when it gets stuck in this
part, there is a timeout that is exceeded, however the
playbook does not fail and the next task is executed as if
the rook-ceph recovery had been done successfully.
To fix this, mgr is scaled to replicas=1. As a result, it was
necessary to make some changes to the code to follow this new
flow. Regarding the timeout, it is now being checked and when
it occurs, the playbook will fail the "Recover rook-ceph" task.
Additionally, a return code check has been added to all ceph
commands to ensure they are executed correctly, and fail if
any command returns anything other than 0.
Test Plan:
- PASS: optimized AIO-SX B&R
- PASS: optimized AIO-SX B&R with wipe_ceph_osds flag
- PASS: legacy DX B&R
- PASS: legacy DX B&R with wipe_ceph_osds flag
- PASS: legacy STD B&R
Closes-Bug: 2097369
Change-Id: Ia0d6b63acfa36a6eb978f98d53d6a6ff06af7071
Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
Upversioned the cert-manager app to latest v1.16.3
to pick up the latest CVE fixes.
Test Cases:
PASS: Build the ISO, during bootstrap notice that the
cert-manager images of 1.15.3 are pulled and pushed
to local registry
Story: 2011330
Task: 51648
Change-Id: Idc7c1e0f295b13b437eaf16da0319e5e4b11dbb8
Signed-off-by: Eduardo Almeida <Eduardo.AlmeidadosSantos@windriver.com>
Ensure default start and end values for a network range cover the entire
available range. For IPv6, include the entire range, while for IPv4,
exclude the last address due to its use as a broadcast address.
Test Plan
=========
[PASS] Bootstrap dual-stack with IPv6 as primary address. Use the last
IPv6 address of the network for the external_oam_floating_address
parameter, and the second-to-last address for IPv4. Do not
specify the values for external_oam_end_address and
external_oam_start_address in the localhost.yml file, so that the
default values can be applied. The OAM address pool range must include
the last address of the network for IPv6, but must not include the last
address of the network for IPv4.
[PASS] Repeat the previous test by reversing the configuration,
setting IPv4 as the primary address and IPv6 as the secondary address.
[PASS] Bootstrap dual-stack with IPv6 as primary address. Apply default
values for cluster_pod_end_address, cluster_service_end_address,
admin_end_address, management_multicast_end_address. The address pool
range must include the last address of the network for IPv6, but must
not include the last address of the network for IPv4 for all networks.
[PASS] Run the subcloud enrollment process using a dual-stack
configuration. Use the last network address for IPv6 as the
admin_end_address and the default value for the
external_oam_end_address parameter. For IPv6, the last network address
must be assigned as the end address for both OAM and admin. For IPv4,
the second-to-last address in the network should be used as the end
address for all network types when not otherwise specified.
Closes-Bug: 2095524
Change-Id: If42795b2a88f5b8238fc116b067127b2bdf40376
Signed-off-by: Ferdinando Terada <Ferdinando.GodoyTerada@windriver.com>
Update ingress-nginx to helm-chart 4.12.0 to pick up new CVE changes.
This change updates these images:
1. ingress-nginx/controller: v1.11.1 -> v1.12.0
2. ingress-nginx/kube-webhook-certgen: v1.4.1 -> v1.5.0
3. ingress-nginx/opentelemetry: v20230721-3e2062ee5 -> REMOVED
The separate OpenTelemetry image was removed since OpenTelemetry is
already built into the NGINX image. Functionality remains unchanged.
Test Cases:
PASS: Build iso, notice during bootstrap, the correct
images are downloaded and pushed to local registry
Story: 2011329
Task: 51641
Change-Id: I0abed3eefadced8c18717722fc1a69d79b840607
Signed-off-by: Eduardo Almeida <Eduardo.AlmeidadosSantos@windriver.com>
Fix the following CIS Benchmark network configurations:
- 3.3.7 Ensure Reverse Path Filtering is enabled
Testing:
- Build successful
- SX and DX deployment successful
- Run CIS Tenable-IO scan with no errors
Story: 2011210
Task: 51626
Change-Id: I6d338c340d653cf51dcaa983f2cea32f8f02d2d5
Signed-off-by: Mohammad Issa <mohammad.issa@windriver.com>
This commit upversions the iso to 25.09 for this repo.
Test Plan:
PASS: Install DX
PASS: Build ISO, verify software version is updated accordingly
Story: 2010651
Task: 51615
Depends-On: https://review.opendev.org/c/starlingx/metal/+/940070
Change-Id: I5905d0af4f72a74687c616133e575eededed9aa8
Signed-off-by: Luis Eduardo Bonatti <luizeduardo.bonatti@windriver.com>
The application framework settings were split
across the ansible playbook and puppet manifest.
The initial changes https://review.opendev.org/c/starlingx/ansible-playbooks/+/913930https://review.opendev.org/c/starlingx/ansible-playbooks/+/938286
added the appframework variables in the playbook.
After further discussions, it was concluded that
these settings should be controlled by service
parameters and set by puppet at bootstrap,
allowing the framework behavior to be adjusted
at runtime. As a result, these settings should
remain in the puppet manifest and can be fully
removed from ansible playbook.
Conversation about the right way to handle this
question: https://review.opendev.org/c/starlingx/stx-puppet/+/938287/comments/ad300f57_d81ca92d
This commit removes the appframework variables
settings from ansible playbook.
Test plan:
PASS: build-pkgs && build-image
PASS: AIO-DX fresh install
PASS: check if sysinv.conf file has
missing_auto_update and
skip_k8s_application_audit appframework
variables set.
PASS: AIO-SX stx-10 install
PASS: upgrade from stx-10 to master
PASS: check if sysinv.conf file has
appframework variables set.
Story: 2011242
Task: 51544
Change-Id: I86aba3d8992a1acdd90e64556a805a672e893f85
Signed-off-by: Edson Dias <edson.dias@windriver.com>
During optimized restore, SM becomes activated after the flag
/etc/platform/.initial_config_complete is created. This causes services
to restart and bounce. On certain systems this leads to stuck/failed
services (dnsmasq). Preventing an unlock.
To fix this, SM is stopped. However, stopping SM also requires stopping
pmon.
TEST PLAN
PASS: AIO-SX restore
PASS: AIO-SX restore with wipe_ceph_osds=true
PASS: AIO-SX subcloud restore
Change-Id: Id273276e99d2626c341b16aeb292321b83eebdfb
Signed-off-by: Joshua Kraitberg <joshua.kraitberg@windriver.com>
Adding backup and restore for openbao. As openbao is a fork of hashicorp
vault, it uses identical procedure to make backup and restore itself.
The process used here is same as the one used for hashicorp vault. The
code is copied over to keep the code separated from the vault version.
This is to keep the work simple once vault backup and restore is
removed.
Test Plan:
PASS Openbao standalone backup is successful without errors
PASS Openbao platform backup is successful without errors
PASS Openbao standalone restore is successful without errors
Story: 2011244
Task: 51419
Change-Id: I49caf7f300563d511a5f609b40defd79420f110c
Signed-off-by: Tae Park <tae.park@windriver.com>
The current implementation only supports a specific version of
release N-1, preventing future release N from being prestaged
in subcloud with a previous release version.
This commit updates the condition so that the same logic can be
applied for any future versions of N-1 release.
Test Plan:
PASS - Bring up a DC with 24.09 with at least 1 N-1 subcloud
with USM supported.
- Upload and deploy the new N release on System Controller
- Prestage the subcloud with the new N release
- Verify that the release is prestaged on the subcloud
- Apply and verify the prestaged release on the subcloud.
Story: 2010676
Task: 51393
Change-Id: Id0022508bdc4094764811faad0465a1ea1af296b
Signed-off-by: Cristian Mondo <cristian.mondo@windriver.com>
After k8s upgrade from k8s 1.24.4 to 1.25.3+, sriov-cni daemonsets
are duplicated due to the name change upstream from sriov-cni 2.6.3
using "kube-sriov-cni-ds-amd64" to 2.8.1 using "kube-sriov-cni-ds".
This change would clean up the old sriov-cni daemonsets
before deploying the up-versioned daemonsets.
In this update:
- It will ignore the delete command if the daemonset is not found
- It will not wait for the delete command to return in case it hangs
- It will only execute the task when the desired upgrade
version is k8s v1.25.3 (which is when the sriov-cni version changes)
Test Plan:
- SX and DX deployed successfully
- K8s upgrade from v1.24.4 to v1.29.2 successfully
- Ensure old sriov daemonset is removed before deploying the updated one
- Ensure this task only executes when upgrading to v1.25.3
- Ensure this task is skipped if the old dameonset does not exist
Closes-Bug: 2095037
Change-Id: I72251a9ba1ea47e7bda9afd6255b6f97c42d660d
Signed-off-by: Mohammad Issa <mohammad.issa@windriver.com>
After k8s upgrade from k8s 1.24.4 to 1.29.2, sriov-cni daemonsets
are duplicated due to the name change upstream from sriov-cni 2.6.3
using "kube-sriov-cni-ds-amd64" to 2.8.1 using "kube-sriov-cni-ds".
This change would clean up the old sriov-cni daemonsets
before deploying the up-versioned daemonsets.
Closes-Bug: 2095037
Change-Id: I26813be387cce725b01de30439be46b73190346c
Signed-off-by: Mohammad Issa <mohammad.issa@windriver.com>
This commits allows dcmanager to specify metadata sync
options over remote install as needed.
Test Plan:
PASS - Verify successful remote install of an N subcloud
PASS - Verify successful remote install of an N-1 subcloud
PASS - Verify a successful restore of an N-1 subcloud with install
PASS - Update dcmanager to generate install overwrite file with set
sync_patch_metadata and sync_software_metadta set to false.
Verify that the patch metadata and content are not synced as
expected.
Note: the test of restore of N subcloud with install was blocked
due to https://bugs.launchpad.net/starlingx/+bug/2093121
Story: 2010676
Task: 51536
Change-Id: I478ec3e2a95895c3dea6592dc262dc430e8e3aaf
Signed-off-by: Cristian Mondo <cristian.mondo@windriver.com>
