The change [1] updates the version of the libpython2.7-minimal,
libpython2.7-stdlib, python2.7 and python2.7-minimal packages, which
caused the build for the stx-openstackclients image to fail, as it
requires the `python-dev` package [2].
Since Python 2 is not used by any Openstack project anymore, this
package wasn't doing anything, so it's safe to upgrade it to the Python
3 version of the same package. This not only solves the build issue but
keeps the image coherent due to the reason mentioned in the beginning of
this paragraph.
[1] 55eebf2d6a
[2] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/upstream/openstack/python-openstackclient/debian/stx-openstackclient.stable_docker_image#L17
Test Plan:
PASS: Build stx-debian base image
PASS: Build stx-openstackclients image
PASS: Upload / apply stx-openstack
PASS: Manually upload built image to the system's registry and perform
helm-override to use it in the clients containers
PASS: Issue Openstack commands for all CLIs and verify they're working:
Cinder, Glance, Heat, Neutron, Nova and Openstack
Closes-Bug: 2043390
Change-Id: Id8d3b3572eefd7e8b0cdbd4b8c3129728647ede7
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
Since we started using Openstack Antelope, it was noticed that, when
creating backups using the Openstack CLI and specifying the `--location`
parameter, the backups would always fail with the error message:
"Backup driver reported an error: Multidriver is set but a backup
driver wasn't provided." This error was caused because, even though
patch 0002 introduced the `--location` parameter to the CLI, it was not
passing it through to the Cinder API.
This change fixes that by passing the location parameter on the backup
creation, when using the Openstack CLI.
Test Plan:
PASS: Build stx-openstackclients image
PASS: Upload / apply stx-openstack
PASS: Manually upload built image to the system's registry and perform
helm-override to use it in the clients containers
PASS: Create a Cinder volume, and boot a VM from it
PASS: Create a backup using the Openstack CLI, and passing the
`--location` parameter
PASS: Remove / delete stx-openstack
Closes-Bug: 2043383
Change-Id: Ie6f8756e3c647d2191489e298e3ea138f32c30d4
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
OpenStack 2023.1 (Antelope) introduced this issue [1][2] when
live-migrating instances, in which Nova uses getCapabilities() to
determine the host CPU model but use the model from the
domCapabilities for the guest VM using host-model [3].
According to the libvirt maintainers nova should never use
getCapabilities for anything any more.
A solution for this issue is being developed in main branch [4], but
taken as a medium priority since there is a workaround config option
already available [5] to avoid this situation.
Setting "skip_cpu_compare_on_dest" to True will, during live
migration, skip comparing guest CPU with the destination host.
When using QEMU >= 2.9 and libvirt >= 4.4.0, libvirt will do the
correct thing with respect to checking CPU compatibility on the
destination host during live migration.
StarlingX currently delivers QEMU 5.2 and stx-openstack uses a libvirt
on 7.0.0-3, therefore this can be safely used as part of our default
nova configuration updates.
[1] https://bugs.launchpad.net/nova/+bug/2023035
[2] https://bugs.launchpad.net/nova/+bug/2039803
[3] https://review.opendev.org/q/topic:fix_compareCPU_usage
[4] https://review.opendev.org/c/openstack/nova/+/899185
[5] https://docs.openstack.org/nova/latest/configuration/config.html#skip_cpu_compare_on_dest
Closes-bug: 2007303
TEST PLAN:
PASS - Build python3-k8sapp-openstack plugins
PASS - Build stx-openstack application
PASS - Upload/Apply/Remove stx-openstack
PASS - Live-migrate an instance
PASS - Manually reboot a compute node in which
an instance is running to ensure it is
correctly evacuated to another compute
Change-Id: Id7a93445af4115ee81b035b4f9dc7a6eb889555b
Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
After upversioning openstack-helm and openstack-helm-infra, update
versions of helm charts introduced new docker images that are not used
by our stx-openstack distro but would still be downloaded during an
application apply if the helm values are not set to null.
This change removes unnecessary neutron-ovn and
libvirtd-export images.
Closes-Bug: 2043037
Test Plan:
PASS - Build stx-openstack-helm-fluxcd
PASS - Build stx-openstack application tarball
PASS - Upload/Apply/Remove stx-openstack application
Change-Id: I36bbde03785ad11b0cf97a9cf3e357ebd8084175
Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
This change adds `python3-osc-placement` to the `stx-openstackclients`
container image so that the end user has access to CLIs provided by
Placement, e.g.:
$ openstack resource provider ...
As mentioned in LP-2042482, these CLIs can be used to compensate for
a change in the `openstack hypervisor` command that cause `vcpus_used`
and `free_disk_gb` fields to no longer be returned.
These are now only accessible via:
$ openstack resource provider inventory list
$ openstack resource provider usage show
Test Plan:
PASS - Build stx-debian base image
PASS - Build stx-openstackclients container image
PASS - Upload/apply stx-openstack
PASS - Upload built image to the system's registry and perform
helm-override to use it in the clients container
PASS - Verify that the CLIs are accessible:
$ openstack resource provider -h
PASS - Remove/delete stx-openstack
Depends-On: https://review.opendev.org/c/starlingx/tools/+/899898
Closes-Bug: 2042482
Change-Id: I1c72b135e7ff662b83de9eac1ec5c049943017f2
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
The management network is used extensively for all internal
communication.
Since the original use of the network was a private network before
it was exposed for external communication in a distributed cloud
configuration, it was never designed to be reconfigured.
To support MGMT network reconfiguration the idea is to configure the
applications to use the hostname/FQDN instead of a static MGMT IP
address.
In this way the MGMT network can be changed and the services and
applications will still work since they are using the hostname/FQDN
and the DNS will be responsible to translate to the current MGMT
IP address.
The use of FQDN will be applied for all installation modes: AIO-SX,
AIO-DX, Standard, AIO-PLUS and DC subclouds. But given the
complexities of supporting the multi-host reconfiguration,
the MGMT network reconfiguration will focus on support for AIO-SX
only.
The DNSMASQ service must start as soon as possible to translate
the FQDN to IP address.
Test plan ( Debian only )
- AIO-SX and AIO-DX virtualbox installation IPv4/IPv6
- Standard virtualbox installation IPv6
- DC virtualbox installation IPv4 ( AIO-SX/DX subclouds )
- AIO-SX and AIO-DX installation IPv4/IPv6
- AIO-DX plus installation IPv6
- DC IPv6 and subcloud AIO-SX
- AIO-DX host-swact
- DC IPv4 virtualbox with subcloud AIO-DX and AIO-DX
- AIO-SX to AIO-DX migration
- netstat -tupl ( no services are using the MGMT IP address )
- Ran sanity/regression tests
- Openstack sanity/regression tests
Story: 2010722
Task: 48467
Depends-On: https://review.opendev.org/c/starlingx/config/+/886208
Change-Id: I8ce844977156f56edb861446640a0c3c6f4a8007
Signed-off-by: Fabiano Correa Mercer <fabiano.correamercer@windriver.com>
The security note OSSN-0090 [1] from Openstack describes a
configuration that makes it possible to open some known attack vectors
by which malicious data modification can occur.
The vulnerability only occurs if one of the following parameters are
set to True in Glance's configuration: `show_multiple_locations` or
`show_image_direct_url`. In the case of the STX-Openstack app, the
`show_image_direct_url` was being set to True in the app's plugin [2].
It looks like this configuration was just transported from when the
app's plugin was dettached from sysinv [3], which itself, was based on
an even older commit [4]. Unfortunately, the commit message of [4] has
no explanation on why this was required.
This commit changes the app's plugin to define both the
`show_image_direct_url` and the `show_multiple_locations` parameters to
`False`, since we can easily avoid the security issue described in [1]
by doing that, and it does not impact on the applications
functionalities, as seen in the test plan for this change.
[1] https://wiki.openstack.org/wiki/OSSN/OSSN-0090
[2] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/glance.py#L155
[3] https://review.opendev.org/c/starlingx/openstack-armada-app/+/688190
[4] https://review.opendev.org/c/starlingx/config/+/611948
Test Plan:
PASS: Build python3-k8sapp-openstack and stx-openstack-helmf-fluxcd
packages
PASS: Build STX-Openstack helm charts
PASS: Upload / apply / remove STX-Openstack app
PASS: Inspection of /etc/glance/glance-api.conf shows that both the
show_multiple_locations and show_image_direct_url parameters are
set to `False`
Using OpenStack's CLI:
PASS: Image commands are executed as expected: `openstack image
list`, `openstack image show`
PASS: Create an image
PASS: Delete an image
PASS: Create a bootable volume from an image
PASS: Boot a VM using an image as the source
PASS: Boot a VM using the bootable volume as a source
PASS: Delete both VMs and the volume
Using Horizon dashboard:
PASS: All tenants are able to see and inspect images
PASS: Create an image
PASS: Delete an image
PASS: Create a bootable volume from an image
PASS: Boot a VM using an image as the source
PASS: Boot a VM using the bootable volume as a source
PASS: Delete both VMs and the volume
Closes-Bug: 1993606
Change-Id: I20f241224234353363c632085e3e41b91f97abf5
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
As a side-effect of the migration from the migration
from helm v2 to v3 and Armada to FluxCD, the helm
override of the horizon rbac policy was failing to
apply this file because it was too large. By splitting
this file and making two helm overrides the apply can
finish successfully.
Test plan:
PASS: system helm-override-update stx-openstack horizon openstack --reuse-values --values=rbac/horizon-policy-overrides.yml
system helm-override-update stx-openstack horizon openstack --reuse-values --values=rbac/horizon-nova-policy-overrides.yml
PASS: Reapply the app and check the helm overrides succeed
Closes bug: 2040165
Change-Id: Ib1c82544cd1f2335554f740bc3fe733ce57370ab
Signed-off-by: Romulo Leite <romulo.leite@windriver.com>
After STX-Openstack upversioned to Antelope, we noticed that it was not
possible to create VMs by volume, as they would be stuck on ERROR
status. The first proposed solution was to create a patch containing
[1] and [2], because, as specified in [3], Nova now requires
a service token in order to be able to manipulate Cinder volumes. This
unfortunately did not solve the issue by itself, as now an error message
showed up on the nova-conductor pods with the following (not full error
message, only important part): "nova.exception.RescheduledException:
Build of instance 2f32c7ea-1720-4f61-bce8-dbe970c40b0c was re-scheduled:
Secret not found: no secret with matching uuid 'a7f3ae2e-cee7-4f04-9402
-a78047747654". This UUID was not the same one present when issuing
`virsh secret-list` on Cinder, Nova and Libvirt containers.
Turns out openstack-helm and openstack-helm-infra have a Ceph UUID
hardcoded in them, in Cinder [4], Nova [5] [6] and Libvirt [7] values.
By changing these values to the UUID that libvirt was trying to find
(7f3ae2e-cee7-4f04-9402-a78047747654), and it worked to solve the issue.
However, it is not a good practice to use hardcoded values, and,
searching on where this UUID was coming from, it turns out it was
defined by the platform's Ceph configuration under
`/etc/ceph/ceph.conf`.
This still leaves the question, why was this working on Ussuri and
stopped working on Antelope? First of all, the Ceph official
documentation [8] [9] about using it with OpenStack explains the
process of adding the secret to libvirt, to store the ceph admin
keyring. You can see that the secret uuid is generated "on the fly" and
both docs mention that old/hard-coded value
(i.e., 457eb676-33da-42ec-9a8c-9293d545c337). This is the reason why it
used to work until our upversion to OpenStack Antelope/2023.1: this
UUID does not really matter (as long as nova and libvirt have the same
value for it)! It is a given UUID to the libvirt secret that will store
ceph keyring [10], and the key ring will ensure proper communication
between our services and the platform ceph.
What changed between Ussuri and Antelope (2023.1), is that now there is
a specific method [11] to set a default value (Ceph's Cluster UUID) for
this UUID when it is not specified in the driver configuration.
What this change does is dynamically read this `/etc/ceph/ceph.conf`
file to search for the UUID value, and use it to override the [4] [5]
[6] and [7] values. It also adds the patch including the Nova service
token configuration. The combination of these 2 changes allows VMs to be
created by volumes.
[1] 91c8a5baf2
[2] 7d39af25fd
[3] https://docs.openstack.org/releasenotes/cinder/2023.1.html#upgrade-notes
[4] https://opendev.org/openstack/openstack-helm/src/branch/master/cinder/values.yaml#L942
[5] https://opendev.org/openstack/openstack-helm/src/branch/master/nova/values.yaml#L594
[6] https://opendev.org/openstack/openstack-helm/src/branch/master/nova/values.yaml#L1432
[7] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/libvirt/values.yaml#L100
[8] https://github.com/ceph/ceph/blob/main/doc/rbd/rbd-openstack.rst
[9] https://docs.huihoo.com/ceph/v0.80.5/rbd/rbd-openstack/index.html
[10] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/libvirt.py#L60
[11] 6464d37d16 (diff-9b122c182b4333b747e7fd7e07f73d68ff30256a)
Test Plan:
PASS: Build openstack-helm, python3-k8sapp-openstack and
stx-openstack-helm-fluxcd
PASS: Upload / apply / remove STX-Openstack
PASS: Create a VM by an image
PASS: Create a volume and launch a VM from it
PASS: Create a VM using the `boot-from-volume` flag
PASS: Delete a VM created by a volume
Closes-Bug: 2037463
Change-Id: Ia00bb8dbe3460ce817d69049f97f56a96ad6a298
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
In [1], two OpenStack clients were moved to the `DIST_PACKAGES` section
of the `stx-openstackclients` container image build recipe due to the
need to use packages patched exclusively for StarlingX:
* python3-cinderclient;
* python3-openstackclient.
Although this change served its purpose, it had a side-effect that
caused the main CLI, `openstack`, to stop recognizing other clients
installed via `PIP_PACKAGES`.
This occurred because once installed via `DIST_PACKAGES`, both
`cinderclient` and `openstackclient` started to reside in the
dist-packages/ directory, while all other clients continued to reside in
site-packages/. As `openstackclient` is the one responsible for the main
CLI, it stopped finding the other clients because its search directory
changed to dist-packages/.
Therefore, in this change, we chose to move all OpenStack clients to
DIST_PACKAGES. This way, it is guaranteed that the main CLI will
correctly identify the other OpenStack clients.
[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/896129
Test Plan:
PASS - Build stx-debian base image
PASS - Build stx-openstackclients container image
PASS - Upload/apply stx-openstack
PASS - Upload built image to the system's local registry and perform
a helm-override to use it in the OpenStack clients container
PASS - Run commands that had disappeared since the merge of [1], e.g.,
the ones provided by `heatclient`:
* openstack stack list
* openstack stack create
...
PASS - Remove/delete stx-openstack
Closes-Bug: 2039096
Co-authored-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
Co-authored-by: Romulo Leite <romulo.leite@windriver.com>
Change-Id: I61d653ceef07f99568a21b4ad6f6dceffed22ca1
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
(cherry picked from commit 5bcbab406a)
After introducing a new set of layer-specific `sources.list` files in
[1], we can now update the `DIST_REPOS` of this image to additionally
use the `openstack` layer binary repository when searching for
dependencies of packages listed in `DIST_PACKAGES`.
This ensures that OpenStack Antelope packages will be built *without* OpenStack Victoria dependencies.
[1] https://review.opendev.org/c/starlingx/root/+/896277
Test Plan:
PASS - Build `stx-debian` base image
PASS - Build `stx-openstackclients` container image and confirm that
OpenStack Antelope dependencies were used in the build of
`python3-cinderclient` and `python3-openstackclient`
Depends-On: https://review.opendev.org/c/starlingx/root/+/897944
Closes-Bug: 2037339
Change-Id: If94d5281c5c2633b59a6aaf39f0881fb5cec61bb
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
(cherry picked from commit 418a7bcb6e)
Ever since we migrated to containerized clients, it has been observed
that two clients -- `cinderclient` and `openstackclient` -- have lost an
additional parameter that was added by patches [1] and [2]:
`--location`.
Apparently, this is happening because the build of this container image
is prioritizing the download of indexed packages rather than the use of
the packages we deliver within the wheels tarball (which are patched
with modifications specific to StarlingX).
According to [3], when we specify an extra source of wheels for `pip`
with `--find-links` -- which is what `openstack/loci` does [4] -- it
does not mean that `pip` will necessarily prioritize this source over
others, such as PyPi.
Therefore, to ensure that our patched clients are used in the container
image, we moved them from `PIP_PACKAGES` to `DIST_PACKAGES` in this
change.
Note: The other clients can continue to be installed from indexed
packages. For this reason, they remain in `PIP_PACKAGES`.
[1] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/upstream/openstack/python-cinderclient/debian/patches/0001-Add-location-parameter-for-volume-backup-creation.patch
[2] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/upstream/openstack/python-openstackclient/debian/patches/0002-Add-location-parameter-for-volume-backup-creation.patch
[3] https://github.com/pypa/pip/issues/9959
[4] efccd0a853/scripts/pip_install.sh (L7)
Test Plan:
PASS - Build `stx-openstackclients` container image
PASS - Run image and confirm that both clients have the --location
parameter:
1. docker run -it <user>/stx-openstackclients:<tag> bash
2. openstack volume backup create -h
3. cinder help backup-create
Closes-Bug: 2036986
Co-Authored-By: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
Co-Authored-By: Romulo Leite <romulo.leite@windriver.com>
Change-Id: I4eeece7fc4a98b5254ca9367ca85c14ef5df7f3c
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
(cherry picked from commit e25852370e)
The purpose of this change is to remove packages that are not used by
STX-Openstack from this repository. At the time these folders were
copied from the `starlingx/upstream` repo to here on [1], the goal was
to simply get the folder structure and all the packages, but, now that
the this repository is being actively used for the STX-Openstack
upversion to Antelope, we noticed that these packages are either not
used (barbican, keystone, openstack-resource-agents, horizon,
python-osc-lib, python-oslo.messaging, python-wsme, rabbitmq-server),
or were dropped from Openstack (gnocchi and the python-gnocchiclient
are not present on Antelope, as these services were removed from the
Openstack project [2] [3]).
Except for gnocchi and python-gnocchiclient, as explained above, these packages were present on the `starlingx/upstream` repository as they're required for the StarlingX builds and ISO, but not for the STX-Openstack application, for that reason, these are safe to be dropped from this repository.
[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/886027
[2] https://opendev.org/openstack/gnocchi
[3] https://opendev.org/openstack/python-gnocchiclient
Test Plan:
PASS: Removed packages are not found when issuing `build-pkgs`
PASS: Build all STX-Openstack images
PASS: Build STX-Openstack tarball
PASS: Upload / apply / remove STX-Openstack
Related-Bug: 2027589
Change-Id: I1697bce6b606fb603a0cb4a2c35b6255e129a080
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
This change updates the application plugins in order to ensure that all
libvirt/live-migration related traffic is happening through the
cluster-host-network. Currently most of the libvirt/live-migration
addresses are being solved through INADDR_ANY (0.0.0.0), and this route
resolution will vary between AIO, routes to oam-network, and Worker,
routes to mgmt-network. Both resolutions are not correct since the
correct network for such traffic should be the cluster-host-network.
Actually, current platform firewall will block any traffic through not
allowed oam-network ports.
The goal will be achieved by setting to the node's cluster-host IP:
* libvirt listen_addr
* nova.conf "live_migration_inbound_addr"
It is important to notice that in the current version of the
openstack-helm nova helm chart, there is a problem with
nova-compute-init.sh for this use case of ours, so an openstack-helm
patch was required to fix it.
Code that was previously implemented only for the Nova plugin and is now
required by the Libvirt plugin, was moved to the parent OpenStack class.
[1] 31be86079d
TEST PLAN:
PASS - Build stx-openstack application
PASS - Apply the application to an AIO-DX system
PASS - "$ sudo netstat -ltnp | grep <libvirtd pid>" to ensure that
libvirtd is listening on the correct cluster-host-net IP
PASS - Verify that the nova-compute.sh script was populated correctly
PASS - Test a VM live-migration on the controller+worker node
PASS - Verify that live_migration data in LibvirtLiveMigrateData has the
correct cluster-host-net IP address in its "target_connect_addr"
PASS - Apply the application to a Standard system
PASS - "$ sudo netstat -ltnp | grep <libvirtd pid>" to ensure that
libvirtd is listening on the correct cluster-host-net IP
PASS - Verify that the nova-compute.sh script was populated correctly
PASS - Test a VM live-migration on the worker node
PASS - Verify that live_migration data in LibvirtLiveMigrateData has the
correct cluster-host-net IP address in its "target_connect_addr"
Closes-Bug: 2037330
Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
Change-Id: I37db601e4b1b0e397a1b8dbdad1a293ff25c2e55
After the introdution of [1], it was noted that Helm overrides were not
working properly for STX-Openstack. Althought they show up on
helm-override-show, they we're not being passed to Helm and Kubernetes.
This was caused because the [1] change updated the way that the
overrides are applied, requiring the `get_namespace` function on the
plugin to define where to apply these overrides, and, if this function
did not exist, it would default to the `default` namespace.
This change fixes that by adding the `get_namespace` function to the
OpenstackBaseHelm plugin, which is extended by the other STX-O plugins.
This change also does the following:
1) Updates the base class for ingress, memcached and garbd from
BaseHelm to OpenstackBaseHelm, as openstack.BaseHelm is the old base
class for the armada app and probably should be cleaned now that we
no longer support Armada.
2) Remove ingress customizations for the kube-system namespace, as
they're not used anymore and can be removed without any impact.
[1] https://review.opendev.org/c/starlingx/config/+/887430
Test Plan:
PASS: Build python3-k8sapp-openstack and stx-openstack-helm-fluxcd
PASS: Build STX-Openstack tarball
PASS: Upload / apply STX-Openstack on an AIO-SX machine with HTTPS
enabled
PASS: Verify that there's only one ingress pod per component
PASS: After passing helm overrides, verify that they're applied
(Example: change number of pods from 1 to 2)
PEND: Confirm overrides work for every Openstack component
Closes-Bug: 2036633
Change-Id: Id01a7367335213015531e1e9186b2b1cc677ef8e
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
When using container-backed OpenStack CLIs, it has been observed that
commands with multi-word options are failing to run due to poor parsing
of parameters by the wrapper script.
Currently, all commands that are redirected to the clients container are
expanded using the * parameter, i.e., $* [1]. However, according to [2],
a more appropriate option would be to expand the * parameter with the
Q operator, since:
> The expansion is a string that is the value of _parameter_ quoted in
a format that can be reused as input.
Which is exactly what we are doing with the wrapper script -- reusing
inputs but in a containerized environment.
[1] 4c238a8063/stx-openstack-helm-fluxcd/stx-openstack-helm-fluxcd/helm-charts/clients/templates/bin/_clients-wrapper.sh.tpl (L63-L64)
[2] https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
Test Plan:
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
PASS - Upload/apply stx-openstack
PASS - Source `/var/opt/openstack/admin-openrc`
PASS - Run command with multi-word option(s), e.g.:
$ openstack project create \
--description="words separated by space" my_project
and confirm that it's executed successfully
PASS - Remove/delete stx-openstack
Closes-Bug: 2036447
Change-Id: I6441cf077e0bcfe386d7086ea90298e6dd247a2a
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
There is no need of including stx-openstack-helm-fluxcd package to
StarlingX ISO, since this package is used only by the script that
generates the application tarball and this application is not delivered
along with the platform by default.
Having this package installed has no side-effects, but it installs
unnecessary files to /usr/lib on application, fluxcd and helm
directories.
TEST PLAN:
PASS - Successfully build a StarlingX ISO
Closes-Bug: 2028959
Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
Change-Id: I342266c0932bb7853aa5a377b3a5663677858b27
It was observed that, when using the STX-Openstack app with HTTPS
enabled, it was not possible to visualize / edit / create images using
Horizon. This was caused because the SSL certificate (ca.crt) volume
mount path defined in Horizon's configuration [1] was not the same as
the default one created when using the Helm toolkit [2].
As specified in [3], one possible solution was to define a path to the
TLS secret to be created, but that caused other services in Horizon to
fail, so instead of doing that, the solution was to change Horizon's
configuration to search the ca.crt file in the correct path.
[1] https://opendev.org/openstack/openstack-helm/src/branch/master/horizon/values.yaml#L469
[2] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/helm-toolkit/templates/snippets/_tls_volume_mount.tpl#L56
[3] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/helm-toolkit/templates/snippets/_tls_volume_mount.tpl#L30
Test Plan:
PASS: Build openstack-helm and stx-openstack-helm-fluxcd
PASS: Build STX-Openstack tarball
PASS: Upload / apply STX-Openstack with HTTPS enabled
PASS: Create an image by using the Glance CLI
PASS: On Horizon, under Project -> Compute -> Images, verify that the
image is present
PASS: Create an image via Horizon
Closes-Bug: 2032980
Change-Id: If34c0d4a287804e4fce0e5f70bb97feb330be174
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
After the recent upversion of openstack-helm [1], it has been observed
that the `cinder-volume-usage-audit` pod is having problems booting on
systems with HTTPS enabled due to a misconfigured TLS-related
volume/volumeMount pair.
Apparently, this pair of volume and volumeMount was introduced with the
upversion of openstack-helm, and ended up being left out of the changes
made by patch `0010-Remove-TLS-from-openstack-services.patch` that, in
theory, would have solved the problem.
Therefore, this change aims to update the patch in question -- along
with any other patches to avoid conflicts -- so that the
`cinder-volume-usage-audit` pod no longer has problems booting on
systems with HTTPS enabled.
[1] 8254cd31bb
Test Plan (on AIO-DX with HTTPS enabled):
PASS - Build openstack-helm package
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
PASS - Upload/apply stx-openstack
PASS - Verify that all pods -- including `cinder-volume-usage-audit` --
are either "Running" or "Completed"
PASS - Remove/delete stx-openstack
Closes-Bug: 2032703
Change-Id: Ic13c6945cc9e43f9153820297e74623520446fcd
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
With the [1] change, the aodh client was added to the
stx-openstackclients image, and with that the scripts to setup and clear
the aliases to use the containerized clients have to be updated.
Although not used this change is also adding the barbican and swift
clients to this list, along with the neutron client, which will be
deprecated in the near future [2] (check note under contribution guide)
[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/891553
[2] https://docs.openstack.org/python-neutronclient/2023.1/
Test Plan:
PASS: Build stx-openstack-helm-fluxcd package
PASS: Build STX-Openstack tarball
PASS: Upload / apply STX-Openstack
PASS: With Helm overrides pointing to the stx-openstackclients image
generated with the [1] change, make sure the aodh, barbican,
neutron and swift clients are accessible
Story: 2010715
Task: 48137
Change-Id: Ied0e7b271d647da0b44a1913f4cc33cc04856d0b
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
It has been observed that, on Standard systems, users are unable to
interact with containerized clients due to the fact that the `clients`
pods are being initialized without the `controller-` suffix in their
names. These names are relevant for the clients wrapper script to be
able to find and execute commands inside the clients containers [1].
The reason pods are being initialized with wrong names on Standard
systems is because the host-specific overrides are being generated for
hosts with `WORKER` personality instead of `CONTROLLER`, which makes no
sense as the containerized clients were designed to *only* run on
controller nodes.
Therefore, the proposed fix is straightforward and only requires us to
change the host personality required to generate host-specific
overrides.
[1] 4c238a8063/stx-openstack-helm-fluxcd/stx-openstack-helm-fluxcd/helm-charts/clients/templates/bin/_clients-wrapper.sh.tpl (L52-L55)
Test Plan (on Standard):
PASS - Build python3-k8sapp-openstack package
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
PASS - Upload/apply stx-openstack
PASS - Source `/var/opt/openstack/admin-openrc` and check that the
containerized clients are responding to commands as expected
PASS - Remove/delete stx-openstack
Closes-Bug: 2032382
Change-Id: I65b3fb64046567d18abb5335042a3df3b887d7c3
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
While testing the upversioned packages for this stx-openstackclients
image, it was noted that the build of it was failing due to some issues
with the dependencies for it, and also that some packages were missing
or unnecessary.
On the Victoria release for Openstack, the python-ceilometerclient was
removed from the project [1], and now that this image is going to be
build on the Antelope (stable/2023.1) release, this package is no longer
required for this image.
Also, it was noted that the python-aodhclient (called as simply
aodhclient) was not present on this image, probably because this image
was not maintained in the past as both the platform and the
STX-Openstack application shared the clients. Now that the clients for
the STX-Openstack application are containerized via this image, we have
to add this client on it.
Regarding the issue with this image's packages dependencies versions,
it was solved by adding the `UPGRADE_PIP_PACKAGES=pip` parameter.
[1] https://opendev.org/openstack/python-ceilometerclient
Test Plan:
PASS: Build all clients packages (aodh, cinder, glance, heat, keystone,
neutron, nova, barbican and openstack clients)
PASS: Build stx-openstackclients image
PASS: Run image locally and verify that the aodh client is present, and
the ceilometer client is not.
PASS: Apply the STX-Openstack app with Helm overrides pointing to this
custom-build image
PASS: Make sure the clients are accessible by sourcing `/var/opt/openstack/admin-openrc` and issuing any openstack command
Story: 2010715
Task: 48137
Change-Id: I86fb4a48c68f3a76d3fdf4c82421fe9e95ccd5a8
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
During the Upversion to Antelope it was noticed that
the nova pod was failing to start because of this
filter added during helm-override. The Retryfilter
was was deprecated in the Train release (20.0.0) and
REMOVED in Victoria (22.4.0)[1] and wont work on Antelope
and is not currently being used in Ussuri as well.
[1] https://docs.openstack.org/releasenotes/nova/victoria.html
Test plan:
PASS - build stx-nova image (Antelope)
PASS - upload custom image and push it to local registry
PASS - apply stx-openstack (with antelope images)
and check pod health
PASS - helm-override nova chart (Ussuri)
PASS - apply stx-openstack (With ussuri images)
and check pods health
Story: 2010715
Task: 48600
Signed-off-by: Romulo Leite <romulo.leite@windriver.com>
Change-Id: Iebe82d2fb9c90907ed6b45b6863870e675cbaa54
Recently, it has been observed that, on systems with multiple controller
nodes, the `clients` pods are failing to initialize on standby
controllers due to the absence of their respective working directories.
In the past, this wasn't a problem because the working directory was
originally mounted with type `DirectoryOrCreate`, that is, K8s was
responsible for ensuring that this directory existed during `clients`
pods initialization.
However, the problem with this parameter is that it creates directories
with `root:root` permissions, which isn't ideal for system setups
involving multiple user accesses.
At the time, we solved this problem by simply moving the working
directory creation logic to the application's lifecycle code, as seen in
[1].
This turned out to have side effects on systems with multiple
controller nodes, however, as not all lifecycle hooks run on standby
controllers. Consequently, the working directories weren't being created
on these nodes.
Simply put, we can solve the pod initialization problem by mounting the
directories with `DirectoryOrCreate` (again). However, we must ensure
that these directories will have the right permissions when a host
swacts, and that's exactly what this change is aimed at.
This change also improves the code, by:
* Replacing the `change_file_mode()` and `change_file_owner()` utility
functions with `os` builtins;
* Synchronizing LDAP groups with Linux groups.
- In some scenarios, e.g., multiple "applies followed by removes",
the `openstack` LDAP group was created with a different GID than
the `openstack` Linux group, which caused issues with checking
the clients' working directory permissions.
[1] b2e10bfc5f/python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/lifecycle/lifecycle_openstack.py (L310)
Test Plan (on AIO-DX):
PASS - Build python3-k8sapp-openstack package
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
PASS - Upload/apply stx-openstack
PASS - Verify that all `clients` pods are running
On active controller:
PASS - Verify that the `clients` working directory has the right
permissions
On standby controller:
PASS - Verify that the `clients` working directory *does not* have the
right permissions
PASS - Perform a host swact
PASS - Verify that the `clients` working directory has the right
permissions on the former standby controller
PASS - Remove/delete stx-openstack
PASS - Repeat the test plan from upload/apply step successfully
Closes-Bug: 2031058
Change-Id: Ic75d1d0d60855e7956b797079313fa369049384b
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
In the current state of the `clients` helm chart, the OpenStack clients'
working directory path is *static* and located at `/var/opt/openstack`.
While this directory is suitable for uploading small files -- such as
small Glance images -- it can quickly run out of space, as its maximum
capacity is, approximately, 20GB. This might not be enough for a single
user, and even worse, it wouldn't be for multiple users.
Therefore, the OpenStack clients' working directory needs to be
*dynamic*, which is the primary goal of this change.
With this change, the system administrator can change the default
clients' working directory by simply performing a helm override and
reapplying the app, such as:
$ system helm-override-update stx-openstack clients openstack \
--reuse-values --set workingDirectoryPath=/var/opt/another-directory
$ system application-apply stx-openstack
During the re-apply, if the default working directory didn't contain
additional files in user subdirectories, it's removed to avoid leftovers
(in the same way as after removing the application).
For custom working directories, however, the handling is different.
They are *never* removed by the application, even during re-applies or
after application removal. The reason we don't remove it is because this
custom directory might be shared with other users and/or applications,
and, therefore, potentially have important files that must be preserved.
This change also improves the overall quality of the helm chart, by:
* Fixing minor bugs;
* Improving in-code documentation;
* Raising exceptions on critical failures;
* Removing functions and modules that are no longer needed;
* Replacing None-returning functions with Boolean-returning functions,
to better indicate success or failure status.
Test Plan:
PASS - Build python3-k8sapp-openstack package
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
Without exceptions:
-------------------
PASS - Upload/apply stx-openstack
PASS - Verify that the working directory was created at
/var/opt/openstack
PASS - Perform a helm override:
$ system helm-override-update stx-openstack clients openstack \
--reuse-values --set \
workingDirectoryPath=/var/opt/another-directory
PASS - Verify that the working directory was created at
/var/opt/another-directory
With exceptions:
----------------
PASS - Upload *faulty* stx-openstack
This faulty version of the application raises, on purpose,
a KubeAppApplyFailure exception during the creation of
application specific resources
PASS - Try to apply stx-openstack and verify that it immediately
fails and goes to "apply-failed" state
PASS - Remove/delete stx-openstack
Story: 2010774
Task: 48410
Change-Id: Ieb06a93c669d362ecd30bff79947ee4c015cf2b4
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
After the introduction of the containerized clients by [1], it was
noted that it was not possible to use them properly if the system had
HTTPS enabled.
This changes enabled the containerized clients to work when HTTPS is
enabled, without any intervention from the user, in the same way as the
platform clients work. For that, a secret had to be created
(secret-ingress-tls.yaml) containing the certificates information, and
this secret had to be mounted on the clients container. For this secret
to be created, a new job had to be created (job-boostrap.yaml) which
required some modifications to the values and static overrides for this
helm chart.
Besides that, some changes were required to the k8sapp-openstack, to
allow dynamic modification of the overrides for this chart when HTTPS
is enabled.
[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/885603
Test Plan (builds):
PASS: Build stx-openstack-helm-fluxcd and python3-k8sapp-openstack
PASS: Build Helm charts
Test Plan (HTTPS disabled):
PASS: Upload / apply the STX-Openstack application
PASS: Verify that all clients resources are up and running
PASS: Verify that none of the TLS resources were created (secret,
volume, volume mount)
PASS: Verify that the containerized clients work as intended (issue
an `openstack user list` after sourcing the /var/opt/openstack/
admin-openrc file)
PASS: Remove / delete the STX-Openstack application
Test Plan (HTTPS enabled):
PASS: Upload / apply the STX-Openstack application
PASS: Verify that the required system overrides are applied
PASS: Verify that all clients resources are up and running
PASS: Verify that the TLS resources were created (secret, volume,
volume mount)
PASS: Verify that the containerized clients work as intended (issue
an `openstack user list` after sourcing the /var/opt/openstack/
admin-openrc file)
PASS: Remove / delete the STX-Openstack application
Story: 2010774
Task: 48411
Change-Id: I8ff9170b55ae5399553a736ce857b50cc7866dc4
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
In this commit, removed NOVA prefix from the constants as this is a
historical artifact tracing back to an openstack-only solution.
TEST PLAN:
PASS: build success of stx-openstack-helm-fluxcd and
python3-k8sapp-openstack
"build-pkgs -c -p stx-openstack-helm-fluxcd"
"build-pkgs -c -p python3-k8sapp-openstack"
PASS: bootstrap is success
Story: 2010604
Task: 48428
Depends-On: https://review.opendev.org/c/starlingx/config/+/889011
Change-Id: If8b0cc8d0750d11f7604ce419e3837b1e7f7c908
Signed-off-by: Md Irshad Sheikh <mdirshad.sheikh@windriver.com>
Zuul