Commit Graph

721 Commits (master)

Author SHA1 Message Date
Zuul 73caa5fe7f Merge "Move python2 to python3 in DIST_PACKAGES" 2023-11-13 19:12:39 +00:00
Lucas de Ataides dd70c154bf Move python2 to python3 in DIST_PACKAGES
The change [1] updates the version of the libpython2.7-minimal,
libpython2.7-stdlib, python2.7 and python2.7-minimal packages, which
caused the build for the stx-openstackclients image to fail, as it
requires the `python-dev` package [2].

Since Python 2 is not used by any Openstack project anymore, this
package wasn't doing anything, so it's safe to upgrade it to the Python
3 version of the same package. This not only solves the build issue but
keeps the image coherent due to the reason mentioned in the beginning of
this paragraph.

[1] 55eebf2d6a
[2] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/upstream/openstack/python-openstackclient/debian/stx-openstackclient.stable_docker_image#L17

Test Plan:
PASS: Build stx-debian base image
PASS: Build stx-openstackclients image
PASS: Upload / apply stx-openstack
PASS: Manually upload built image to the system's registry and perform
      helm-override to use it in the clients containers
PASS: Issue Openstack commands for all CLIs and verify they're working:
      Cinder, Glance, Heat, Neutron, Nova and Openstack

Closes-Bug: 2043390

Change-Id: Id8d3b3572eefd7e8b0cdbd4b8c3129728647ede7
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-11-13 17:35:39 +00:00
Zuul 91407c9c4e Merge "Fix volume backup creation with Openstack CLI" 2023-11-13 16:57:04 +00:00
Lucas de Ataides a515f0dc54 Fix volume backup creation with Openstack CLI
Since we started using Openstack Antelope, it was noticed that, when
creating backups using the Openstack CLI and specifying the `--location`
parameter, the backups would always fail with the error message:
"Backup driver reported an error: Multidriver is set but a backup
driver wasn't provided." This error was caused because, even though
patch 0002 introduced the `--location` parameter to the CLI, it was not
passing it through to the Cinder API.

This change fixes that by passing the location parameter on the backup
creation, when using the Openstack CLI.

Test Plan:
PASS: Build stx-openstackclients image
PASS: Upload / apply stx-openstack
PASS: Manually upload built image to the system's registry and perform
      helm-override to use it in the clients containers
PASS: Create a Cinder volume, and boot a VM from it
PASS: Create a backup using the Openstack CLI, and passing the
      `--location` parameter
PASS: Remove / delete stx-openstack

Closes-Bug: 2043383

Change-Id: Ie6f8756e3c647d2191489e298e3ea138f32c30d4
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-11-13 12:06:04 -03:00
Thales Elero Cervi c133617339 Skip destination CPU check during live-migration
OpenStack 2023.1 (Antelope) introduced this issue [1][2] when
live-migrating instances, in which Nova uses getCapabilities() to
determine the host CPU model but use the model from the
domCapabilities for the guest VM using host-model [3].
According to the libvirt maintainers nova should never use
getCapabilities for anything any more.
A solution for this issue is being developed in main branch [4], but
taken as a medium priority since there is a workaround config option
already available [5] to avoid this situation.

Setting "skip_cpu_compare_on_dest" to True will, during live
migration, skip comparing guest CPU with the destination host.
When using QEMU >= 2.9 and libvirt >= 4.4.0, libvirt will do the
correct thing with respect to checking CPU compatibility on the
destination host during live migration.
StarlingX currently delivers QEMU 5.2 and stx-openstack uses a libvirt
on 7.0.0-3, therefore this can be safely used as part of our default
nova configuration updates.

[1] https://bugs.launchpad.net/nova/+bug/2023035
[2] https://bugs.launchpad.net/nova/+bug/2039803
[3] https://review.opendev.org/q/topic:fix_compareCPU_usage
[4] https://review.opendev.org/c/openstack/nova/+/899185
[5] https://docs.openstack.org/nova/latest/configuration/config.html#skip_cpu_compare_on_dest

Closes-bug: 2007303

TEST PLAN:
PASS - Build python3-k8sapp-openstack plugins
PASS - Build stx-openstack application
PASS - Upload/Apply/Remove stx-openstack
PASS - Live-migrate an instance
PASS - Manually reboot a compute node in which
       an instance is running to ensure it is
       correctly evacuated to another compute

Change-Id: Id7a93445af4115ee81b035b4f9dc7a6eb889555b
Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
2023-11-13 10:20:26 -03:00
Thales Elero Cervi 1ded5ebc08 Removing unused image tags
After upversioning openstack-helm and openstack-helm-infra, update
versions of helm charts introduced new docker images that are not used
by our stx-openstack distro but would still be downloaded during an
application apply if the helm values are not set to null.

This change removes unnecessary neutron-ovn and
libvirtd-export images.

Closes-Bug: 2043037

Test Plan:
PASS - Build stx-openstack-helm-fluxcd
PASS - Build stx-openstack application tarball
PASS - Upload/Apply/Remove stx-openstack application

Change-Id: I36bbde03785ad11b0cf97a9cf3e357ebd8084175
Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
2023-11-08 14:27:48 -03:00
Zuul 374fbd76e0 Merge "Use FQDN for MGMT network" 2023-11-06 18:51:27 +00:00
Luan Nunes Utimura 9b0fbb30f0 stx-openstackclients: Add python3-osc-placement
This change adds `python3-osc-placement` to the `stx-openstackclients`
container image so that the end user has access to CLIs provided by
Placement, e.g.:

  $ openstack resource provider ...

As mentioned in LP-2042482, these CLIs can be used to compensate for
a change in the `openstack hypervisor` command that cause `vcpus_used`
and `free_disk_gb` fields to no longer be returned.

These are now only accessible via:

  $ openstack resource provider inventory list
  $ openstack resource provider usage show

Test Plan:
PASS - Build stx-debian base image
PASS - Build stx-openstackclients container image
PASS - Upload/apply stx-openstack
PASS - Upload built image to the system's registry and perform
       helm-override to use it in the clients container
PASS - Verify that the CLIs are accessible:
        $ openstack resource provider -h
PASS - Remove/delete stx-openstack

Depends-On: https://review.opendev.org/c/starlingx/tools/+/899898

Closes-Bug: 2042482

Change-Id: I1c72b135e7ff662b83de9eac1ec5c049943017f2
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
2023-11-01 20:47:16 -03:00
Zuul b5e8cd6e7a Merge "Update Glance config according to OSSN-0090" 2023-10-31 11:54:36 +00:00
Zuul 468ecac6a1 Merge "Split horizon helm-override file" 2023-10-27 12:21:20 +00:00
Fabiano Correa Mercer 3d16b9f5e3 Use FQDN for MGMT network
The management network is used extensively for all internal
communication.
Since the original use of the network was a private network before
it was exposed for external communication in a distributed cloud
configuration, it was never designed to be reconfigured.

To support MGMT network reconfiguration the idea is to configure the
applications to use the hostname/FQDN instead of a static MGMT IP
address.

In this way the MGMT network can be changed and the services and
applications will still work since they are using the hostname/FQDN
and the DNS will be responsible to translate to the current MGMT
IP address.

The use of FQDN will be applied for all installation modes: AIO-SX,
AIO-DX, Standard, AIO-PLUS and DC subclouds. But given the
complexities of supporting the multi-host reconfiguration,
the MGMT network reconfiguration will focus on support for AIO-SX
only.

The DNSMASQ service must start as soon as possible to translate
the FQDN to IP address.

Test plan ( Debian only )
 - AIO-SX and AIO-DX virtualbox installation IPv4/IPv6
 - Standard virtualbox installation IPv6
 - DC virtualbox installation IPv4 ( AIO-SX/DX subclouds )
 - AIO-SX and AIO-DX installation IPv4/IPv6
 - AIO-DX plus installation IPv6
 - DC IPv6 and subcloud AIO-SX
 - AIO-DX host-swact
 - DC IPv4 virtualbox with subcloud AIO-DX and AIO-DX
 - AIO-SX to AIO-DX migration
 - netstat -tupl ( no services are using the MGMT IP address )
 - Ran sanity/regression tests
 - Openstack sanity/regression tests


Story: 2010722
Task: 48467

Depends-On: https://review.opendev.org/c/starlingx/config/+/886208

Change-Id: I8ce844977156f56edb861446640a0c3c6f4a8007
Signed-off-by: Fabiano Correa Mercer <fabiano.correamercer@windriver.com>
2023-10-26 15:57:41 -03:00
Lucas de Ataides 57bb421e85 Update Glance config according to OSSN-0090
The security note OSSN-0090 [1] from Openstack describes a
configuration that makes it possible to open some known attack vectors
by which malicious data modification can occur.

The vulnerability only occurs if one of the following parameters are
set to True in Glance's configuration: `show_multiple_locations` or
`show_image_direct_url`. In the case of the STX-Openstack app, the
`show_image_direct_url` was being set to True in the app's plugin [2].

It looks like this configuration was just transported from when the
app's plugin was dettached from sysinv [3], which itself, was based on
an even older commit [4]. Unfortunately, the commit message of [4] has
no explanation on why this was required.

This commit changes the app's plugin to define both the
`show_image_direct_url` and the `show_multiple_locations` parameters to
`False`, since we can easily avoid the security issue described in [1]
by doing that, and it does not impact on the applications
functionalities, as seen in the test plan for this change.

[1] https://wiki.openstack.org/wiki/OSSN/OSSN-0090
[2] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/glance.py#L155
[3] https://review.opendev.org/c/starlingx/openstack-armada-app/+/688190
[4] https://review.opendev.org/c/starlingx/config/+/611948

Test Plan:
PASS: Build python3-k8sapp-openstack and stx-openstack-helmf-fluxcd
      packages
PASS: Build STX-Openstack helm charts
PASS: Upload / apply / remove STX-Openstack app
PASS: Inspection of /etc/glance/glance-api.conf shows that both the
      show_multiple_locations and show_image_direct_url parameters are
      set to `False`

Using OpenStack's CLI:
    PASS: Image commands are executed as expected: `openstack image
          list`, `openstack image show`
    PASS: Create an image
    PASS: Delete an image
    PASS: Create a bootable volume from an image
    PASS: Boot a VM using an image as the source
    PASS: Boot a VM using the bootable volume as a source
    PASS: Delete both VMs and the volume

Using Horizon dashboard:
    PASS: All tenants are able to see and inspect images
    PASS: Create an image
    PASS: Delete an image
    PASS: Create a bootable volume from an image
    PASS: Boot a VM using an image as the source
    PASS: Boot a VM using the bootable volume as a source
    PASS: Delete both VMs and the volume

Closes-Bug: 1993606

Change-Id: I20f241224234353363c632085e3e41b91f97abf5
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-10-26 14:04:22 -03:00
Zuul 0406f947a3 Merge "stx-openstackclients: Move all clients to DIST" 2023-10-23 17:36:04 +00:00
Zuul 8d79ca54e0 Merge "stx-openstackclients: Add openstack in DIST_REPOS" 2023-10-23 17:36:02 +00:00
Zuul 0f618f9659 Merge "stx-openstackclients: Move patched clients to DIST" 2023-10-23 17:36:01 +00:00
Zuul ec1ae7691b Merge "Uncomment Openstack Debian packages and images" 2023-10-23 17:35:55 +00:00
Zuul 1f1fc31e0d Merge "Remove unused packages from openstack-armada-app" 2023-10-23 17:35:53 +00:00
Romulo Leite ff3bed6e77 Split horizon helm-override file
As a side-effect of the migration from the migration
from helm v2 to v3 and Armada to FluxCD, the helm
override of the horizon rbac policy was failing to
apply this file because it was too large. By splitting
this file and making two helm overrides the apply can
finish successfully.

Test plan:

PASS: system helm-override-update stx-openstack horizon openstack --reuse-values --values=rbac/horizon-policy-overrides.yml
      system helm-override-update stx-openstack horizon openstack --reuse-values --values=rbac/horizon-nova-policy-overrides.yml
PASS: Reapply the app and check the helm overrides succeed

Closes bug: 2040165

Change-Id: Ib1c82544cd1f2335554f740bc3fe733ce57370ab
Signed-off-by: Romulo Leite <romulo.leite@windriver.com>
2023-10-23 13:20:58 +00:00
Lucas de Ataides eaa8b41cb0 Allow VMs to be created via volumes
After STX-Openstack upversioned to Antelope, we noticed that it was not
possible to create VMs by volume, as they would be stuck on ERROR
status. The first proposed solution was to create a patch containing
[1] and [2], because, as specified in [3], Nova now requires
a service token in order to be able to manipulate Cinder volumes. This
unfortunately did not solve the issue by itself, as now an error message
showed up on the nova-conductor pods with the following (not full error
message, only important part): "nova.exception.RescheduledException:
Build of instance 2f32c7ea-1720-4f61-bce8-dbe970c40b0c was re-scheduled:
Secret not found: no secret with matching uuid 'a7f3ae2e-cee7-4f04-9402
-a78047747654". This UUID was not the same one present when issuing
`virsh secret-list` on Cinder, Nova and Libvirt containers.

Turns out openstack-helm and openstack-helm-infra have a Ceph UUID
hardcoded in them, in Cinder [4], Nova [5] [6] and Libvirt [7] values.
By changing these values to the UUID that libvirt was trying to find
(7f3ae2e-cee7-4f04-9402-a78047747654), and it worked to solve the issue.
However, it is not a good practice to use hardcoded values, and,
searching on where this UUID was coming from, it turns out it was
defined by the platform's Ceph configuration under
`/etc/ceph/ceph.conf`.

This still leaves the question, why was this working on Ussuri and
stopped working on Antelope? First of all, the Ceph official
documentation [8] [9] about using it with OpenStack explains the
process of adding the secret to libvirt, to store the ceph admin
keyring. You can see that the secret uuid is generated "on the fly" and
both docs mention that old/hard-coded value
(i.e., 457eb676-33da-42ec-9a8c-9293d545c337). This is the reason why it
used to work until our upversion to OpenStack Antelope/2023.1: this
UUID does not really matter (as long as nova and libvirt have the same
value for it)! It is a given UUID to the libvirt secret that will store
ceph keyring [10], and the key ring will ensure proper communication
between our services and the platform ceph.

What changed between Ussuri and Antelope (2023.1), is that now there is
a specific method [11] to set a default value (Ceph's Cluster UUID) for
this UUID when it is not specified in the driver configuration.

What this change does is dynamically read this `/etc/ceph/ceph.conf`
file to search for the UUID value, and use it to override the [4] [5]
[6] and [7] values. It also adds the patch including the Nova service
token configuration. The combination of these 2 changes allows VMs to be
created by volumes.

[1] 91c8a5baf2
[2] 7d39af25fd
[3] https://docs.openstack.org/releasenotes/cinder/2023.1.html#upgrade-notes
[4] https://opendev.org/openstack/openstack-helm/src/branch/master/cinder/values.yaml#L942
[5] https://opendev.org/openstack/openstack-helm/src/branch/master/nova/values.yaml#L594
[6] https://opendev.org/openstack/openstack-helm/src/branch/master/nova/values.yaml#L1432
[7] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/libvirt/values.yaml#L100
[8] https://github.com/ceph/ceph/blob/main/doc/rbd/rbd-openstack.rst
[9] https://docs.huihoo.com/ceph/v0.80.5/rbd/rbd-openstack/index.html
[10] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/libvirt.py#L60
[11] 6464d37d16 (diff-9b122c182b4333b747e7fd7e07f73d68ff30256a)

Test Plan:
PASS: Build openstack-helm, python3-k8sapp-openstack and
      stx-openstack-helm-fluxcd
PASS: Upload / apply / remove STX-Openstack
PASS: Create a VM by an image
PASS: Create a volume and launch a VM from it
PASS: Create a VM using the `boot-from-volume` flag
PASS: Delete a VM created by a volume

Closes-Bug: 2037463

Change-Id: Ia00bb8dbe3460ce817d69049f97f56a96ad6a298
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-10-17 10:03:20 -03:00
Luan Nunes Utimura 3793f3ca4b stx-openstackclients: Move all clients to DIST
In [1], two OpenStack clients were moved to the `DIST_PACKAGES` section
of the `stx-openstackclients` container image build recipe due to the
need to use packages patched exclusively for StarlingX:

  * python3-cinderclient;
  * python3-openstackclient.

Although this change served its purpose, it had a side-effect that
caused the main CLI, `openstack`, to stop recognizing other clients
installed via `PIP_PACKAGES`.

This occurred because once installed via `DIST_PACKAGES`, both
`cinderclient` and `openstackclient` started to reside in the
dist-packages/ directory, while all other clients continued to reside in
site-packages/. As `openstackclient` is the one responsible for the main
CLI, it stopped finding the other clients because its search directory
changed to dist-packages/.

Therefore, in this change, we chose to move all OpenStack clients to
DIST_PACKAGES. This way, it is guaranteed that the main CLI will
correctly identify the other OpenStack clients.

[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/896129

Test Plan:
PASS - Build stx-debian base image
PASS - Build stx-openstackclients container image
PASS - Upload/apply stx-openstack
PASS - Upload built image to the system's local registry and perform
       a helm-override to use it in the OpenStack clients container
PASS - Run commands that had disappeared since the merge of [1], e.g.,
       the ones provided by `heatclient`:
       * openstack stack list
       * openstack stack create
       ...
PASS - Remove/delete stx-openstack

Closes-Bug: 2039096

Co-authored-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
Co-authored-by: Romulo Leite <romulo.leite@windriver.com>
Change-Id: I61d653ceef07f99568a21b4ad6f6dceffed22ca1
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
(cherry picked from commit 5bcbab406a)
2023-10-16 14:24:40 -03:00
Luan Nunes Utimura 22a84736a3 stx-openstackclients: Add openstack in DIST_REPOS
After introducing a new set of layer-specific `sources.list` files in
[1], we can now update the `DIST_REPOS` of this image to additionally
use the `openstack` layer binary repository when searching for
dependencies of packages listed in `DIST_PACKAGES`.

This ensures that OpenStack Antelope packages will be built *without* OpenStack Victoria dependencies.

[1] https://review.opendev.org/c/starlingx/root/+/896277

Test Plan:
PASS - Build `stx-debian` base image
PASS - Build `stx-openstackclients` container image and confirm that
       OpenStack Antelope dependencies were used in the build of
       `python3-cinderclient` and `python3-openstackclient`

Depends-On: https://review.opendev.org/c/starlingx/root/+/897944

Closes-Bug: 2037339

Change-Id: If94d5281c5c2633b59a6aaf39f0881fb5cec61bb
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
(cherry picked from commit 418a7bcb6e)
2023-10-11 09:49:35 -03:00
Luan Nunes Utimura e7abc4762e stx-openstackclients: Move patched clients to DIST
Ever since we migrated to containerized clients, it has been observed
that two clients -- `cinderclient` and `openstackclient` -- have lost an
additional parameter that was added by patches [1] and [2]:
`--location`.

Apparently, this is happening because the build of this container image
is prioritizing the download of indexed packages rather than the use of
the packages we deliver within the wheels tarball (which are patched
with modifications specific to StarlingX).

According to [3], when we specify an extra source of wheels for `pip`
with `--find-links` -- which is what `openstack/loci` does [4] -- it
does not mean that `pip` will necessarily prioritize this source over
others, such as PyPi.

Therefore, to ensure that our patched clients are used in the container
image, we moved them from `PIP_PACKAGES` to `DIST_PACKAGES` in this
change.

Note: The other clients can continue to be installed from indexed
      packages. For this reason, they remain in `PIP_PACKAGES`.

[1] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/upstream/openstack/python-cinderclient/debian/patches/0001-Add-location-parameter-for-volume-backup-creation.patch
[2] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/upstream/openstack/python-openstackclient/debian/patches/0002-Add-location-parameter-for-volume-backup-creation.patch
[3] https://github.com/pypa/pip/issues/9959
[4] efccd0a853/scripts/pip_install.sh (L7)

Test Plan:
PASS - Build `stx-openstackclients` container image
PASS - Run image and confirm that both clients have the --location
       parameter:

       1. docker run -it <user>/stx-openstackclients:<tag> bash
       2. openstack volume backup create -h
       3. cinder help backup-create

Closes-Bug: 2036986

Co-Authored-By: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
Co-Authored-By: Romulo Leite <romulo.leite@windriver.com>
Change-Id: I4eeece7fc4a98b5254ca9367ca85c14ef5df7f3c
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
(cherry picked from commit e25852370e)
2023-10-11 09:48:41 -03:00
Lucas de Ataides c3bdd8a3a4 Uncomment Openstack Debian packages and images
With the new STX-Openstack manifest [1], we can now uncomment the
packages in this repository so they can be build. These were previously
commented as they're also present on [2] and having duplicate packages
would cause the build to fail, or grab the incorrect packages.

This change also updates the build layer for this repo to the
openstack layer, that is being created by [3]

Note that since this change is on the `f/antelope` branch, it will not
impact the default manifest, nor the default StarlingX build.

[1] https://review.opendev.org/c/starlingx/manifest/+/891875
[2] https://opendev.org/starlingx/upstream/src/branch/master/debian_pkg_dirs
[3] https://review.opendev.org/c/starlingx/tools/+/892166

Test Plan:
PASS: Build all packages from this repository
PASS: Build all images from this repository

Depends-On: https://review.opendev.org/c/starlingx/manifest/+/893244
Depends-On: https://review.opendev.org/c/starlingx/manifest/+/893245

Story: 2010797
Task: 48691

Change-Id: I24677513f42f92258162ea09fe7e6c11b52093a1
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
(cherry picked from commit fb512b9c60)
2023-10-02 10:07:36 -03:00
Lucas de Ataides a8969f2988 Remove unused packages from openstack-armada-app
The purpose of this change is to remove packages that are not used by
STX-Openstack from this repository. At the time these folders were
copied from the `starlingx/upstream` repo to here on [1], the goal was
to simply get the folder structure and all the packages, but, now that
the this repository is being actively used for the STX-Openstack
upversion to Antelope, we noticed that these packages are either not
used (barbican, keystone, openstack-resource-agents, horizon,
python-osc-lib, python-oslo.messaging, python-wsme, rabbitmq-server),
or were dropped from Openstack (gnocchi and the python-gnocchiclient
are not present on Antelope, as these services were removed from the
Openstack project [2] [3]).

Except for gnocchi and python-gnocchiclient, as explained above, these packages were present on the `starlingx/upstream` repository as they're required for the StarlingX builds and ISO, but not for the STX-Openstack application, for that reason, these are safe to be dropped from this repository.

[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/886027
[2] https://opendev.org/openstack/gnocchi
[3] https://opendev.org/openstack/python-gnocchiclient

Test Plan:
PASS: Removed packages are not found when issuing `build-pkgs`
PASS: Build all STX-Openstack images
PASS: Build STX-Openstack tarball
PASS: Upload / apply / remove STX-Openstack

Related-Bug: 2027589

Change-Id: I1697bce6b606fb603a0cb4a2c35b6255e129a080
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-09-28 20:23:29 -03:00
Thales Elero Cervi 310f677d29 Move live-migration traffic to cluster-host-net
This change updates the application plugins in order to ensure that all
libvirt/live-migration related traffic is happening through the
cluster-host-network. Currently most of the libvirt/live-migration
addresses are being solved through INADDR_ANY (0.0.0.0), and this route
resolution will vary between AIO, routes to oam-network, and Worker,
routes to mgmt-network. Both resolutions are not correct since the
correct network for such traffic should be the cluster-host-network.
Actually, current platform firewall will block any traffic through not
allowed oam-network ports.

The goal will be achieved by setting to the node's cluster-host IP:
* libvirt listen_addr
* nova.conf "live_migration_inbound_addr"

It is important to notice that in the current version of the
openstack-helm nova helm chart, there is a problem with
nova-compute-init.sh for this use case of ours, so an openstack-helm
patch was required to fix it.

Code that was previously implemented only for the Nova plugin and is now
required by the Libvirt plugin, was moved to the parent OpenStack class.

[1] 31be86079d

TEST PLAN:
PASS - Build stx-openstack application
PASS - Apply the application to an AIO-DX system
PASS - "$ sudo netstat -ltnp | grep <libvirtd pid>" to ensure that
       libvirtd is listening on the correct cluster-host-net IP
PASS - Verify that the nova-compute.sh script was populated correctly
PASS - Test a VM live-migration on the controller+worker node
PASS - Verify that live_migration data in LibvirtLiveMigrateData has the
       correct cluster-host-net IP address in its "target_connect_addr"
PASS - Apply the application to a Standard system
PASS - "$ sudo netstat -ltnp | grep <libvirtd pid>" to ensure that
       libvirtd is listening on the correct cluster-host-net IP
PASS - Verify that the nova-compute.sh script was populated correctly
PASS - Test a VM live-migration on the worker node
PASS - Verify that live_migration data in LibvirtLiveMigrateData has the
       correct cluster-host-net IP address in its "target_connect_addr"

Closes-Bug: 2037330

Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
Change-Id: I37db601e4b1b0e397a1b8dbdad1a293ff25c2e55
2023-09-27 10:01:11 -03:00
Thales Elero Cervi 398d9e0472 Revert "Remove Nova prefix"
Since [1] was reverted [2], this follow-up change also need to be reverted.

[1] https://review.opendev.org/c/starlingx/config/+/889011
[2] https://review.opendev.org/c/starlingx/config/+/894816

This reverts commit da73bc7582.

Reason for revert: Original related change to stx/config was reverted

Change-Id: Id6372f939c9386d75ff6ae50a4256d27de805bc9
2023-09-25 18:07:58 +00:00
Zuul 7190f7fb50 Merge "Add `get_namespace` function to OpenstackBaseHelm" 2023-09-20 19:33:06 +00:00
Lucas de Ataides aaddc97db4 Add `get_namespace` function to OpenstackBaseHelm
After the introdution of [1], it was noted that Helm overrides were not
working properly for STX-Openstack. Althought they show up on
helm-override-show, they we're not being passed to Helm and Kubernetes.
This was caused because the [1] change updated the way that the
overrides are applied, requiring the `get_namespace` function on the
plugin to define where to apply these overrides, and, if this function
did not exist, it would default to the `default` namespace.

This change fixes that by adding the `get_namespace` function to the
OpenstackBaseHelm plugin, which is extended by the other STX-O plugins.

This change also does the following:

1) Updates the base class for ingress, memcached and garbd from
   BaseHelm to OpenstackBaseHelm, as openstack.BaseHelm is the old base
   class for the armada app and probably should be cleaned now that we
   no longer support Armada.
2) Remove ingress customizations for the kube-system namespace, as
   they're not used anymore and can be removed without any impact.

[1] https://review.opendev.org/c/starlingx/config/+/887430

Test Plan:
PASS: Build python3-k8sapp-openstack and stx-openstack-helm-fluxcd
PASS: Build STX-Openstack tarball
PASS: Upload / apply STX-Openstack on an AIO-SX machine with HTTPS
      enabled
PASS: Verify that there's only one ingress pod per component
PASS: After passing helm overrides, verify that they're applied
      (Example: change number of pods from 1 to 2)
PEND: Confirm overrides work for every Openstack component

Closes-Bug: 2036633

Change-Id: Id01a7367335213015531e1e9186b2b1cc677ef8e
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-09-20 15:07:44 -03:00
Luan Nunes Utimura 670ec0c6e5 clients: Fix multi-word option parsing in CLIs
When using container-backed OpenStack CLIs, it has been observed that
commands with multi-word options are failing to run due to poor parsing
of parameters by the wrapper script.

Currently, all commands that are redirected to the clients container are
expanded using the * parameter, i.e., $* [1]. However, according to [2],
a more appropriate option would be to expand the * parameter with the
Q operator, since:

> The expansion is a string that is the value of _parameter_ quoted in
  a format that can be reused as input.

Which is exactly what we are doing with the wrapper script -- reusing
inputs but in a containerized environment.

[1] 4c238a8063/stx-openstack-helm-fluxcd/stx-openstack-helm-fluxcd/helm-charts/clients/templates/bin/_clients-wrapper.sh.tpl (L63-L64)
[2] https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html

Test Plan:
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
PASS - Upload/apply stx-openstack
PASS - Source `/var/opt/openstack/admin-openrc`
PASS - Run command with multi-word option(s), e.g.:
       $ openstack project create \
         --description="words separated by space" my_project
       and confirm that it's executed successfully
PASS - Remove/delete stx-openstack

Closes-Bug: 2036447

Change-Id: I6441cf077e0bcfe386d7086ea90298e6dd247a2a
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
2023-09-18 16:04:26 -03:00
Zuul 8c6e116a3a Merge "Removing debian_iso_image.inc file" 2023-08-31 19:56:30 +00:00
Thales Elero Cervi 4f4c4774e4 Removing debian_iso_image.inc file
There is no need of including stx-openstack-helm-fluxcd package to
StarlingX ISO, since this package is used only by the script that
generates the application tarball and this application is not delivered
along with the platform by default.

Having this package installed has no side-effects, but it installs
unnecessary files to /usr/lib on application, fluxcd and helm
directories.

TEST PLAN:
PASS - Successfully build a StarlingX ISO

Closes-Bug: 2028959

Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
Change-Id: I342266c0932bb7853aa5a377b3a5663677858b27
2023-08-31 13:56:39 -03:00
Lucas de Ataides efd6f74d40 Update Horizon SSL Cert path
It was observed that, when using the STX-Openstack app with HTTPS
enabled, it was not possible to visualize / edit / create images using
Horizon. This was caused because the SSL certificate (ca.crt) volume
mount path defined in Horizon's configuration [1] was not the same as
the default one created when using the Helm toolkit [2].

As specified in [3], one possible solution was to define a path to the
TLS secret to be created, but that caused other services in Horizon to
fail, so instead of doing that, the solution was to change Horizon's
configuration to search the ca.crt file in the correct path.

[1] https://opendev.org/openstack/openstack-helm/src/branch/master/horizon/values.yaml#L469
[2] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/helm-toolkit/templates/snippets/_tls_volume_mount.tpl#L56
[3] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/helm-toolkit/templates/snippets/_tls_volume_mount.tpl#L30

Test Plan:
PASS: Build openstack-helm and stx-openstack-helm-fluxcd
PASS: Build STX-Openstack tarball
PASS: Upload / apply STX-Openstack with HTTPS enabled
PASS: Create an image by using the Glance CLI
PASS: On Horizon, under Project -> Compute -> Images, verify that the
      image is present
PASS: Create an image via Horizon

Closes-Bug: 2032980

Change-Id: If34c0d4a287804e4fce0e5f70bb97feb330be174
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-08-28 10:10:28 -03:00
Luan Nunes Utimura be56c15bc0 Fix TLS vol. in cinder-volume-usage-audit CronJob
After the recent upversion of openstack-helm [1], it has been observed
that the `cinder-volume-usage-audit` pod is having problems booting on
systems with HTTPS enabled due to a misconfigured TLS-related
volume/volumeMount pair.

Apparently, this pair of volume and volumeMount was introduced with the
upversion of openstack-helm, and ended up being left out of the changes
made by patch `0010-Remove-TLS-from-openstack-services.patch` that, in
theory, would have solved the problem.

Therefore, this change aims to update the patch in question -- along
with any other patches to avoid conflicts -- so that the
`cinder-volume-usage-audit` pod no longer has problems booting on
systems with HTTPS enabled.

[1] 8254cd31bb

Test Plan (on AIO-DX with HTTPS enabled):
PASS - Build openstack-helm package
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
PASS - Upload/apply stx-openstack
PASS - Verify that all pods -- including `cinder-volume-usage-audit` --
       are either "Running" or "Completed"
PASS - Remove/delete stx-openstack

Closes-Bug: 2032703

Change-Id: Ic13c6945cc9e43f9153820297e74623520446fcd
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
2023-08-23 13:56:58 -03:00
Zuul 8d0da7fc3b Merge "Upversion python-barbicanclient to v5.5.0-2" 2023-08-22 18:17:14 +00:00
Zuul b46a2a736d Merge "Upversion python-aodhclient to v3.2.0-2" 2023-08-22 18:16:45 +00:00
Zuul f2bf00d966 Merge "Add missing clients to the aliases scripts" 2023-08-22 13:53:03 +00:00
Lucas de Ataides baa69a29ab Add missing clients to the aliases scripts
With the [1] change, the aodh client was added to the
stx-openstackclients image, and with that the scripts to setup and clear
the aliases to use the containerized clients have to be updated.

Although not used this change is also adding the barbican and swift
clients to this list, along with the neutron client, which will be
deprecated in the near future [2] (check note under contribution guide)

[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/891553
[2] https://docs.openstack.org/python-neutronclient/2023.1/

Test Plan:
PASS: Build stx-openstack-helm-fluxcd package
PASS: Build STX-Openstack tarball
PASS: Upload / apply STX-Openstack
PASS: With Helm overrides pointing to the stx-openstackclients image
      generated with the [1] change, make sure the aodh, barbican,
      neutron and swift clients are accessible

Story: 2010715
Task: 48137

Change-Id: Ied0e7b271d647da0b44a1913f4cc33cc04856d0b
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-08-22 13:13:41 +00:00
Zuul eeb44a64a2 Merge "Update stx-openstackclients image definition" 2023-08-22 13:12:08 +00:00
Luan Nunes Utimura 44cc84b24b clients: Fix host-specific config overrides
It has been observed that, on Standard systems, users are unable to
interact with containerized clients due to the fact that the `clients`
pods are being initialized without the `controller-` suffix in their
names. These names are relevant for the clients wrapper script to be
able to find and execute commands inside the clients containers [1].

The reason pods are being initialized with wrong names on Standard
systems is because the host-specific overrides are being generated for
hosts with `WORKER` personality instead of `CONTROLLER`, which makes no
sense as the containerized clients were designed to *only* run on
controller nodes.

Therefore, the proposed fix is straightforward and only requires us to
change the host personality required to generate host-specific
overrides.

[1] 4c238a8063/stx-openstack-helm-fluxcd/stx-openstack-helm-fluxcd/helm-charts/clients/templates/bin/_clients-wrapper.sh.tpl (L52-L55)

Test Plan (on Standard):
PASS - Build python3-k8sapp-openstack package
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
PASS - Upload/apply stx-openstack
PASS - Source `/var/opt/openstack/admin-openrc` and check that the
       containerized clients are responding to commands as expected
PASS - Remove/delete stx-openstack

Closes-Bug: 2032382

Change-Id: I65b3fb64046567d18abb5335042a3df3b887d7c3
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
2023-08-21 14:11:47 -03:00
Lucas de Ataides 1dddbe9194 Update stx-openstackclients image definition
While testing the upversioned packages for this stx-openstackclients
image, it was noted that the build of it was failing due to some issues
with the dependencies for it, and also that some packages were missing
or unnecessary.

On the Victoria release for Openstack, the python-ceilometerclient was
removed from the project [1], and now that this image is going to be
build on the Antelope (stable/2023.1) release, this package is no longer
required for this image.

Also, it was noted that the python-aodhclient (called as simply
aodhclient) was not present on this image, probably because this image
was not maintained in the past as both the platform and the
STX-Openstack application shared the clients. Now that the clients for
the STX-Openstack application are containerized via this image, we have
to add this client on it.

Regarding the issue with this image's packages dependencies versions,
it was solved by adding the `UPGRADE_PIP_PACKAGES=pip` parameter.

[1] https://opendev.org/openstack/python-ceilometerclient

Test Plan:
PASS: Build all clients packages (aodh, cinder, glance, heat, keystone,
      neutron, nova, barbican and openstack clients)
PASS: Build stx-openstackclients image
PASS: Run image locally and verify that the aodh client is present, and
      the ceilometer client is not.
PASS: Apply the STX-Openstack app with Helm overrides pointing to this
      custom-build image
PASS: Make sure the clients are accessible by sourcing `/var/opt/openstack/admin-openrc` and issuing any openstack command

Story: 2010715
Task: 48137

Change-Id: I86fb4a48c68f3a76d3fdf4c82421fe9e95ccd5a8
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-08-17 16:05:07 +00:00
Zuul 561571252d Merge "Remove RetryFilter" 2023-08-14 14:58:03 +00:00
Romulo Leite cd66d861b3 Remove RetryFilter
During the Upversion to Antelope it was noticed that
the nova pod was failing to start because of this
filter added during helm-override. The Retryfilter
was was deprecated in the Train release (20.0.0) and
REMOVED in Victoria (22.4.0)[1] and wont work on Antelope
and is not currently being used in Ussuri as well.

[1] https://docs.openstack.org/releasenotes/nova/victoria.html

Test plan:
PASS - build stx-nova image (Antelope)
PASS - upload custom image and push it to local registry
PASS - apply stx-openstack (with antelope images)
       and check pod health
PASS - helm-override nova chart (Ussuri)
PASS - apply stx-openstack (With ussuri images)
       and check pods health

Story: 2010715
Task: 48600

Signed-off-by: Romulo Leite <romulo.leite@windriver.com>
Change-Id: Iebe82d2fb9c90907ed6b45b6863870e675cbaa54
2023-08-14 14:30:58 +00:00
Luan Nunes Utimura dca8b75192 clients: Fix dir. creation on standby controllers
Recently, it has been observed that, on systems with multiple controller
nodes, the `clients` pods are failing to initialize on standby
controllers due to the absence of their respective working directories.

In the past, this wasn't a problem because the working directory was
originally mounted with type `DirectoryOrCreate`, that is, K8s was
responsible for ensuring that this directory existed during `clients`
pods initialization.

However, the problem with this parameter is that it creates directories
with `root:root` permissions, which isn't ideal for system setups
involving multiple user accesses.

At the time, we solved this problem by simply moving the working
directory creation logic to the application's lifecycle code, as seen in
[1].

This turned out to have side effects on systems with multiple
controller nodes, however, as not all lifecycle hooks run on standby
controllers. Consequently, the working directories weren't being created
on these nodes.

Simply put, we can solve the pod initialization problem by mounting the
directories with `DirectoryOrCreate` (again). However, we must ensure
that these directories will have the right permissions when a host
swacts, and that's exactly what this change is aimed at.

This change also improves the code, by:
  * Replacing the `change_file_mode()` and `change_file_owner()` utility
    functions with `os` builtins;
  * Synchronizing LDAP groups with Linux groups.
      - In some scenarios, e.g., multiple "applies followed by removes",
        the `openstack` LDAP group was created with a different GID than
        the `openstack` Linux group, which caused issues with checking
        the clients' working directory permissions.

[1] b2e10bfc5f/python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/lifecycle/lifecycle_openstack.py (L310)

Test Plan (on AIO-DX):
PASS - Build python3-k8sapp-openstack package
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts
PASS - Upload/apply stx-openstack
PASS - Verify that all `clients` pods are running

On active controller:
  PASS - Verify that the `clients` working directory has the right
         permissions

On standby controller:
  PASS - Verify that the `clients` working directory *does not* have the
         right permissions

PASS - Perform a host swact
PASS - Verify that the `clients` working directory has the right
       permissions on the former standby controller
PASS - Remove/delete stx-openstack
PASS - Repeat the test plan from upload/apply step successfully

Closes-Bug: 2031058

Change-Id: Ic75d1d0d60855e7956b797079313fa369049384b
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
2023-08-11 10:34:41 -03:00
Zuul f0ac7b2761 Merge "Fix openvswitch DaemonSet ServiceAccount patch" 2023-08-09 15:09:14 +00:00
Thales Elero Cervi 58ec7994b1 Fix openvswitch DaemonSet ServiceAccount patch
During the openstack-helm-infra upversion [1] it was noticed that the
updated version of openvswitch chart (1.1.15) was missing the custom
ServiceAccount definition for its DaemonSet template.
This fix was proposed upstream [2] and currently implemented to
stx-openstack via an OSH-I patch [3]. The patch though, was missing the
serviceAccountNamedefinition in the daemonset template.

This change fixes the stx-openstack patch, including the
serviceAccountNamedefinition to openvswitch daemonset template.

[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/887637
[2] https://review.opendev.org/c/openstack/openstack-helm-infra/+/888504
[3] https://review.opendev.org/c/starlingx/openstack-armada-app/+/887637/16/openstack-helm-infra/debian/deb_folder/patches/0016-Add-ServiceAccount-to-openvswitch-pod.patch

TEST PLAN:
PASS - build-pkgs -c -p openstack-helm-infra,openstack-helm
PASS - build-pkgs -c -p stx-openstack-helm-fluxcd
PASS - Upload stx-openstack application
PASS - Apply stx-openstack application

Closes-Bug: 2030749

Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
Change-Id: Ia0c42466cada50cb3af9490f5ff1b36e839a5915
2023-08-08 10:08:48 -03:00
Luan Nunes Utimura 4c238a8063 clients: Make working dir. dyn. and improve chart
In the current state of the `clients` helm chart, the OpenStack clients'
working directory path is *static* and located at `/var/opt/openstack`.

While this directory is suitable for uploading small files -- such as
small Glance images -- it can quickly run out of space, as its maximum
capacity is, approximately, 20GB. This might not be enough for a single
user, and even worse, it wouldn't be for multiple users.

Therefore, the OpenStack clients' working directory needs to be
*dynamic*, which is the primary goal of this change.

With this change, the system administrator can change the default
clients' working directory by simply performing a helm override and
reapplying the app, such as:

  $ system helm-override-update stx-openstack clients openstack \
    --reuse-values --set workingDirectoryPath=/var/opt/another-directory

  $ system application-apply stx-openstack

During the re-apply, if the default working directory didn't contain
additional files in user subdirectories, it's removed to avoid leftovers
(in the same way as after removing the application).

For custom working directories, however, the handling is different.

They are *never* removed by the application, even during re-applies or
after application removal. The reason we don't remove it is because this
custom directory might be shared with other users and/or applications,
and, therefore, potentially have important files that must be preserved.

This change also improves the overall quality of the helm chart, by:
  * Fixing minor bugs;
  * Improving in-code documentation;
  * Raising exceptions on critical failures;
  * Removing functions and modules that are no longer needed;
  * Replacing None-returning functions with Boolean-returning functions,
    to better indicate success or failure status.

Test Plan:

PASS - Build python3-k8sapp-openstack package
PASS - Build stx-openstack-helm-fluxcd package
PASS - Build stx-openstack helm charts

Without exceptions:
-------------------
  PASS - Upload/apply stx-openstack
  PASS - Verify that the working directory was created at
         /var/opt/openstack
  PASS - Perform a helm override:
         $ system helm-override-update stx-openstack clients openstack \
           --reuse-values --set \
           workingDirectoryPath=/var/opt/another-directory
  PASS - Verify that the working directory was created at
        /var/opt/another-directory

With exceptions:
----------------
  PASS - Upload *faulty* stx-openstack
         This faulty version of the application raises, on purpose,
         a KubeAppApplyFailure exception during the creation of
         application specific resources
  PASS - Try to apply stx-openstack and verify that it immediately
         fails and goes to "apply-failed" state

PASS - Remove/delete stx-openstack

Story: 2010774
Task: 48410

Change-Id: Ieb06a93c669d362ecd30bff79947ee4c015cf2b4
Signed-off-by: Luan Nunes Utimura <LuanNunes.Utimura@windriver.com>
2023-08-04 23:32:55 -03:00
Zuul eae2b1066a Merge "Enable TLS for the containerized clients" 2023-08-04 18:58:02 +00:00
Lucas de Ataides dd778c9dfa Enable TLS for the containerized clients
After the introduction of the containerized clients by [1], it was
noted that it was not possible to use them properly if the system had
HTTPS enabled.

This changes enabled the containerized clients to work when HTTPS is
enabled, without any intervention from the user, in the same way as the
platform clients work. For that, a secret had to be created
(secret-ingress-tls.yaml) containing the certificates information, and
this secret had to be mounted on the clients container. For this secret
to be created, a new job had to be created (job-boostrap.yaml) which
required some modifications to the values and static overrides for this
helm chart.

Besides that, some changes were required to the k8sapp-openstack, to
allow dynamic modification of the overrides for this chart when HTTPS
is enabled.

[1] https://review.opendev.org/c/starlingx/openstack-armada-app/+/885603

Test Plan (builds):
PASS: Build stx-openstack-helm-fluxcd and python3-k8sapp-openstack
PASS: Build Helm charts

Test Plan (HTTPS disabled):
PASS: Upload / apply the STX-Openstack application
PASS: Verify that all clients resources are up and running
PASS: Verify that none of the TLS resources were created (secret,
      volume, volume mount)
PASS: Verify that the containerized clients work as intended (issue
      an `openstack user list` after sourcing the /var/opt/openstack/
      admin-openrc file)
PASS: Remove / delete the STX-Openstack application

Test Plan (HTTPS enabled):
PASS: Upload / apply the STX-Openstack application
PASS: Verify that the required system overrides are applied
PASS: Verify that all clients resources are up and running
PASS: Verify that the TLS resources were created (secret, volume,
      volume mount)
PASS: Verify that the containerized clients work as intended (issue
      an `openstack user list` after sourcing the /var/opt/openstack/
      admin-openrc file)
PASS: Remove / delete the STX-Openstack application

Story: 2010774
Task: 48411

Change-Id: I8ff9170b55ae5399553a736ce857b50cc7866dc4
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
2023-08-04 17:53:10 +00:00
Zuul 3da711c839 Merge "Remove Nova prefix" 2023-07-27 16:14:26 +00:00
Md Irshad Sheikh da73bc7582 Remove Nova prefix
In this commit, removed NOVA prefix from the constants as this is a
historical artifact tracing back to an openstack-only solution.

TEST PLAN:

PASS: build success of stx-openstack-helm-fluxcd and
      python3-k8sapp-openstack
      "build-pkgs -c -p stx-openstack-helm-fluxcd"
      "build-pkgs -c -p python3-k8sapp-openstack"
PASS: bootstrap is success

Story: 2010604
Task: 48428

Depends-On: https://review.opendev.org/c/starlingx/config/+/889011
Change-Id: If8b0cc8d0750d11f7604ce419e3837b1e7f7c908
Signed-off-by: Md Irshad Sheikh <mdirshad.sheikh@windriver.com>
2023-07-26 16:44:27 +00:00