starlingx/openstack-armada-app
Code Issues Proposed changes

Allow VMs to be created via volumes

Browse Source 
After STX-Openstack upversioned to Antelope, we noticed that it was not
possible to create VMs by volume, as they would be stuck on ERROR
status. The first idea I had to solve this was to create a patch
containing [1] and [2], because, as specified in [3], Nova now requires
a service token in order to be able to manipulate Cinder volumes. This
unfortunately did not solve the issue by itself, as now an error message
showed up on the nova-conductor pods with the following (not full error
message, only important part): "nova.exception.RescheduledException:
Build of instance 2f32c7ea-1720-4f61-bce8-dbe970c40b0c was re-scheduled:
Secret not found: no secret with matching uuid 'a7f3ae2e-cee7-4f04-9402
-a78047747654". This UUID was not the same one present when issuing
`virsh secret-list` on Cinder, Nova and Libvirt containers.

Turns out openstack-helm and openstack-helm-infra have a Ceph UUID
hardcoded in them, in Cinder [4], Nova [5] [6] and Libvirt [7] values.
By changing these values to the UUID that libvirt was trying to find
(7f3ae2e-cee7-4f04-9402-a78047747654), and it worked to solve the issue.
However, it is not a good practice to use hardcoded values, and,
searching on where this UUID was coming from, it turns out it was
defined by the platform's Ceph configuration under
`/etc/ceph/ceph.conf`.

What this change does is dynamically read this `/etc/ceph/ceph.conf`
file to search for the UUID value, and use it to override the [4] [5]
[6] and [7] values. It also adds the patch including the Nova service
token configuration. The combination of these 2 changes allows VMs to be
created by volumes.

[1] 91c8a5baf2
[2] 7d39af25fd
[3] https://docs.openstack.org/releasenotes/cinder/2023.1.html#upgrade-notes
[4] https://opendev.org/openstack/openstack-helm/src/branch/master/cinder/values.yaml#L942
[5] https://opendev.org/openstack/openstack-helm/src/branch/master/nova/values.yaml#L594
[6] https://opendev.org/openstack/openstack-helm/src/branch/master/nova/values.yaml#L1432
[7] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/libvirt/values.yaml#L100

Test Plan:
PASS: Build openstack-helm, python3-k8sapp-openstack and
      stx-openstack-helm-fluxcd
PASS: Upload / apply / remove STX-Openstack
PASS: Create a VM by an image
PASS: Create a volume and launch a VM from it
PASS: Create a VM using the `boot-from-volume` flag
PASS: Delete a VM created by a volume

Closes-Bug: 2037463

Change-Id: Ia00bb8dbe3460ce817d69049f97f56a96ad6a298
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
This commit is contained in:
Lucas de Ataides 2023-09-28 14:03:49 -03:00
parent 190502fb6b
commit cba7b53fb1
6 changed files with 253 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

192
openstack-helm/debian/deb_folder/patches/0020-Add-service-tokens-for-Cinder-auth.patch Normal file
View File

@ -0,0 +1,192 @@
From b327cf3e379dd1a28e9e1491774c11e1209a1476 Mon Sep 17 00:00:00 2001
From: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
Date: Tue, 26 Sep 2023 16:16:04 -0300
Subject: [PATCH] Add service tokens for Cinder auth
Since version 22.1.0 of Cinder, it requires Nova to configure service
tokens in order to manipulate volumes [1], meaning that if that's not
configured, Nova will not be able to create VMs by volume. The commit
configured for StarlingX's openstack-helm package does not include this
configuration, which was done later by [2] and [3].
This patch includes commits [2] and [3] in order to allow Nova to create
VMs by volumes.
[1] https://docs.openstack.org/releasenotes/cinder/2023.1.html#upgrade-notes
[2] https://opendev.org/openstack/openstack-helm/commit/91c8a5baf2cf2f0dddded57d88f00ea11dd4ff4a
[3] https://opendev.org/openstack/openstack-helm/commit/7d39af25fddbf5fc67e15c92a9265f28567a214e
Signed-off-by: Lucas de Ataides <lucas.deataidesbarreto@windriver.com>
---
cinder/values.yaml | 6 ++++--
glance/values.yaml | 2 ++
neutron/values.yaml | 2 ++
nova/templates/configmap-etc.yaml | 23 +++++++++++++++++++++++
nova/values.yaml | 18 ++++++++++++++++--
placement/values.yaml | 2 ++
6 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/cinder/values.yaml b/cinder/values.yaml
index b95bd618..ddc5d632 100644
--- a/cinder/values.yaml
+++ b/cinder/values.yaml
@@ -826,6 +826,8 @@ conf:
database:
max_retries: -1
keystone_authtoken:
+ service_token_roles: service
+ service_token_roles_required: true
auth_version: v3
auth_type: password
memcache_security_strategy: ENCRYPT
@@ -848,7 +850,7 @@ conf:
backend_url: file:///var/lib/cinder/coordination
service_user:
auth_type: password
- send_service_user_token: false
+ send_service_user_token: true
logging:
loggers:
keys:
@@ -1224,7 +1226,7 @@ endpoints:
user_domain_name: default
project_domain_name: default
cinder:
- role: admin
+ role: admin,service
region_name: RegionOne
username: cinder
password: password
diff --git a/glance/values.yaml b/glance/values.yaml
index d23674bb..d15b77ac 100644
--- a/glance/values.yaml
+++ b/glance/values.yaml
@@ -256,6 +256,8 @@ conf:
oslo_middleware:
enable_proxy_headers_parsing: true
keystone_authtoken:
+ service_token_roles: service
+ service_token_roles_required: true
auth_type: password
auth_version: v3
memcache_security_strategy: ENCRYPT
diff --git a/neutron/values.yaml b/neutron/values.yaml
index 0a84322d..c71a40cf 100644
--- a/neutron/values.yaml
+++ b/neutron/values.yaml
@@ -1809,6 +1809,8 @@ conf:
ironic:
endpoint_type: internal
keystone_authtoken:
+ service_token_roles: service
+ service_token_roles_required: true
memcache_security_strategy: ENCRYPT
auth_type: password
auth_version: v3
diff --git a/nova/templates/configmap-etc.yaml b/nova/templates/configmap-etc.yaml
index 5e3b61d3..1d8fbd97 100644
--- a/nova/templates/configmap-etc.yaml
+++ b/nova/templates/configmap-etc.yaml
@@ -228,6 +228,29 @@ limitations under the License.
{{- $_ := tuple "oslo_cache" "internal" "memcache" . | include "helm-toolkit.endpoints.host_and_port_endpoint_uri_lookup" | set .Values.conf.nova.ironic "memcache_servers" -}}
{{- end -}}
+{{- if empty .Values.conf.nova.cinder.auth_url -}}
+{{- $_ := tuple "identity" "internal" "api" . | include "helm-toolkit.endpoints.keystone_endpoint_uri_lookup" | set .Values.conf.nova.cinder "auth_url" -}}
+{{- end -}}
+
+{{- if empty .Values.conf.nova.cinder.os_region_name -}}
+{{- $_ := set .Values.conf.nova.cinder "os_region_name" .Values.endpoints.identity.auth.cinder.region_name -}}
+{{- end -}}
+{{- if empty .Values.conf.nova.cinder.project_name -}}
+{{- $_ := set .Values.conf.nova.cinder "project_name" .Values.endpoints.identity.auth.cinder.project_name -}}
+{{- end -}}
+{{- if empty .Values.conf.nova.cinder.project_domain_name -}}
+{{- $_ := set .Values.conf.nova.cinder "project_domain_name" .Values.endpoints.identity.auth.cinder.project_domain_name -}}
+{{- end -}}
+{{- if empty .Values.conf.nova.cinder.user_domain_name -}}
+{{- $_ := set .Values.conf.nova.cinder "user_domain_name" .Values.endpoints.identity.auth.cinder.user_domain_name -}}
+{{- end -}}
+{{- if empty .Values.conf.nova.cinder.username -}}
+{{- $_ := set .Values.conf.nova.cinder "username" .Values.endpoints.identity.auth.cinder.username -}}
+{{- end -}}
+{{- if empty .Values.conf.nova.cinder.password -}}
+{{- $_ := set .Values.conf.nova.cinder "password" .Values.endpoints.identity.auth.cinder.password -}}
+{{- end -}}
+
{{- if empty .Values.conf.nova.DEFAULT.osapi_compute_listen_port -}}
{{- $_ := tuple "compute" "service" "api" . | include "helm-toolkit.endpoints.endpoint_port_lookup" | set .Values.conf.nova.DEFAULT "osapi_compute_listen_port" -}}
{{- end -}}
diff --git a/nova/values.yaml b/nova/values.yaml
index 882470c6..63b8d02f 100644
--- a/nova/values.yaml
+++ b/nova/values.yaml
@@ -1402,6 +1402,10 @@ conf:
service_metadata_proxy: True
auth_type: password
auth_version: v3
+ cinder:
+ catalog_info: volumev3::internalURL
+ auth_url: null
+ auth_type: password
database:
max_retries: -1
api_database:
@@ -1409,6 +1413,8 @@ conf:
cell0_database:
max_retries: -1
keystone_authtoken:
+ service_token_roles: service
+ service_token_roles_required: true
auth_type: password
auth_version: v3
memcache_security_strategy: ENCRYPT
@@ -1417,7 +1423,7 @@ conf:
notify_on_state_change: vm_and_task_state
service_user:
auth_type: password
- send_service_user_token: false
+ send_service_user_token: true
libvirt:
connection_uri: "qemu+unix:///system?socket=/run/libvirt/libvirt-sock"
images_type: qcow2
@@ -1708,7 +1714,7 @@ endpoints:
user_domain_name: default
project_domain_name: default
nova:
- role: admin
+ role: admin,service
region_name: RegionOne
username: nova
password: password
@@ -1743,6 +1749,14 @@ endpoints:
project_name: service
user_domain_name: service
project_domain_name: service
+ cinder:
+ role: admin,service
+ region_name: RegionOne
+ username: cinder
+ password: password
+ project_name: service
+ user_domain_name: service
+ project_domain_name: service
test:
role: admin
region_name: RegionOne
diff --git a/placement/values.yaml b/placement/values.yaml
index aa864620..4a702ace 100644
--- a/placement/values.yaml
+++ b/placement/values.yaml
@@ -82,6 +82,8 @@ conf:
placement_database:
connection: null
keystone_authtoken:
+ service_token_roles: service
+ service_token_roles_required: true
auth_version: v3
auth_type: password
memcache_security_strategy: ENCRYPT
--
2.25.1

1
openstack-helm/debian/deb_folder/patches/series
View File

@ -17,3 +17,4 @@
0017-Support-ceph-dev-version-during-pool-creation.patch
0018-Update-charts-requirements-to-use-local-server.patch
0019-Make-nova-address-search-optional.patch
0020-Add-service-tokens-for-Cinder-auth.patch

11
python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/cinder.py
View File

@ -13,6 +13,7 @@ from tsconfig import tsconfig as tsc
from k8sapp_openstack.common import constants as app_constants
from k8sapp_openstack.helm import openstack
from k8sapp_openstack.utils import get_ceph_uuid
ROOK_CEPH_BACKEND_NAME = 'ceph-store'
@ -226,6 +227,10 @@ class CinderHelm(openstack.OpenstackBaseHelm):
constants.SB_TYPE_CEPH_CONF_FILENAME),
}
ceph_uuid = get_ceph_uuid()
if ceph_uuid:
conf_backends[bk_name]['rbd_secret_uuid'] = ceph_uuid
return conf_backends
def _get_endpoints_overrides(self):
@ -357,6 +362,12 @@ class CinderHelm(openstack.OpenstackBaseHelm):
(constants.CEPH_CONF_PATH +
constants.SB_TYPE_CEPH_CONF_FILENAME),
}
ceph_uuid = get_ceph_uuid()
if ceph_uuid:
conf_backends['rbd1']['rbd_secret_uuid'] = ceph_uuid
conf_backends[ROOK_CEPH_BACKEND_NAME]['rbd_secret_uuid'] = ceph_uuid
return conf_backends
def _get_ceph_client_rook_overrides(self):

7
python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/libvirt.py
View File

@ -11,6 +11,7 @@ from sysinv.helm import common
from k8sapp_openstack.common import constants as app_constants
from k8sapp_openstack.helm import openstack
from k8sapp_openstack.utils import get_ceph_uuid
class LibvirtHelm(openstack.OpenstackBaseHelm):
@ -80,6 +81,12 @@ class LibvirtHelm(openstack.OpenstackBaseHelm):
},
}
ceph_uuid = get_ceph_uuid()
if ceph_uuid:
overrides['ceph']['cinder'] = {
'secret_uuid': ceph_uuid,
}
return overrides
def _update_host_addresses(self, host, libvirt_config):

10
python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/nova.py
View File

@ -18,6 +18,7 @@ from sysinv.helm import common
from k8sapp_openstack.common import constants as app_constants
from k8sapp_openstack.helm import openstack
from k8sapp_openstack.utils import get_ceph_uuid
LOG = logging.getLogger(__name__)
@ -799,6 +800,15 @@ class NovaHelm(openstack.OpenstackBaseHelm):
},
}
ceph_uuid = get_ceph_uuid()
if ceph_uuid:
overrides['ceph']['cinder'] = {
'secret_uuid': ceph_uuid,
}
overrides['nova']['libvirt'] = {
'rbd_secret_uuid': ceph_uuid,
}
if self._is_openstack_https_ready():
overrides = self._update_overrides(overrides, {
'nova': {

32
python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/utils.py
View File

@ -8,6 +8,7 @@ from grp import getgrnam
import os
from pathlib import Path
from pwd import getpwnam
import re
import shutil
from typing import Generator
@ -267,3 +268,34 @@ def delete_clients_working_directory(
shutil.rmtree(working_directory, ignore_errors=True)
return True
def get_ceph_uuid():
"""Get Ceph secret UUID for Cinder backend configuration
:returns: str -- The Ceph's secret UUID
"""
ceph_config_file = os.path.join(constants.CEPH_CONF_PATH,
constants.SB_TYPE_CEPH_CONF_FILENAME)
# If the file doesn't exist, return nothing, as not to change
# the default value.
if not os.path.isfile(ceph_config_file):
LOG.warning(f"`{ceph_config_file}` does not exist. Using "
"default configuration.")
return None
# Search for the line that contains the `fsid` parameter, which is
# the Ceph UUID required for Cinder's backend configuration.
with open(ceph_config_file) as file:
line = next((line for line in file if "fsid" in line), None)
if not line:
LOG.warning(f"`{ceph_config_file}` does not contain the "
"'fsid' value. Using default configuration")
return None
# This Regex pattern searches for an UUID
pattern = re.compile(r"[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]"
r"{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}",
re.IGNORECASE)
return pattern.findall(line)[0]