openstack/tripleo-heat-templates
Code Issues Proposed changes

Simplify libvirt/qemu ssl certificates

Browse Source 
On the compute nodes, right now ssl certificates got created for
libvirt, qemu-default, qemu-vnc and qemu-nbd. This is not required
because the all services use the same NovaLibvirtNetwork network and
therefore multiple certificates for the same hostname get created.
Also from qemu point of view, if default_tls_x509_cert_dir and
default_tls_x509_verify parameters get set for all certificates, there
is no need to specify any of the other *_tls* config options. From [1]

The intention (of libvirt) is that you can just use the
default_tls_x509_* config attributes so that you don’t need to set any
other *_tls* parameters, unless_ you need different certificates for
some services. The rationale for that is that some services (e.g.
migration / NBD) are only exposed to internal infrastructure; while
some sevices (VNC, Spice) might be exposed publically, so might need
different certificates. For OpenStack this does not matter, though,
we will stick with the defaults.

Therefore with this change InternalTLSNbdCAFile, InternalTLSVncCAFile
and InternalTLSQemuCAFile get removed (which defaulted to
/etc/ipa/ca.crt anyways) and just use InternalTLSCAFile.

Also all cerfificates get created when EnableInternalTLS is true to
and mount all SSL certificates from the host. This is to prevent
certificate information is not available in a qemu's process container
environment if features get switched later, which has shown to be
problematic.

[1] https://docs.openstack.org/nova/latest/admin/secure-live-migration-with-qemu-native-tls.html

Also squashes c904c7555c into this
backport:
Explicit set qemu certificate group ownership

While the certificates get requested with the appropriate group
root:qemu [1] and copied to /etc/pki/qemu/ with -a it has seen
that the group ownership is not correct on the target certificate
files. Lets set explicit group ownership via the run_after
script.

Closes-Bug: #1933330

[1] https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/nova/nova-libvirt-container-puppet.yaml#L777-L779

Change-Id: Ie2c78fc3a07be1cd22cb6cac240047b5d2b9cd0a
(cherry picked from commit d20f295f3a)
This commit is contained in:
Martin Schuppert 2021-05-05 09:36:25 +02:00
parent 74385e9e5e
commit 59a235340c
2 changed files with 103 additions and 163 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

236
deployment/nova/nova-libvirt-container-puppet.yaml
View File

@ -118,18 +118,6 @@ parameters:
type: string
description: Specifies the default CA cert to use if TLS is used for
services in the internal network.
InternalTLSNbdCAFile:
default: '/etc/ipa/ca.crt'
type: string
description: Specifies the CA cert to use for NBD TLS.
InternalTLSVncCAFile:
default: '/etc/ipa/ca.crt'
type: string
description: Specifies the CA cert to use for VNC TLS.
InternalTLSQemuCAFile:
default: '/etc/ipa/ca.crt'
type: string
description: Specifies the CA cert to use for qemu.
CertificateKeySize:
type: string
default: '2048'
@ -140,11 +128,6 @@ parameters:
default: ''
description: Override the private key size used when creating the
certificate for this service
LibvirtVNCServerCertificateKeySize:
type: string
default: ''
description: Override the private key size used when creating the
certificate for this service
QemuServerCertificateKeySize:
type: string
default: ''
@ -174,27 +157,7 @@ parameters:
This file will be symlinked to the default CA path,
which is /etc/pki/qemu/ca-cert.pem.
This parameter should be used if the default (which comes from
the InternalTLSQemuCAFile parameter) is not desired. The current
default reflects TripleO's default CA, which is FreeIPA.
It will only be used if internal TLS is enabled.
LibvirtVncCACert:
type: string
default: ''
description: This specifies the CA certificate to use for VNC TLS.
This file will be symlinked to the default CA path,
which is /etc/pki/libvirt-vnc/ca-cert.pem.
This parameter should be used if the default (which comes from
the InternalTLSVncCAFile parameter) is not desired. The current
default reflects TripleO's default CA, which is FreeIPA.
It will only be used if internal TLS is enabled.
LibvirtNbdCACert:
type: string
default: ''
description: This specifies the CA certificate to use for NBD TLS.
This file will be symlinked to the default CA path,
which is /etc/pki/libvirt-nbd/ca-cert.pem.
This parameter should be used if the default (which comes from
the InternalTLSNbdCAFile parameter) is not desired. The current
the InternalTLSCAFile parameter) is not desired. The current
default reflects TripleO's default CA, which is FreeIPA.
It will only be used if internal TLS is enabled.
VhostuserSocketGroup:
@ -340,14 +303,10 @@ conditions:
and:
- {get_param: EnableInternalTLS}
- {get_param: UseTLSTransportForNbd}
libvirt_nbd_specific_ca_set:
not: {equals: [{get_param: LibvirtNbdCACert}, '']}
qemu_specific_ca_set:
not: {equals: [{get_param: QemuCACert}, '']}
key_size_libvirt_override_set:
not: {equals: [{get_param: LibvirtCertificateKeySize}, '']}
key_size_libvirtvnc_override_set:
not: {equals: [{get_param: LibvirtVNCServerCertificateKeySize}, '']}
key_size_qemu_client_override_set:
not: {equals: [{get_param: QemuClientCertificateKeySize}, '']}
key_size_qemu_server_override_set:
@ -500,11 +459,6 @@ outputs:
dest: "/"
merge: true
preserve_properties: true
- source: "/var/lib/kolla/config_files/src-tls/*"
dest: "/"
merge: true
preserve_properties: true
optional: true
- source: "/var/lib/kolla/config_files/src-ceph/"
dest: "/etc/ceph/"
merge: true
@ -671,41 +625,10 @@ outputs:
- /sys/fs/selinux:/sys/fs/selinux
- /etc/selinux/config:/etc/selinux/config:ro
- if:
- use_tls_for_live_migration
- - /etc/pki/libvirt:/etc/pki/libvirt/:ro
- /etc/pki/libvirt-nbd:/etc/pki/libvirt-nbd:ro
- str_replace:
template: "CACERT:/etc/pki/CA/cacert.pem:ro"
params:
CACERT:
if:
- libvirt_specific_ca_set
- get_param: LibvirtCACert
- get_param: InternalTLSCAFile
- str_replace:
template: "CACERT:/etc/pki/qemu/ca-cert.pem:ro"
params:
CACERT:
if:
- libvirt_nbd_specific_ca_set
- get_param: LibvirtNbdCACert
- get_param: InternalTLSNbdCAFile
- /etc/pki/qemu/server-cert.pem:/etc/pki/qemu/server-cert.pem:ro
- /etc/pki/qemu/server-key.pem:/etc/pki/qemu/server-key.pem:ro
- /etc/pki/qemu/server-cert.pem:/etc/pki/qemu/client-cert.pem:ro
- /etc/pki/qemu/server-key.pem:/etc/pki/qemu/client-key.pem:ro
- if:
- use_tls_for_vnc
- - /etc/pki/libvirt-vnc/server-cert.pem:/etc/pki/libvirt-vnc/server-cert.pem:ro
- /etc/pki/libvirt-vnc/server-key.pem:/etc/pki/libvirt-vnc/server-key.pem:ro
- str_replace:
template: "CACERT:/etc/pki/libvirt-vnc/ca-cert.pem:ro"
params:
CACERT:
if:
- libvirt_vnc_specific_ca_set
- get_param: LibvirtVncCACert
- get_param: InternalTLSVncCAFile
- {get_param: EnableInternalTLS}
- - /etc/pki/CA/cacert.pem:/etc/pki/CA/cacert.pem:ro
- /etc/pki/libvirt:/etc/pki/libvirt:ro
- /etc/pki/qemu:/etc/pki/qemu:ro
- if:
- memory_backing_dir_set
- - str_replace:
@ -776,7 +699,7 @@ outputs:
- not container_healthcheck_disabled
- step|int == 4
- if:
- use_tls_for_live_migration
- {get_param: EnableInternalTLS}
- - name: Certificate generation
when: step|int == 1
block:
@ -791,7 +714,6 @@ outputs:
- '/etc/pki/libvirt'
- '/etc/pki/libvirt/private'
- '/etc/pki/qemu'
- '/etc/pki/libvirt-nbd'
- include_role:
name: linux-system-roles.certificate
vars:
@ -807,14 +729,25 @@ outputs:
template: "libvirt/{{fqdn_$NETWORK}}@{{idm_realm}}"
params:
$NETWORK: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
run_after: |
# Copy cert and key to libvirt dirs
cp /etc/pki/tls/certs/libvirt-server-cert.crt /etc/pki/libvirt/servercert.pem
cp /etc/pki/tls/private/libvirt-server-cert.key /etc/pki/libvirt/private/serverkey.pem
systemctl reload libvirtd
run_after:
str_replace:
template: |
# Copy cert and key to libvirt dirs
cp CACERT /etc/pki/CA/cacert.pem
chown root:root /etc/pki/CA/cacert.pem
chmod 644 /etc/pki/CA/cacert.pem
cp /etc/pki/tls/certs/libvirt-server-cert.crt /etc/pki/libvirt/servercert.pem
cp /etc/pki/tls/private/libvirt-server-cert.key /etc/pki/libvirt/private/serverkey.pem
systemctl reload tripleo_nova_libvirt
params:
CACERT:
if:
- libvirt_specific_ca_set
- get_param: LibvirtCACert
- get_param: InternalTLSCAFile
key_size:
if:
- key_size_libvirtvnc_override_set
- key_size_libvirt_override_set
- {get_param: LibvirtCertificateKeySize}
- {get_param: CertificateKeySize}
ca: ipa
@ -833,14 +766,54 @@ outputs:
# Copy cert and key to libvirt dirs
cp /etc/pki/tls/certs/libvirt-client-cert.crt /etc/pki/libvirt/clientcert.pem
cp /etc/pki/tls/private/libvirt-client-cert.key /etc/pki/libvirt/private/clientkey.pem
systemctl reload libvirtd
systemctl reload tripleo_nova_libvirt
key_size:
if:
- key_size_libvirtvnc_override_set
- key_size_libvirt_override_set
- {get_param: LibvirtCertificateKeySize}
- {get_param: CertificateKeySize}
ca: ipa
- name: qemu-server-cert
owner: root
group: qemu
dns:
str_replace:
template: "{{fqdn_$NETWORK}}"
params:
$NETWORK: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
principal:
str_replace:
template: "qemu/{{fqdn_$NETWORK}}@{{idm_realm}}"
params:
$NETWORK: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
run_after:
str_replace:
template: |
# Copy cert and key to qemu dir
cp CACERT /etc/pki/qemu/ca-cert.pem
chown root:root /etc/pki/qemu/ca-cert.pem
chmod 644 /etc/pki/qemu/ca-cert.pem
cp -a /etc/pki/tls/certs/qemu-server-cert.crt /etc/pki/qemu/server-cert.pem
cp -a /etc/pki/tls/private/qemu-server-cert.key /etc/pki/qemu/server-key.pem
chgrp qemu /etc/pki/qemu/server-*
chmod 0640 /etc/pki/qemu/server-cert.pem
chmod 0640 /etc/pki/qemu/server-key.pem
systemctl reload tripleo_nova_libvirt
params:
CACERT:
if:
- qemu_specific_ca_set
- get_param: QemuCACert
- get_param: InternalTLSCAFile
key_size:
if:
- key_size_qemu_server_override_set
- {get_param: QemuServerCertificateKeySize}
- {get_param: CertificateKeySize}
ca: ipa
- name: qemu-client-cert
owner: root
group: qemu
dns:
str_replace:
template: "{{fqdn_$NETWORK}}"
@ -853,79 +826,18 @@ outputs:
$NETWORK: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
run_after: |
# Copy cert and key to qemu dir
cp /etc/pki/tls/certs/qemu-server-cert.crt /etc/pki/qemu/server-cert.pem
cp /etc/pki/tls/private/qemu-server-cert.key /etc/pki/qemu/server-key.pem
systemctl reload libvirtd
key_size:
if:
- key_size_qemu_server_override_set
- {get_param: QemuServerCertificateKeySize}
- {get_param: CertificateKeySize}
ca: ipa
- name: qemu-nbd-client-cert
dns:
str_replace:
template: "{{fqdn_$NETWORK}}"
params:
$NETWORK: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
principal:
str_replace:
template: "qemu/{{fqdn_$NETWORK}}@{{idm_realm}}"
params:
$NETWORK: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
run_after: |
# Copy cert and key to libvirt-nbd dir
cp /etc/pki/tls/certs/qemu-nbd-client-cert.crt /etc/pki/libvirt-nbd/client-cert.pem
cp /etc/pki/tls/private/qemu-nbd-client-cert.key /etc/pki/libvirt-nbd/client-key.pem
systemctl reload libvirtd
cp -a /etc/pki/tls/certs/qemu-client-cert.crt /etc/pki/qemu/client-cert.pem
cp -a /etc/pki/tls/private/qemu-client-cert.key /etc/pki/qemu/client-key.pem
chgrp qemu /etc/pki/qemu/client-*
chmod 0640 /etc/pki/qemu/client-cert.pem
chmod 0640 /etc/pki/qemu/client-key.pem
systemctl reload tripleo_nova_libvirt
key_size:
if:
- key_size_qemu_client_override_set
- {get_param: QemuClientCertificateKeySize}
- {get_param: CertificateKeySize}
ca: ipa
- if:
- use_tls_for_vnc
- - name: Create dirs for certificates and keys
file:
path: "{{ item }}"
state: directory
serole: object_r
setype: cert_t
seuser: system_u
with_items:
- '/etc/pki/libvirt-vnc'
- name: Certificate generation
when: step|int == 1
block:
- include_role:
name: linux-system-roles.certificate
vars:
certificate_requests:
- name: libvirt-vnc-server-cert
dns:
str_replace:
template: "{{fqdn_$NETWORK}}"
params:
$NETWORK: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
principal:
str_replace:
template: "libvirt-vnc/{{fqdn_$NETWORK}}@{{idm_realm}}"
params:
$NETWORK: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
run_after: |
# Copy cert and key to libvirt-vnc dir
cp /etc/pki/tls/certs/libvirt-vnc-server-cert.crt /etc/pki/libvirt-vnc/server-cert.pem
cp /etc/pki/tls/private/libvirt-vnc-server-cert.key /etc/pki/libvirt-vnc/server-key.pem
chmod 0644 /etc/pki/libvirt-vnc/server-cert.pem
chmod 0640 /etc/pki/libvirt-vnc/server-key.pem
chgrp qemu /etc/pki/libvirt-vnc/server-key.pem
key_size:
if:
- key_size_libvirtvnc_override_set
- {get_param: LibvirtVNCServerCertificateKeySize}
- {get_param: CertificateKeySize}
ca: ipa
host_prep_tasks:
list_concat:
- {get_attr: [NovaLibvirtLogging, host_prep_tasks]}
@ -998,16 +910,14 @@ outputs:
metadata_settings:
list_concat:
- if:
- use_tls_for_live_migration
- {get_param: EnableInternalTLS}
- - service: libvirt
network: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
type: node
- service: qemu
network: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
type: node
- if:
- use_tls_for_vnc
- - service: libvirt-vnc
- service: libvirt-vnc
network: {get_param: [ServiceNetMap, NovaLibvirtNetwork]}
type: node
upgrade_tasks:

30
releasenotes/notes/nova_libvirt_ssl_cert_simplification-dbee541be9f55ce5.yaml Normal file
View File

@ -0,0 +1,30 @@
---
fixes:
- |
On the compute nodes, right now ssl certificates got created for
libvirt, qemu-default, qemu-vnc and qemu-nbd. This is not required
because the all services use the same NovaLibvirtNetwork network and
therefore multiple certificates for the same hostname get created.
Also from qemu point of view, if default_tls_x509_cert_dir and
default_tls_x509_verify parameters get set for all certificates, there
is no need to specify any of the other `*_tls*` config options. From
`Secure live migration with QEMU-native TLS <https://docs.openstack.org/nova/latest/admin/secure-live-migration-with-qemu-native-tls.html>`_
The intention (of libvirt) is that you can just use the
default_tls_x509_* config attributes so that you don’t need to set any
other `*_tls*` parameters, unless you need different certificates for
some services. The rationale for that is that some services (e.g.
migration / NBD) are only exposed to internal infrastructure; while
some sevices (VNC, Spice) might be exposed publically, so might need
different certificates. For OpenStack this does not matter, though,
we will stick with the defaults.
Therefore with this change InternalTLSNbdCAFile, InternalTLSVncCAFile
and InternalTLSQemuCAFile get removed (which defaulted to
/etc/ipa/ca.crt anyways) and just use InternalTLSCAFile.
Also all cerfificates get created when EnableInternalTLS is true to
and mount all SSL certificates from the host. This is to prevent
certificate information is not available in a qemu's process container
environment if features get switched later, which has shown to be
problematic.