[victoria/ussuri/train] Change nbd, vnc and qemu default cacert file

InternalTLSNbdCAFile, InternalTLSVncCAFile and InternalTLSQemuCAFile
do not point to the default IPA ca.crt file and instead are requested
to be loaded to component specific CA files (even if they are the same).
This can lead to a race where the CA cert is not being written by
certmonger in time and the following issue is seen after the 60s timeout:

May  1 16:47:39 puppet-user: Notice: /Stage[main]/Tripleo::Profile::Base::Certmonger_user/Tripleo::Certmonger::Libvirt_vnc[libvirt-vnc-server-cert]/Certmonger_certificate[libvirt-vnc-server-cert]/ensure: created
May  1 16:48:40 puppet-user: Error: 'test -f /etc/pki/CA/certs/vnc.crt' returned 1 instead of one of [0]
May  1 16:48:40 puppet-user: Error: /Stage[main]/Tripleo::Profile::Base::Certmonger_user/Tripleo::Certmonger::Libvirt_vnc[libvirt-vnc-server-cert]/Exec[/etc/pki/CA/certs/vnc.crt]/returns: change from 'notrun' to ['0'] failed: 'test -f /etc/pki/CA/certs/vnc.crt' returned 1 instead of one of [0]

The problem here is that certmonger doesn't behave in the way that we
expect it to do. When we make the cert request and ask for the ca cert to
be retrieved, it issues the cert and schedules the cert to be returned
asynchronously, even if you specify -w to wait for the cert.  -w will block
pending the cert being retrieved, but not for the CA cert.

You can always force the retrieval to happen by restarting certmonger, and
this has helped in some cases in the past, but is a less than ideal
solution.

This is a bug in certmonger IMHO, in that we should expect the CA cert to
be returned synchronously along with the cert if we specify -w.

The BZ for certmonger is unlikely to be fixed anytime soon though, so we
need to look at other options.

Ib868465c20d97c62cbcb214bfc62d949bd6efc62 already changed the default to
use the IPA system cacert file '/etc/ipa/ca.crt' per default starting with
the wallaby release using the ansible role. This change backports to also
use the IPA system cacert file '/etc/ipa/ca.crt' to previous release when
managing the certs via puppet-tripleo.

Conflicts:
deployment/nova/nova-vnc-proxy-container-puppet.yaml

Change-Id: I8a00ab81c16b21c9b1f703015a2a2eaa66fd556f
Closes-Bug: #1927201
(cherry picked from commit d54d63285d)

deployment/nova/nova-libvirt-container-puppet.yaml

@@ -105,15 +105,15 @@ parameters:
     description: Specifies the default CA cert to use if TLS is used for
                  services in the internal network.
   InternalTLSNbdCAFile:
-    default: '/etc/pki/qemu/ca-cert.pem'
+    default: '/etc/ipa/ca.crt'
     type: string
     description: Specifies the CA cert to use for NBD TLS.
   InternalTLSVncCAFile:
-    default: '/etc/pki/CA/certs/vnc.crt'
+    default: '/etc/ipa/ca.crt'
     type: string
     description: Specifies the CA cert to use for VNC TLS.
   InternalTLSQemuCAFile:
-    default: '/etc/pki/CA/certs/qemu.pem'
+    default: '/etc/ipa/ca.crt'
     type: string
     description: Specifies the CA cert to use for qemu.
   CertificateKeySize:
@@ -311,6 +311,8 @@ conditions:
       - {get_param: LibvirtVncCACert}
       - ''
 
+  libvirt_vnc_ca_non_default: {not: {equals: [{get_param: InternalTLSVncCAFile}, '/etc/ipa/ca.crt']}}
+
   memory_backing_dir_set:
     not:
       and:
@@ -340,6 +342,8 @@ conditions:
       - {get_param: QemuCACert}
       - ''
 
+  qemu_ca_non_default: {not: {equals: [{get_param: InternalTLSQemuCAFile}, '/etc/ipa/ca.crt']}}
+
   key_size_libvirt_override_unset: {equals: [{get_param: LibvirtCertificateKeySize}, '']}
   key_size_libvirtvnc_override_unset: {equals: [{get_param: LibvirtVNCServerCertificateKeySize}, '']}
   key_size_qemu_client_override_unset: {equals: [{get_param: QemuClientCertificateKeySize}, '']}
@@ -510,7 +514,7 @@ outputs:
                   qemu-server-cert:
                     cacertfile:
                       if:
-                        - qemu_specific_ca_unset
+                        - qemu_ca_non_default
                         - get_param: InternalTLSQemuCAFile
                         - null
                     service_certificate: '/etc/pki/qemu/server-cert.pem'
@@ -572,7 +576,7 @@ outputs:
                   libvirt-vnc-server-cert:
                     cacertfile:
                       if:
-                        - libvirt_vnc_specific_ca_unset
+                        - libvirt_vnc_ca_non_default
                         - get_param: InternalTLSVncCAFile
                         - null
                     service_certificate: '/etc/pki/libvirt-vnc/server-cert.pem'

deployment/nova/nova-vnc-proxy-container-puppet.yaml

@@ -51,7 +51,7 @@ parameters:
                  enable TLS transaport for libvirt VNC and configure the
                  relevant keys for libvirt.
   InternalTLSVncProxyCAFile:
-    default: '/etc/pki/CA/certs/vnc.crt'
+    default: '/etc/ipa/ca.crt'
     type: string
     description: Specifies the CA cert to use for VNC TLS.
   CertificateKeySize:
@@ -122,6 +122,8 @@ conditions:
       - {get_param: LibvirtVncCACert}
       - ''
 
+  libvirt_vnc_ca_non_default: {not: {equals: [{get_param: InternalTLSVncProxyCAFile}, '/etc/ipa/ca.crt']}}
+
   allow_noauth:
     # Allow noauth VNC connections during P->Q upgrade. Remove in Rocky.
     equals: [{get_param: StackUpdateType}, 'UPGRADE']
@@ -219,7 +221,7 @@ outputs:
                   libvirt-vnc-client-cert:
                     cacertfile:
                       if:
-                        - libvirt_vnc_specific_ca_unset
+                        - libvirt_vnc_ca_non_default
                         - get_param: InternalTLSVncProxyCAFile
                         - null
                     service_certificate: '/etc/pki/libvirt-vnc/client-cert.pem'

releasenotes/notes/nova_libvirt_default_ca_cert-d1a7acaa5cbc9293.yaml

@@ -0,0 +1,13 @@
+---
+fixes:
+  - |
+    InternalTLSNbdCAFile, InternalTLSVncCAFile and InternalTLSQemuCAFile
+    do not point to the default IPA ca.crt file and instead are requested
+    to be loaded to component specific CA files (even if they are the same).
+    This can lead to a race where the CA cert is not being written by
+    certmonger in time.
+    Ib868465c20d97c62cbcb214bfc62d949bd6efc62 already changed the default to
+    use the IPA system cacert file '/etc/ipa/ca.crt' per default starting with
+    the wallaby release using the ansible role. This change backports to also
+    use the IPA system cacert file '/etc/ipa/ca.crt' to previous release when
+    managing the certs via puppet-tripleo.