0f54889408
There are certain HA clustered services (e.g. galera) that don't have the ability natively to reload their TLS certificate without being restarted. If too many replicas are restarted concurrently this might result in full service disruption. To ensure service availability, provide a means to ensure that only one service replica is restarted at a time in the cluster. This works by using pacemaker's CIB to implement a cluster-wide restart lock for a service. The lock has a TTL so it's guaranteed to be eventually released without requiring complex contingency cleanup in case of failures. Tested locally by running the following: 1. force recreate certificate on all nodes at once for galera (ipa-cert resubmit -i mysql), and verify that the resources restart one after the other 2. create a lock manually in pacemaker, recreate certificate for galera on all nodes, and verify that no resource is restarted before the manually created lock expires. 3. create a lock manually, let it expires, recreate a certificate, and verify that the resource is restarted appropriately and the lock gets cleaned up from pacemaker once the restart finished. Closes-Bug: #1885113 Change-Id: Ib2b62e33b34cf72edfdae6299cf432259bf960a2
84 lines
2.6 KiB
YAML
84 lines
2.6 KiB
YAML
heat_template_version: rocky
|
|
|
|
description: >
|
|
Requests certificates using certmonger through Puppet
|
|
|
|
parameters:
|
|
ServiceData:
|
|
default: {}
|
|
description: Dictionary packing service data
|
|
type: json
|
|
ServiceNetMap:
|
|
default: {}
|
|
description: Mapping of service_name -> network name. Typically set
|
|
via parameter_defaults in the resource registry. This
|
|
mapping overrides those in ServiceNetMapDefaults.
|
|
type: json
|
|
DefaultPasswords:
|
|
default: {}
|
|
type: json
|
|
RoleName:
|
|
default: ''
|
|
description: Role name on which the service is applied
|
|
type: string
|
|
RoleParameters:
|
|
default: {}
|
|
description: Parameters specific to the role
|
|
type: json
|
|
EndpointMap:
|
|
default: {}
|
|
description: Mapping of service endpoint -> protocol. Typically set
|
|
via parameter_defaults in the resource registry.
|
|
type: json
|
|
EnableInternalTLS:
|
|
type: boolean
|
|
default: false
|
|
DefaultCRLURL:
|
|
default: 'http://ipa-ca/ipa/crl/MasterCRL.bin'
|
|
description: URI where to get the CRL to be configured in the nodes.
|
|
type: string
|
|
# NOTE(jaosorior): This is being set as IPA as it's the first
|
|
# CA we'll actually be testing out. But we can change this if
|
|
# people request it.
|
|
CertmongerCA:
|
|
type: string
|
|
default: 'IPA'
|
|
# TODO: default to a dedicated CA once the ipa sub-CA setup has been
|
|
# automated and upgrades are addressed
|
|
CertmongerVncCA:
|
|
type: string
|
|
default: 'IPA'
|
|
CertmongerQemuCA:
|
|
type: string
|
|
default: 'IPA'
|
|
|
|
conditions:
|
|
|
|
internal_tls_enabled: {equals: [{get_param: EnableInternalTLS}, true]}
|
|
|
|
outputs:
|
|
role_data:
|
|
description: Role data for the certmonger-user service
|
|
value:
|
|
service_name: certmonger_user
|
|
config_settings:
|
|
map_merge:
|
|
- certmonger_ca: {get_param: CertmongerCA}
|
|
- if:
|
|
- internal_tls_enabled
|
|
- tripleo::certmonger::ca::crl::crl_source: {get_param: DefaultCRLURL}
|
|
certmonger_ca_vnc: {get_param: CertmongerVncCA}
|
|
certmonger_ca_qemu: {get_param: CertmongerQemuCA}
|
|
- {}
|
|
step_config: |
|
|
include tripleo::profile::base::certmonger_user
|
|
host_prep_tasks:
|
|
- name: create certificate rotation script for HA services
|
|
copy:
|
|
dest: /usr/bin/certmonger-ha-resource-refresh.sh
|
|
setype: certmonger_unconfined_exec_t
|
|
mode: "0700"
|
|
content: |
|
|
#!/bin/bash
|
|
/var/lib/container-config-scripts/pacemaker_mutex_restart_bundle.sh --lock $* 2>&1 | logger -t certmonger
|