system-config/testinfra/test_letsencrypt.py

164 lines
5.9 KiB
Python
Raw Normal View History

letsencrypt support This change contains the roles and testing for deploying certificates on hosts using letsencrypt with domain authentication. From a top level, the process is implemented in the roles as follows: 1) letsencrypt-acme-sh-install This role installs the acme.sh tool on hosts in the letsencrypt group, along with a small custom driver script to help parse output that is used by later roles. 2) letsencrypt-request-certs This role runs on each host, and reads a host variable describing the certificates required. It uses the acme.sh tool (via the driver) to request the certificates from letsencrypt. It populates a global Ansible variable with the authentication TXT records required. If the certificate exists on the host and is not within the renewal period, it should do nothing. 3) letsencrypt-install-txt-record This role runs on the adns server. It installs the TXT records generated in step 2 to the acme.opendev.org domain and then refreshes the server. Hosts wanting certificates will have pre-provisioned CNAME records for _acme-challenge.host.opendev.org pointing to acme.opendev.org. 4) letsencrypt-create-certs This role runs on each host, reading the same variable as in step 2. However this time the acme.sh tool is run to authenticate and create the certificates, which should now work correctly via the TXT records from step 3. After this, the host will have the full certificate material. Testing is added via testinfra. For testing purposes requests are made to the staging letsencrypt servers and a self-signed certificate is provisioned in step 4 (as the authentication is not available during CI). We test that the DNS TXT records are created locally on the CI adns server, however. Related-Spec: https://review.openstack.org/587283 Change-Id: I1f66da614751a29cc565b37cdc9ff34d70fdfd3f
2019-02-14 08:10:51 +11:00
# Copyright 2019 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import pytest
testinfra_hosts = ['adns-letsencrypt.opendev.org',
Generate ssl check list directly from letsencrypt variables This autogenerates the list of ssl domains for the ssl-cert-check tool directly from the letsencrypt list. The first step is the install-certcheck role that replaces the puppet-ssl_cert_check module that does the same. The reason for this is so that during gate testing we can test this on the test bridge.openstack.org server, and avoid adding another node as a requirement for this test. letsencrypt-request-certs is updated to set a fact letsencrypt_certcheck_domains for each host that is generating a certificate. As described in the comments, this defaults to the first host specified for the certificate and the listening port can be indicated (if set, this new port value is stripped when generating certs as is not necessary for certificate generation). The new letsencrypt-config-certcheck role runs and iterates all letsencrypt hosts to build the final list of domains that should be checked. This is then extended with the letsencrypt_certcheck_additional_domains value that covers any hosts using certificates not provisioned by letsencrypt using this mechanism. These additional domains are pre-populated from the openstack.org domains in the extant check file, minus those openstack.org domain certificates we are generating via letsencrypt (see letsencrypt-create-certs/handlers/main.yaml). Additionally, we update some of the certificate variables in host_vars that are listening on port !443. As mentioned, bridge.openstack.org is placed in the new certcheck group for gate testing, so the tool and config file will be deployed to it. For production, cacti is added to the group, which is where the tool currently runs. The extant puppet installation is disabled, pending removal in a follow-on change. Change-Id: Idbe084f13f3684021e8efd9ac69b63fe31484606
2020-05-18 14:43:16 +10:00
'bridge.openstack.org',
letsencrypt support This change contains the roles and testing for deploying certificates on hosts using letsencrypt with domain authentication. From a top level, the process is implemented in the roles as follows: 1) letsencrypt-acme-sh-install This role installs the acme.sh tool on hosts in the letsencrypt group, along with a small custom driver script to help parse output that is used by later roles. 2) letsencrypt-request-certs This role runs on each host, and reads a host variable describing the certificates required. It uses the acme.sh tool (via the driver) to request the certificates from letsencrypt. It populates a global Ansible variable with the authentication TXT records required. If the certificate exists on the host and is not within the renewal period, it should do nothing. 3) letsencrypt-install-txt-record This role runs on the adns server. It installs the TXT records generated in step 2 to the acme.opendev.org domain and then refreshes the server. Hosts wanting certificates will have pre-provisioned CNAME records for _acme-challenge.host.opendev.org pointing to acme.opendev.org. 4) letsencrypt-create-certs This role runs on each host, reading the same variable as in step 2. However this time the acme.sh tool is run to authenticate and create the certificates, which should now work correctly via the TXT records from step 3. After this, the host will have the full certificate material. Testing is added via testinfra. For testing purposes requests are made to the staging letsencrypt servers and a self-signed certificate is provisioned in step 4 (as the authentication is not available during CI). We test that the DNS TXT records are created locally on the CI adns server, however. Related-Spec: https://review.openstack.org/587283 Change-Id: I1f66da614751a29cc565b37cdc9ff34d70fdfd3f
2019-02-14 08:10:51 +11:00
'letsencrypt01.opendev.org',
'letsencrypt02.opendev.org']
def test_acme_zone(host):
if host.backend.get_hostname() != 'adns-letsencrypt.opendev.org':
pytest.skip()
acme_opendev_zone = host.file('/var/lib/bind/zones/acme.opendev.org/zone.db')
assert acme_opendev_zone.exists
# On our test nodes, unbound is listening on 127.0.0.1:53; this
# ensures the query hits bind
query_addr = host.ansible("setup")["ansible_facts"]["ansible_default_ipv4"]["address"]
cmd = host.run("dig -t txt acme.opendev.org @" + query_addr)
count = 0
for line in cmd.stdout.split('\n'):
if line.startswith('acme.opendev.org. 60 IN TXT'):
count = count + 1
if count != 6:
# NOTE(ianw): I'm sure there's more pytest-y ways to save this
# for debugging ...
print(cmd.stdout)
assert count == 6, "Did not see required number of TXT records!"
def test_certs_created(host):
if host.backend.get_hostname() == 'letsencrypt01.opendev.org':
domain_one = host.file(
'/etc/letsencrypt-certs/'
'letsencrypt01.opendev.org/letsencrypt01.opendev.org.key')
assert domain_one.exists
assert domain_one.user == "root"
assert domain_one.group == "letsencrypt"
assert domain_one.mode == 0o640
cert_one = host.file(
'/etc/letsencrypt-certs/'
'letsencrypt01.opendev.org/letsencrypt01.opendev.org.cer')
assert cert_one.exists
assert cert_one.user == "root"
assert cert_one.group == "letsencrypt"
assert cert_one.mode == 0o640
ca_one = host.file(
'/etc/letsencrypt-certs/'
'letsencrypt01.opendev.org/ca.cer')
assert ca_one.exists
assert ca_one.user == "root"
assert ca_one.group == "letsencrypt"
assert ca_one.mode == 0o640
letsencrypt support This change contains the roles and testing for deploying certificates on hosts using letsencrypt with domain authentication. From a top level, the process is implemented in the roles as follows: 1) letsencrypt-acme-sh-install This role installs the acme.sh tool on hosts in the letsencrypt group, along with a small custom driver script to help parse output that is used by later roles. 2) letsencrypt-request-certs This role runs on each host, and reads a host variable describing the certificates required. It uses the acme.sh tool (via the driver) to request the certificates from letsencrypt. It populates a global Ansible variable with the authentication TXT records required. If the certificate exists on the host and is not within the renewal period, it should do nothing. 3) letsencrypt-install-txt-record This role runs on the adns server. It installs the TXT records generated in step 2 to the acme.opendev.org domain and then refreshes the server. Hosts wanting certificates will have pre-provisioned CNAME records for _acme-challenge.host.opendev.org pointing to acme.opendev.org. 4) letsencrypt-create-certs This role runs on each host, reading the same variable as in step 2. However this time the acme.sh tool is run to authenticate and create the certificates, which should now work correctly via the TXT records from step 3. After this, the host will have the full certificate material. Testing is added via testinfra. For testing purposes requests are made to the staging letsencrypt servers and a self-signed certificate is provisioned in step 4 (as the authentication is not available during CI). We test that the DNS TXT records are created locally on the CI adns server, however. Related-Spec: https://review.openstack.org/587283 Change-Id: I1f66da614751a29cc565b37cdc9ff34d70fdfd3f
2019-02-14 08:10:51 +11:00
domain_two = host.file(
'/etc/letsencrypt-certs/'
'someotherservice.opendev.org/someotherservice.opendev.org.key')
assert domain_two.exists
assert domain_two.user == "root"
assert domain_two.group == "letsencrypt"
assert domain_two.mode == 0o640
letsencrypt support This change contains the roles and testing for deploying certificates on hosts using letsencrypt with domain authentication. From a top level, the process is implemented in the roles as follows: 1) letsencrypt-acme-sh-install This role installs the acme.sh tool on hosts in the letsencrypt group, along with a small custom driver script to help parse output that is used by later roles. 2) letsencrypt-request-certs This role runs on each host, and reads a host variable describing the certificates required. It uses the acme.sh tool (via the driver) to request the certificates from letsencrypt. It populates a global Ansible variable with the authentication TXT records required. If the certificate exists on the host and is not within the renewal period, it should do nothing. 3) letsencrypt-install-txt-record This role runs on the adns server. It installs the TXT records generated in step 2 to the acme.opendev.org domain and then refreshes the server. Hosts wanting certificates will have pre-provisioned CNAME records for _acme-challenge.host.opendev.org pointing to acme.opendev.org. 4) letsencrypt-create-certs This role runs on each host, reading the same variable as in step 2. However this time the acme.sh tool is run to authenticate and create the certificates, which should now work correctly via the TXT records from step 3. After this, the host will have the full certificate material. Testing is added via testinfra. For testing purposes requests are made to the staging letsencrypt servers and a self-signed certificate is provisioned in step 4 (as the authentication is not available during CI). We test that the DNS TXT records are created locally on the CI adns server, however. Related-Spec: https://review.openstack.org/587283 Change-Id: I1f66da614751a29cc565b37cdc9ff34d70fdfd3f
2019-02-14 08:10:51 +11:00
cert_two = host.file(
'/etc/letsencrypt-certs/'
'someotherservice.opendev.org/someotherservice.opendev.org.cer')
assert cert_two.exists
assert cert_two.user == "root"
assert cert_two.group == "letsencrypt"
assert cert_two.mode == 0o640
ca_two = host.file(
'/etc/letsencrypt-certs/'
'someotherservice.opendev.org/ca.cer')
assert ca_one.exists
assert ca_one.user == "root"
assert ca_one.group == "letsencrypt"
assert ca_one.mode == 0o640
letsencrypt support This change contains the roles and testing for deploying certificates on hosts using letsencrypt with domain authentication. From a top level, the process is implemented in the roles as follows: 1) letsencrypt-acme-sh-install This role installs the acme.sh tool on hosts in the letsencrypt group, along with a small custom driver script to help parse output that is used by later roles. 2) letsencrypt-request-certs This role runs on each host, and reads a host variable describing the certificates required. It uses the acme.sh tool (via the driver) to request the certificates from letsencrypt. It populates a global Ansible variable with the authentication TXT records required. If the certificate exists on the host and is not within the renewal period, it should do nothing. 3) letsencrypt-install-txt-record This role runs on the adns server. It installs the TXT records generated in step 2 to the acme.opendev.org domain and then refreshes the server. Hosts wanting certificates will have pre-provisioned CNAME records for _acme-challenge.host.opendev.org pointing to acme.opendev.org. 4) letsencrypt-create-certs This role runs on each host, reading the same variable as in step 2. However this time the acme.sh tool is run to authenticate and create the certificates, which should now work correctly via the TXT records from step 3. After this, the host will have the full certificate material. Testing is added via testinfra. For testing purposes requests are made to the staging letsencrypt servers and a self-signed certificate is provisioned in step 4 (as the authentication is not available during CI). We test that the DNS TXT records are created locally on the CI adns server, however. Related-Spec: https://review.openstack.org/587283 Change-Id: I1f66da614751a29cc565b37cdc9ff34d70fdfd3f
2019-02-14 08:10:51 +11:00
elif host.backend.get_hostname() == 'letsencrypt02.opendev.org':
domain_one = host.file(
'/etc/letsencrypt-certs/'
'letsencrypt02.opendev.org/letsencrypt02.opendev.org.key')
assert domain_one.exists
assert domain_one.user == "root"
assert domain_one.group == "letsencrypt"
assert domain_one.mode == 0o640
letsencrypt support This change contains the roles and testing for deploying certificates on hosts using letsencrypt with domain authentication. From a top level, the process is implemented in the roles as follows: 1) letsencrypt-acme-sh-install This role installs the acme.sh tool on hosts in the letsencrypt group, along with a small custom driver script to help parse output that is used by later roles. 2) letsencrypt-request-certs This role runs on each host, and reads a host variable describing the certificates required. It uses the acme.sh tool (via the driver) to request the certificates from letsencrypt. It populates a global Ansible variable with the authentication TXT records required. If the certificate exists on the host and is not within the renewal period, it should do nothing. 3) letsencrypt-install-txt-record This role runs on the adns server. It installs the TXT records generated in step 2 to the acme.opendev.org domain and then refreshes the server. Hosts wanting certificates will have pre-provisioned CNAME records for _acme-challenge.host.opendev.org pointing to acme.opendev.org. 4) letsencrypt-create-certs This role runs on each host, reading the same variable as in step 2. However this time the acme.sh tool is run to authenticate and create the certificates, which should now work correctly via the TXT records from step 3. After this, the host will have the full certificate material. Testing is added via testinfra. For testing purposes requests are made to the staging letsencrypt servers and a self-signed certificate is provisioned in step 4 (as the authentication is not available during CI). We test that the DNS TXT records are created locally on the CI adns server, however. Related-Spec: https://review.openstack.org/587283 Change-Id: I1f66da614751a29cc565b37cdc9ff34d70fdfd3f
2019-02-14 08:10:51 +11:00
cert_one = host.file(
'/etc/letsencrypt-certs/'
'letsencrypt02.opendev.org/letsencrypt02.opendev.org.cer')
assert cert_one.exists
assert cert_one.user == "root"
assert cert_one.group == "letsencrypt"
assert cert_one.mode == 0o640
ca_one = host.file(
'/etc/letsencrypt-certs/'
'letsencrypt02.opendev.org/ca.cer')
assert ca_one.exists
assert ca_one.user == "root"
assert ca_one.group == "letsencrypt"
assert ca_one.mode == 0o640
letsencrypt support This change contains the roles and testing for deploying certificates on hosts using letsencrypt with domain authentication. From a top level, the process is implemented in the roles as follows: 1) letsencrypt-acme-sh-install This role installs the acme.sh tool on hosts in the letsencrypt group, along with a small custom driver script to help parse output that is used by later roles. 2) letsencrypt-request-certs This role runs on each host, and reads a host variable describing the certificates required. It uses the acme.sh tool (via the driver) to request the certificates from letsencrypt. It populates a global Ansible variable with the authentication TXT records required. If the certificate exists on the host and is not within the renewal period, it should do nothing. 3) letsencrypt-install-txt-record This role runs on the adns server. It installs the TXT records generated in step 2 to the acme.opendev.org domain and then refreshes the server. Hosts wanting certificates will have pre-provisioned CNAME records for _acme-challenge.host.opendev.org pointing to acme.opendev.org. 4) letsencrypt-create-certs This role runs on each host, reading the same variable as in step 2. However this time the acme.sh tool is run to authenticate and create the certificates, which should now work correctly via the TXT records from step 3. After this, the host will have the full certificate material. Testing is added via testinfra. For testing purposes requests are made to the staging letsencrypt servers and a self-signed certificate is provisioned in step 4 (as the authentication is not available during CI). We test that the DNS TXT records are created locally on the CI adns server, however. Related-Spec: https://review.openstack.org/587283 Change-Id: I1f66da614751a29cc565b37cdc9ff34d70fdfd3f
2019-02-14 08:10:51 +11:00
else:
pytest.skip()
def test_updated_handler(host):
if host.backend.get_hostname() == 'letsencrypt01.opendev.org':
stamp_file = host.file('/tmp/letsencrypt01-main-service.stamp')
assert stamp_file.exists
stamp_file = host.file('/tmp/letsencrypt01-other-service.stamp')
assert stamp_file.exists
elif host.backend.get_hostname() == 'letsencrypt02.opendev.org':
stamp_file = host.file('/tmp/letsencrypt02-main-service.stamp')
assert stamp_file.exists
else:
pytest.skip()
def test_acme_sh_config(host):
if not host.backend.get_hostname().startswith('letsencrypt0'):
pytest.skip()
config = host.file('/root/.acme.sh/account.conf')
assert config.exists
assert config.contains("^ACCOUNT_EMAIL=le-test@opendev.org")
Generate ssl check list directly from letsencrypt variables This autogenerates the list of ssl domains for the ssl-cert-check tool directly from the letsencrypt list. The first step is the install-certcheck role that replaces the puppet-ssl_cert_check module that does the same. The reason for this is so that during gate testing we can test this on the test bridge.openstack.org server, and avoid adding another node as a requirement for this test. letsencrypt-request-certs is updated to set a fact letsencrypt_certcheck_domains for each host that is generating a certificate. As described in the comments, this defaults to the first host specified for the certificate and the listening port can be indicated (if set, this new port value is stripped when generating certs as is not necessary for certificate generation). The new letsencrypt-config-certcheck role runs and iterates all letsencrypt hosts to build the final list of domains that should be checked. This is then extended with the letsencrypt_certcheck_additional_domains value that covers any hosts using certificates not provisioned by letsencrypt using this mechanism. These additional domains are pre-populated from the openstack.org domains in the extant check file, minus those openstack.org domain certificates we are generating via letsencrypt (see letsencrypt-create-certs/handlers/main.yaml). Additionally, we update some of the certificate variables in host_vars that are listening on port !443. As mentioned, bridge.openstack.org is placed in the new certcheck group for gate testing, so the tool and config file will be deployed to it. For production, cacti is added to the group, which is where the tool currently runs. The extant puppet installation is disabled, pending removal in a follow-on change. Change-Id: Idbe084f13f3684021e8efd9ac69b63fe31484606
2020-05-18 14:43:16 +10:00
def test_certcheck_config(host, zuul_data):
if host.backend.get_hostname() != 'bridge.openstack.org':
pytest.skip()
if zuul_data['extra']['zuul']['job'] != 'system-config-run-letsencrypt':
pytest.skip()
domainlist = host.file('/var/lib/certcheck/ssldomains')
# TODO(ianw): figure out a flag or something from the
# system-config-run-letsencrypt test so that we can assert this
# file exists only in that case.
if not domainlist.exists:
pytest.skip()
assert domainlist.exists
assert domainlist.user == 'certcheck'
# from variables
assert domainlist.contains('^letsencrypt01.opendev.org 5000')
# from extra list; may need to change if list is modified
assert domainlist.contains('^wiki.openstack.org 443')