219 lines
5.9 KiB
YAML
Raw Normal View History

plugin: yamlgroup
groups:
adns: adns*.open*.org
afs-server-common:
- afs[0-9]*.openstack.org
- afsdb[0-9]*.openstack.org
afs-file-server:
- afs[0-9]*.openstack.org
afs-db-server:
- afsdb[0-9]*.openstack.org
afs-client:
- mirror[0-9]*.opendev.org
- mirror-update[0-9]*.opendev.org
- ze[0-9]*.open*.org
- afsdb*.open*.org
- afs[0-9]*.open*.org
- static[0-9]*.opendev.org
borg-backup:
- etherpad[0-9]*.opendev.org
- gitea01.opendev.org
- review02.opendev.org
- zuul[0-9]*.opendev.org
- refstack01.openstack.org
- kdc03.openstack.org
- eavesdrop01.opendev.org
- paste01.opendev.org
# These are test specific hosts that we add to the backup
# group to mimic as much as possible what their prod version
# end up doing.
- gitea99.opendev.org
- review99.opendev.org
# All these servers are "special-cased" in specifically
# as they are puppet and should be replaced "soon"
- ethercalc02.openstack.org
- lists.openstack.org
- storyboard01.opendev.org
- translate01.openstack.org
borg-backup-server:
- backup02.ca-ymq-1.vexxhost.opendev.org
- backup01.ord.rax.opendev.org
cacti: cacti[0-9]*.open*.org
Generate ssl check list directly from letsencrypt variables This autogenerates the list of ssl domains for the ssl-cert-check tool directly from the letsencrypt list. The first step is the install-certcheck role that replaces the puppet-ssl_cert_check module that does the same. The reason for this is so that during gate testing we can test this on the test bridge.openstack.org server, and avoid adding another node as a requirement for this test. letsencrypt-request-certs is updated to set a fact letsencrypt_certcheck_domains for each host that is generating a certificate. As described in the comments, this defaults to the first host specified for the certificate and the listening port can be indicated (if set, this new port value is stripped when generating certs as is not necessary for certificate generation). The new letsencrypt-config-certcheck role runs and iterates all letsencrypt hosts to build the final list of domains that should be checked. This is then extended with the letsencrypt_certcheck_additional_domains value that covers any hosts using certificates not provisioned by letsencrypt using this mechanism. These additional domains are pre-populated from the openstack.org domains in the extant check file, minus those openstack.org domain certificates we are generating via letsencrypt (see letsencrypt-create-certs/handlers/main.yaml). Additionally, we update some of the certificate variables in host_vars that are listening on port !443. As mentioned, bridge.openstack.org is placed in the new certcheck group for gate testing, so the tool and config file will be deployed to it. For production, cacti is added to the group, which is where the tool currently runs. The extant puppet installation is disabled, pending removal in a follow-on change. Change-Id: Idbe084f13f3684021e8efd9ac69b63fe31484606
2020-05-18 14:43:16 +10:00
certcheck:
- cacti[0-9]*.open*.org
cloud-launcher:
- bridge.openstack.org
codesearch:
Migrate codesearch site to container The hound project has undergone a small re-birth and moved to https://github.com/hound-search/hound which has broken our deployment. We've talked about leaving codesearch up to gitea, but it's not quite there yet. There seems to be no point working on the puppet now. This builds a container than runs houndd. It's an opendev specific container; the config is pulled from project-config directly. There's some custom scripts that drive things. Some points for reviewers: - update-hound-config.sh uses "create-hound-config" (which is in jeepyb for historical reasons) to generate the config file. It grabs the latest projects.yaml from project-config and exits with a return code to indicate if things changed. - when the container starts, it runs update-hound-config.sh to populate the initial config. There is a testing environment flag and small config so it doesn't have to clone the entire opendev for functional testing. - it runs under supervisord so we can restart the daemon when projects are updated. Unlike earlier versions that didn't start listening till indexing was done, this version now puts up a "Hound is not ready yet" message when while it is working; so we can drop all the magic we were doing to probe if hound is listening via netstat and making Apache redirect to a status page. - resync-hound.sh is run from an external cron job daily, and does this update and restart check. Since it only reloads if changes are made, this should be relatively rare anyway. - There is a PR to monitor the config file (https://github.com/hound-search/hound/pull/357) which would mean the restart is unnecessary. This would be good in the near and we could remove the cron job. - playbooks/roles/codesearch is unexciting and deploys the container, certificates and an apache proxy back to localhost:6080 where hound is listening. I've combined removal of the old puppet bits here as the "-codesearch" namespace was already being used. Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
2020-11-17 17:13:46 +11:00
- codesearch[0-9]*.opendev.org
control-plane-clouds:
- bridge.openstack.org
disabled: []
dns:
- adns*.opendev.org
- ns*.opendev.org
eavesdrop: eavesdrop[0-9]*.opendev.org
elasticsearch: elasticsearch[0-9]*.open*.org
ethercalc: ethercalc*.open*.org
etherpad: etherpad[0-9]*.open*.org
gitea:
- gitea[0-9]*.opendev.org
gitea-lb:
- gitea-lb[0-9]*.opendev.org
grafana:
- grafana[0-9]*.opendev.org
graphite:
- graphite*.opendev.org
health:
- health[0-9]*.openstack.org
jvb:
- jvb[0-9]*.opendev.org
kerberos-client:
- afs[0-9]*.open*.org
- afsdb*.open*.org
- kdc[0-9]*.openstack.org
- mirror[0-9]*.opendev.org
- mirror-update[0-9]*.opendev.org
- static[0-9]*.opendev.org
- ze[0-9]*.open*.org
kerberos-kdc:
- kdc03.openstack.org
- kdc04.openstack.org
kerberos-kdc-primary:
- kdc03.openstack.org
kerberos-kdc-replica:
- kdc04.openstack.org
Add a keycloak server This adds a keycloak server so we can start experimenting with it. It's based on the docker-compose file Matthieu made for Zuul (see https://review.opendev.org/819745 ) We should be able to configure a realm and federate with openstackid and other providers as described in the opendev auth spec. However, I am unable to test federation with openstackid due its inability to configure an oauth app at "localhost". Therefore, we will need an actual deployed system to test it. This should allow us to do so. It will also allow use to connect realms to the newly available Zuul admin api on opendev. It should be possible to configure the realm the way we want, then export its configuration into a JSON file and then have our playbooks or the docker-compose file import it. That would allow us to drive change to the configuration of the system through code review. Because of the above limitation with openstackid, I think we should regard the current implementation as experimental. Once we have a realm configuration that we like (which we will create using the GUI), we can chose to either continue to maintain the config with the GUI and appropriate file backups, or switch to a gitops model based on an export. My understanding is that all the data (realms configuration and session) are kept in an H2 database. This is probably sufficient for now and even production use with Zuul, but we should probably switch to mariadb before any heavy (eg gerrit, etc) production use. This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html We can re-deploy with a new domain when it exists. Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753 Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
2021-11-30 13:03:12 -08:00
keycloak: keycloak[0-9]*.opendev.org
letsencrypt:
Migrate codesearch site to container The hound project has undergone a small re-birth and moved to https://github.com/hound-search/hound which has broken our deployment. We've talked about leaving codesearch up to gitea, but it's not quite there yet. There seems to be no point working on the puppet now. This builds a container than runs houndd. It's an opendev specific container; the config is pulled from project-config directly. There's some custom scripts that drive things. Some points for reviewers: - update-hound-config.sh uses "create-hound-config" (which is in jeepyb for historical reasons) to generate the config file. It grabs the latest projects.yaml from project-config and exits with a return code to indicate if things changed. - when the container starts, it runs update-hound-config.sh to populate the initial config. There is a testing environment flag and small config so it doesn't have to clone the entire opendev for functional testing. - it runs under supervisord so we can restart the daemon when projects are updated. Unlike earlier versions that didn't start listening till indexing was done, this version now puts up a "Hound is not ready yet" message when while it is working; so we can drop all the magic we were doing to probe if hound is listening via netstat and making Apache redirect to a status page. - resync-hound.sh is run from an external cron job daily, and does this update and restart check. Since it only reloads if changes are made, this should be relatively rare anyway. - There is a PR to monitor the config file (https://github.com/hound-search/hound/pull/357) which would mean the restart is unnecessary. This would be good in the near and we could remove the cron job. - playbooks/roles/codesearch is unexciting and deploys the container, certificates and an apache proxy back to localhost:6080 where hound is listening. I've combined removal of the old puppet bits here as the "-codesearch" namespace was already being used. Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
2020-11-17 17:13:46 +11:00
- codesearch[0-9]*.opendev.org
- eavesdrop[0-9]*.opendev.org
- etherpad[0-9]*.opendev.org
- ethercalc[0-9]*.open*.org
- gitea[0-9]*.opendev.org
- grafana[0-9]*.opendev.org
- graphite[0-9]*.opendev.org
- insecure-ci-registry[0-9]*.opendev.org
Add a keycloak server This adds a keycloak server so we can start experimenting with it. It's based on the docker-compose file Matthieu made for Zuul (see https://review.opendev.org/819745 ) We should be able to configure a realm and federate with openstackid and other providers as described in the opendev auth spec. However, I am unable to test federation with openstackid due its inability to configure an oauth app at "localhost". Therefore, we will need an actual deployed system to test it. This should allow us to do so. It will also allow use to connect realms to the newly available Zuul admin api on opendev. It should be possible to configure the realm the way we want, then export its configuration into a JSON file and then have our playbooks or the docker-compose file import it. That would allow us to drive change to the configuration of the system through code review. Because of the above limitation with openstackid, I think we should regard the current implementation as experimental. Once we have a realm configuration that we like (which we will create using the GUI), we can chose to either continue to maintain the config with the GUI and appropriate file backups, or switch to a gitops model based on an export. My understanding is that all the data (realms configuration and session) are kept in an H2 database. This is probably sufficient for now and even production use with Zuul, but we should probably switch to mariadb before any heavy (eg gerrit, etc) production use. This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html We can re-deploy with a new domain when it exists. Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753 Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
2021-11-30 13:03:12 -08:00
- keycloak[0-9]*.opendev.org
- lists.katacontainers.io
- lists.openstack.org
- meetpad[0-9]*.opendev.org
- mirror[0-9]*.opendev.org
- nb[0-9]*.opendev.org
- paste[0-9]*.opendev.org
- refstack[0-9]*.openstack.org
- review[0-9]*.opendev.org
- static[0-9]*.opendev.org
- storyboard[0-9]*.opendev.org
- translate[0-9]*.open*.org
- zuul[0-9]*.opendev.org
logstash:
- logstash[0-9]*.open*.org
logstash-worker:
- logstash-worker[0-9]*.open*.org
mailman:
- lists*.katacontainers.io
- lists*.open*.org
meetpad:
- meetpad[0-9]*.opendev.org
mirror:
- mirror[0-9]*.opendev.org
mirror-update:
- mirror-update[0-9]*.opendev.org
nodepool:
- nb[0-9]*.opendev.org
- nl[0-9]*.open*.org
nodepool-builder:
- nb[0-9]*.opendev.org
nodepool-launcher:
- nl[0-9]*.open*.org
ns:
- ns[0-9]*.open*.org
paste:
- paste[0-9]*.opendev.org
puppet:
- cacti[0-9]*.open*.org
- elasticsearch[0-9]*.open*.org
- ethercalc[0-9]*.open*.org
- health[0-9]*.openstack.org
- logstash-worker[0-9]*.open*.org
- logstash[0-9]*.open*.org
- status*.open*.org
- storyboard-dev[0-9]*.opendev.org
- storyboard[0-9]*.opendev.org
- subunit-worker[0-9]*.open*.org
- translate-dev[0-9]*.open*.org
- translate[0-9]*.open*.org
puppet4:
- cacti[0-9]*.open*.org
- elasticsearch[0-9]*.open*.org
- ethercalc[0-9]*.open*.org
- health[0-9]*.openstack.org
- logstash-worker[0-9]*.open*.org
- logstash[0-9]*.open*.org
- status*.open*.org
- storyboard[0-9]*.opendev.org
- storyboard-dev[0-9]*.opendev.org
- subunit-worker[0-9]*.open*.org
- translate[0-9]*.open*.org
- translate-dev[0-9]*.open*.org
refstack:
- refstack[0-9]*.openstack.org
registry:
- insecure-ci-registry[0-9]*.opendev.org
review:
- review[0-9]*.opendev.org
# This group disables operations like project-managment and
# replication. It is intended for staging new production servers.
#review-staging:
static:
- static[0-9]*.opendev.org
status:
- status*.open*.org
storyboard:
- storyboard[0-9]*.opendev.org
storyboard-dev:
- storyboard-dev[0-9]*.opendev.org
subunit-worker:
- subunit-worker[0-9]*.open*.org
translate-dev:
- translate-dev[0-9]*.open*.org
translate:
- translate[0-9]*.open*.org
webservers:
- cacti[0-9]*.open*.org
Migrate codesearch site to container The hound project has undergone a small re-birth and moved to https://github.com/hound-search/hound which has broken our deployment. We've talked about leaving codesearch up to gitea, but it's not quite there yet. There seems to be no point working on the puppet now. This builds a container than runs houndd. It's an opendev specific container; the config is pulled from project-config directly. There's some custom scripts that drive things. Some points for reviewers: - update-hound-config.sh uses "create-hound-config" (which is in jeepyb for historical reasons) to generate the config file. It grabs the latest projects.yaml from project-config and exits with a return code to indicate if things changed. - when the container starts, it runs update-hound-config.sh to populate the initial config. There is a testing environment flag and small config so it doesn't have to clone the entire opendev for functional testing. - it runs under supervisord so we can restart the daemon when projects are updated. Unlike earlier versions that didn't start listening till indexing was done, this version now puts up a "Hound is not ready yet" message when while it is working; so we can drop all the magic we were doing to probe if hound is listening via netstat and making Apache redirect to a status page. - resync-hound.sh is run from an external cron job daily, and does this update and restart check. Since it only reloads if changes are made, this should be relatively rare anyway. - There is a PR to monitor the config file (https://github.com/hound-search/hound/pull/357) which would mean the restart is unnecessary. This would be good in the near and we could remove the cron job. - playbooks/roles/codesearch is unexciting and deploys the container, certificates and an apache proxy back to localhost:6080 where hound is listening. I've combined removal of the old puppet bits here as the "-codesearch" namespace was already being used. Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
2020-11-17 17:13:46 +11:00
- codesearch[0-9]*.opendev.org
# eavesdrop has its own group with custom ports
- ethercalc[0-9]*.open*.org
- etherpad[0-9]*.open*.org
- grafana[0-9]*.opendev.org
- graphite*.opendev.org
- health[0-9]*.openstack.org
Add a keycloak server This adds a keycloak server so we can start experimenting with it. It's based on the docker-compose file Matthieu made for Zuul (see https://review.opendev.org/819745 ) We should be able to configure a realm and federate with openstackid and other providers as described in the opendev auth spec. However, I am unable to test federation with openstackid due its inability to configure an oauth app at "localhost". Therefore, we will need an actual deployed system to test it. This should allow us to do so. It will also allow use to connect realms to the newly available Zuul admin api on opendev. It should be possible to configure the realm the way we want, then export its configuration into a JSON file and then have our playbooks or the docker-compose file import it. That would allow us to drive change to the configuration of the system through code review. Because of the above limitation with openstackid, I think we should regard the current implementation as experimental. Once we have a realm configuration that we like (which we will create using the GUI), we can chose to either continue to maintain the config with the GUI and appropriate file backups, or switch to a gitops model based on an export. My understanding is that all the data (realms configuration and session) are kept in an H2 database. This is probably sufficient for now and even production use with Zuul, but we should probably switch to mariadb before any heavy (eg gerrit, etc) production use. This is a partial implementation of https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html We can re-deploy with a new domain when it exists. Change-Id: I2e069b1b220dbd3e0a5754ac094c2b296c141753 Co-Authored-By: Matthieu Huin <mhuin@redhat.com>
2021-11-30 13:03:12 -08:00
- keycloak[0-9]*.opendev.org
- nb[0-9]*.opendev.org
- nl[0-9]*.open*.org
- paste[0-9]*.opendev.org
- refstack[0-9]*.openstack.org
- static[0-9]*.opendev.org
- status*.open*.org
- storyboard-dev[0-9]*.opendev.org
- storyboard[0-9]*.opendev.org
- translate-dev[0-9]*.open*.org
- translate[0-9]*.open*.org
zookeeper:
- zk[0-9]*.open*.org
zuul-lb:
- zuul-lb[0-9]*.opendev.org
zuul:
- ze[0-9]*.opendev.org
- zm[0-9]*.opendev.org
- zuul[0-9]*.opendev.org
zuul-executor:
- ze[0-9]*.opendev.org
zuul-merger:
- zm[0-9]*.opendev.org
zuul-preview:
- zp[0-9]*.opendev.org
zuul-scheduler:
- zuul[0-9]*.opendev.org
zuul-web:
- zuul[0-9]*.opendev.org