101 Commits

Author SHA1 Message Date
Clark Boylan
3981c02322 Provision LE cert for zuul.opendev.org
This provisions the cert but does not use it yet. We will do the
switchover once the cert is confirmed to be in place.

Depends-On: https://review.opendev.org/701819
Change-Id: I04fee48b9a79758527d8f9e8128c0fa915cd133e
2020-01-09 11:36:41 -08:00
Monty Taylor
e42862af73 Plumb through storyboard hiera data
NOTE: We should update storyboard-dev to be driven by
letsencrypt first, otherwise we need to plumb in the
self-signed cert, which gets weird with needing to
import it for java which in this case is in the container
image, meaning we either need to bind-mount java certs in
or build it in to the image.

Change-Id: Ida9dd15ca8262925c54579660fe9c16e2b573907
2019-12-17 08:13:34 -05:00
Monty Taylor
cc65640ef4 Add replication config to gerrit ansible
We need to configure replication.

Change-Id: I1bb61969a5adf37b07e43701d9a867b409a5ccf0
2019-12-17 08:13:34 -05:00
Zuul
7975a8648b Merge "Retire the Women-of-OpenStack list" 2019-12-02 19:19:42 +00:00
Clark Boylan
4ab6673092 Add necessary ansible vars for inap mirror LE
This was missed when adding the new inap mirror host to our inventory
and groups.

Change-Id: I02d7088ce1722f0a55fe6b17192fd462028aae5c
2019-11-26 13:52:51 -08:00
Clark Boylan
f7a305afbf Manage opendev.org with LE on all giteas
This catches up gitea02-07 with 01 managing ssl certs with LE.

Change-Id: I06228edca2204c5c57ebc5cb60b9d1308a393058
2019-11-18 12:47:08 -08:00
Clark Boylan
5392f8a27c Manage opendev.org cert with LE
This is the first step in managing the opendev.org cert with LE. We
modify gitea01.opendev.org only to request the cert so that if this
breaks the other 7 giteas can continue to serve opendev.org. When we are
happy with the results we can merge the followup change to update the
other 7 giteas.

Depends-On: https://review.opendev.org/694182
Change-Id: I9587b8c2896975aa0148cc3d9b37f325a0be8970
2019-11-18 12:07:10 -08:00
Ian Wienand
9c7136448a Add mirror01.mtl01.opendev.org
This server is a replacement for the .openstack.org version, which no
longer exists.

Depends-On: https://review.opendev.org/690767
Change-Id: I0d2eeb609219ad96db39d1d59b99ae376419df0e
2019-10-24 11:00:05 +11:00
Amy Marrich (spotz)
0dac2f3f87 Retire the Women-of-OpenStack list
As WoO is now part of the D&I WG this list is not needed.

Change-Id: I6af5f537357d3523aaa2d4b2e673296687e33a7d
2019-09-16 12:00:45 -05:00
Clark Boylan
4b4eb02f32 Replace the fn mirror again
Networking got weird on the previous host so we rebuilt this one going
back to networking we expect to work (FIPs and all that). This updates
the inventory so that we configure the host properly.

Change-Id: I0dcdbc9efdd330d66b57da0b01d23dd3d747f79b
2019-07-30 15:15:01 -07:00
Jeremy Stanley
b45c672de5 Replace fortnebula mirror
The fortnebula mirror is being rebuilt while the environment there
is under some refactoring. The old mirror isn't reachable any longer
so removing it from our inventory while adding its replacement
should be safe.

Also update the letsencrypt playbooks for the new name.

Change-Id: I789248e4216f4cf059ccc5b071c2a784f9c629e9
2019-07-29 13:08:58 +00:00
James E. Blair
ee3b273876 Exclude ansible_python_interpreter from write-inventory
Zuul now includes an ansible_python_interpreter hostvar in every
host in its inventory.  It defaults to python2.  The write-inventory
role, which takes the Zuul inventory and makes an inventory for
the fake bridge server in the gate passes that through.  Because it's
in /etc/ansible/inventory.yaml, it overrides any settings which may
arrive via group vars, but this is the way we set the interpreter
for all the hosts on bridge (we do not do so in the actual inventory
file).

To correct this, tell write-inventory to strip the
ansible_python_interpreter variable when it writes out the new
inventory.  This restores the behavior to match what happens on
the real bridge host.  One instance of setting the interpreter
for the fake "trusty" host used in base platform tests is moved to
a hostvars file to match the rest of the real hosts.

Change-Id: I60f0acb64e7b90ed8af266f21f2114fd598f4a3c
2019-07-10 10:10:02 -07:00
Clark Boylan
58b5fd8022 Add fortnebula mirror LE details dict
We need to supply information to ansible about how to provision LE certs
for the new fortnebula mirror. Add this dict to host_vars for
mirror01.regionone.fortnebula.opendev.org.

Change-Id: I02218e26ab6e9fad67e634f22de207740506d9e1
2019-07-03 14:39:25 -07:00
Ian Wienand
7810230408 Add OVH GRA1 mirror
This mirror will be manually configured with kafs (see
https://review.opendev.org/623974).  This should be a nice distant
geographic counterpoint to the IAD RAX server.

This will need to be manually configured with a custom kernel for now,
but fixes are making their way upstream and this host will be
converted when available.

Depends-On: https://review.opendev.org/667529
Change-Id: I6a22933029c096c781c93c33e6edf03bf59223c9
2019-06-27 10:07:44 +10:00
Ian Wienand
0041f4f673 Add certificates for IAD/ORD opendev.org mirrors
Change-Id: I509517c7601989cff18e29277f3391a813d5ba99
2019-06-12 11:25:50 +10:00
James E. Blair
3199e3b225 Enable SPF checking on lists
This requires an external program and only works on Debian hosts.

Newer versions of exim (4.91) have SPF functionality built-in, but
they are not yet available to us.

Change-Id: Idfe6bfa5a404b61c8761aa1bfa2212e4b4e32be9
2019-06-07 10:34:33 -07:00
Ian Wienand
2e9992af9e mirror01.dfw.rax.opendev.org : use python3 for ansible
This is a bionic host, so requires this to run as it has no
/usr/bin/python.  This is the same as the other bionic hosts, I just
forgot it.

Change-Id: Ifdd1df2fa83dd25dcc20596ce17e2f0c88279c62
2019-05-22 10:03:11 +10:00
Ian Wienand
73bbc6787f Bringup mirror01.dfw.rax.opendev.org
This is an initial host for testing opendev.org mirrors

Change-Id: I26b9ed1e21e2111f48bc7ecc384880c274eed213
Depends-On: https://review.opendev.org/660235
2019-05-21 11:08:30 +10:00
Ian Wienand
733122f0df Use handlers for letsencrypt cert updates
This change proposes calling a handler each time a certificate is
created/updated.  The handler name is based on the name of the
certificate given in the letsencrypt_certs variable, as described in
the role documentation.

Because Ansible considers calling a handler with no listeners an error
this means each letsencrypt user will need to provide a handler.

One simple option illustrated here is just to produce a stamp file.
This can facilitate cross-playbook and even cross-orchestration-tool
communication.  For example, puppet or other ansible playbooks can
detect this stamp file and schedule their reloads, etc. then remove
the stamp file.  It is conceivable more complex listeners could be
setup via other roles, etc. should the need arise.

A test is added to make sure the stamp file is created for the
letsencrypt test hosts, which are always generating a new certificate
in the gate test.

Change-Id: I4e0609c4751643d6e0c8d9eaa38f184e0ce5452e
2019-05-14 08:14:51 +10:00
Jeremy Stanley
74649101f8 Blackhole spam for airship-discuss-owner address
The airship-discuss-owner address for lists.airshipit.org is now
besieged with a flood of unsolicited messages. Reject anything sent
to it with an SMTP error explaining the situation.

Change-Id: I19fcea2a502c41cc9438f2710dae3cd686eecc05
2019-04-23 18:12:11 +00:00
Ian Wienand
a88d836661 Enable production cert generation for graphite01
Testing with the staging cert has shown nothing is going crazy and
making unreasonable letsencrypt requests ... switch this to generate a
real cert.

Change-Id: I861ea295312f83c66dd9b37271969d6e7f8fc2a2
2019-04-11 10:32:30 +10:00
Ian Wienand
86c5bc2b45 letsencrypt: split staging and self-signed generation
We currently only have letsencrypt_test_only as a single flag that
sets tests to use the letsencrypt staging environment and also
generates a self-signed certificate.

However, for initial testing we actually want to fully generate
certificates on hosts, but using the staging environment (i.e. *not*
generate self-signed certs).  Thus we need to split this option into
two, so the gate tests still use staging+self-signed, but in-progress
production hosts can just using the staging flag.

These variables are split, and graphite01.opendev.org is made to
create staging certificates.

Also remove some debugging that is no longer necessary.

Change-Id: I08959ba904f821c9408d8f363542502cd76a30a4
2019-04-10 08:47:32 +10:00
Ian Wienand
00efd089a2 Really add graphite01.opendev.org certificate details
This file was accidentally dropped from
I3e762d071cc609856950898b36f1903fe52840a6 during a rebase.

Change-Id: Iabc1db2aa029d7ff73b742ed63d367d8daa39187
2019-04-09 12:06:23 +10:00
Clark Boylan
9342c2aa6d Add zuul user to bridge.openstack.org
We want to trigger ansible runs on bridge.o.o from zuul jobs. First
iteration of this tried to login as root but this is not allowed by our
ssh config. That config seems reasonable so we add a zuul user instead
which we can ssh in as then run things as root from zuul jobs. This
makes use of our existing user management system.

Change-Id: I257ebb6ffbade4eb645a08d3602a7024069e60b3
2019-03-04 14:47:51 -08:00
James E. Blair
175a337e01 Handle registry role running under py3
Also, correct the host_vars filename.  Again.
Also, make sure we run the test on changes to the host_vars filename.

Change-Id: I95fb61531bae677f5c68f4e56ed718da6c507eb9
2019-02-08 09:13:06 -08:00
James E. Blair
ab557b2a98 Correct insecure-ci-registry01 hostvar filename
It's in opendev, not openstack.

Change-Id: Ib2f4154b18e6c15210e1f6ded9c1628b2fe5ef7a
2019-02-07 13:32:35 -08:00
James E. Blair
2766ec2595 Tell ansible to use py3 on insecure-ci-registry01
Change-Id: If0cf34f55524b7c87133fe04abb5513ad57e893b
2019-02-07 09:44:39 -08:00
James E. Blair
7610682b6f Configure .kube/config on bridge
Add the gitea k8s cluster to root's .kube/config file on bridge.

The default context does not exist in order to force us to explicitly
specify a context for all commands (so that we do not inadvertently
deploy something on the wrong k8s cluster).

Change-Id: I53368c76e6f5b3ab45b1982e9a977f9ce9f08581
2019-02-06 15:43:19 -08:00
Zuul
133116a430 Merge "Save a copy of all messages to openstack-discuss" 2019-02-05 21:07:47 +00:00
Jeremy Stanley
fea54bc43c Move OpenStackID host_vars to group_vars
Both staging and production OpenStackID servers are being updated to
our enumerated host naming convention as part of their upgrade from
Ubuntu Trusty to Xenial. Move their host-specific Ansible variables
to the new host groups we've created for each of them.

Change-Id: I359a51812b749bf9937943bae1cf1850bc1f85c3
2019-02-05 17:19:17 +00:00
James E. Blair
9e6db28ffb Save a copy of all messages to openstack-discuss
To debug DMARC issues, save a copy of every message sent to
openstack-discuss with as little manipulation as possible.

Change-Id: Ic1156849957bc326e9216c2aca0ab9d180e158e6
2019-01-23 11:19:00 -08:00
Jeremy Stanley
159f012d47 Reject messages to starlingx-discuss-owner
The owner address for the starlingx-discuss list on
lists.starlingx.io has started receiving large volumes of
unsolicited messages unrelated to its intended purpose. As there's
no easy way to discern them from legitimate messages, we'll do the
same as we've done for other owner addresses and reject them with a
brief error explaining the situation.

Change-Id: I95a910c2e6206098ca268a0e10e86b66455ad1bd
2019-01-02 19:40:15 +00:00
Jeremy Stanley
a507b6b401 Add lists.opendev.org to Mailman
Set up the initial boilerplate to enable addition of new
project-neutral Mailman mailing lists on lists.opendev.org.

Change-Id: I8cad4149bdd7b51d10f43b928cdb9362d4bde835
2018-12-13 20:36:08 +00:00
Jeremy Stanley
8017415779 Retire the interop-wg mailing list
This list's owners have asked for it to be shut down, as they will
be using an [interop-wg] tag on the new openstack-discuss ML for
future communication. Once this merges (so that Puppet won't
recreate it), the list can be removed with the `rmlist` utility
(this will still leave the archives available but will remove it
from the list index and no longer accept subscriptions/posts).

Set the old list address as an alias for the new openstack-discuss
ML so that replies to previous messages from the list will be routed
there for the foreseeable future.

Change-Id: Ib5fd5aece2465d569e0e7c180ee14ba94882f2b7
2018-11-30 18:39:16 +00:00
Jeremy Stanley
e9d49b4839 Shut down openstack general, dev, ops and sigs mls
The general openstack, openstack-dev, openstack-operators and
openstack-sigs mailing lists have been deprecated since November 19
and are slated to be removed on December 3. Merging this on that
date will ensure any further replies to messages from those lists
are rerouted to the new openstack-discuss mailing list for the
foreseeable future.

The openstack-tc list is included in this batch as it has already
been closed down with a recommendation to send further such
communications to the openstack-discuss ML.

Additionally remove the Puppet mailman resource for the
openstack-sigs ML so it won't be automatically recreated after it
gets deleted (the other lists predate our use of Puppet for this
purpose).

Clean up the corresponding -owner spam rejection aliases since these
addresses will no longer be accepting E-mail anyway.

Change-Id: I9a7fae465c3f6bdcf3ebbadb8926eb4feb8fad79
2018-11-30 18:22:00 +00:00
Ian Y. Choi
72781811aa Blackhole messages to openstack-ko-owner@l.o.o
The OpenStack Korean mailing list's owner address have
become overrun by the same mass spam we've seen hitting our other ML
owner addresses. Add a blackhole alias for it.

Change-Id: Ia6c7e6701a69ee56076062aa85f8699121648501
2018-11-29 02:23:35 +09:00
Jeremy Stanley
33ec337b42 Blackhole messages to openstack-sigs-owner@l.o.o
The OpenStack SIGS mailing list's owner address is starting to
become overrun by the same mass spam we've seen hitting our other ML
owner addresses. Add a blackhole alias for it.

Change-Id: Iefc5b5fa600c5d1de75d3302c8ddf0e1a03301e5
2018-11-19 16:16:33 +00:00
Jeremy Stanley
4fb2143f3c Blackhole messages to edge-computing-owner@l.o.o
The OpenStack edge-computing mailing list's owner address is
starting to become overrun by the same mass spam we've seen hitting
our other ML owner addresses. Add a blackhole alias for it.

Change-Id: I97a2db5d0565cc166604352e397f580ea2d9e767
2018-11-12 10:15:02 +00:00
James E. Blair
2780973330 Set ansible python version for opendev nameservers
Change-Id: Icaee291e872b6a19793a1ba003e55a43f3898ea7
2018-11-01 09:20:26 -07:00
James E. Blair
46a643f823 lists: run mailman verp router before dnslookup
The mailman verp router handles remote addresses like dnslookup.
It needs to run before dnslookup in order to be effective, so run
it first.  It's only for outgoing messages, not incoming, so won't
affect the blackhole aliases we have for incoming fake bounce
messages.

Note that the verp router hasn't been used in about a year due to
this oversight, so we should merge this change with caution.

Change-Id: I7d2a0f05f82485a54c1e7048f09b4edf6e0f0612
2018-10-16 13:04:13 -07:00
James E. Blair
c49d5d6f2b Allow Zuul to log into bridge
Allow post-review jobs running under system-config and project-config
to ssh into bridge in order to run Ansible.

Change-Id: I841f87425349722ee69e2f4265b99b5ee0b5a2c8
2018-09-12 10:20:26 -06:00
Monty Taylor
a634593a05
Set mgmt_hieradata in puppet group_vars
This is not a variable describing the system-under-management
bridge.openstack.org - it's a variable that is always true for all
systems in the puppet group.

As a result, update the puppet apply test to figure out which directory
we should be copying modules _from_ - since the puppet4 tests will be
unhappy otherwise.

Change-Id: Iddee83944bd85f69acf4fcfde83dc70304386baf
2018-08-17 14:25:50 -05:00
Monty Taylor
7a0ac4ce03
Set mgmt_puppet_module_dir publically
This was set in the private variables on brige for the transition. But
it can go here now.

Change-Id: I3883672bf549681f8a4f26871c485a71de8ee056
2018-08-17 09:38:35 -05:00
Zuul
f3036203c3 Merge "Remove base.yaml things from openstack_project::server" 2018-08-17 10:43:53 +00:00
Monty Taylor
bab6fcad3c
Remove base.yaml things from openstack_project::server
Now that we've got base server stuff rewritten in ansible, remove the
old puppet versions.

Depends-On: https://review.openstack.org/588326
Change-Id: I5c82fe6fd25b9ddaa77747db377ffa7e8bf23c7b
2018-08-16 17:25:10 -05:00
James E. Blair
40c6e6d7ad Template all exim routers
So that we can have complete control of the router order, always
template the full set of routers, including the "default" ones.
So that it's easy to use the defaults but put them in a different
order, define each router in its own variable which can be used
in host or group vars to "copy" that router in.

Apply this change to lists, firehose, and storyboard, all of which
have custom exim routers.  Note that firehose intentionally has
its localuser router last.

Change-Id: I737942b8c15f7020b54e350db885e968a93f806a
2018-08-16 13:49:55 -07:00
Monty Taylor
f78f871afe
Make a firehose group with firehose01 in it
We want to configure firehose logically as the firehose service, but the
host that is in the group is called firehose01.openstack.org. Make a
group and put the config variables for firehose into it.

Change-Id: I17c8e8a72f41c5e2730af81f70cef81dd3ed7bca
2018-08-16 15:11:20 -05:00
Monty Taylor
0d1f235fce
Add exim config for firehose and storyboard
In order to get puppet out of the business of mucking with exim and
fighting ansible, finish moving the config to ansible.

This introduces a storyboard group that we can use to apply the exim
config across both servers. It also splits the base playbook so that we
can avoid running exim on the backup servers. And we set
purge_apt_sources the same as was set in puppet. We should probably
remove it though, since none of us have any clue why it's here.

Change-Id: I43ee891a9c1beead7f97808208829b01a0a7ced6
2018-08-15 15:11:48 -05:00
Monty Taylor
4cca3f8d2a
Add lists exim config to ansible
The mailing list servers have a more complex exim config. Put the
routers and transports into ansible variables.

While we're doing it, role variables with an exim_ prefix - since 'routers'
as a global variable might be a little broad.

iteritems isn't a thing in python3, only items.

We need to escape the exim config with ${if or{{ - because of the {{
which looks like jinja. Wrap it in a {% raw %} block.

Getting the yaml indentation right for things here is non-trivial. Make
them strings instead.

Add a README.rst file - and use the zuul:rolevar construct in it,
because it's nice.

Change-Id: Ieccfce99a1d278440c5baa207479a1887898298e
2018-08-15 15:11:48 -05:00
Monty Taylor
ee02ba0123
Set mgmt_hieradata variable for bridge.openstack.org
ansible-role-puppet attempts to infer where it should copy hieradata
from based on puppet3 or puppet4. On bridge there is no puppet and thus
there is no puppet version. Set mgmt_hieradata to tell
ansible-role-puppet from where it should copy hiera secrets.

Change-Id: I0c518b8a5a8ee2155e2125d6bc7f4e0a3bf4faeb
2018-08-10 12:21:39 -05:00