Commit Graph

128 Commits (master)

Author SHA1 Message Date
Jeremy Stanley a6ab3543fc Move Airship and Kata lists to Mailman 3
This uncomments the list additions for the and sites on the new mailman server, removing
the configuration for them from the server and, in
the case of the latter, removing all our configuration management
for the server as it was the only site hosted there.

Change-Id: Ic1c735469583e922313797f709182f960e691efc
2023-09-14 12:08:34 +00:00
Zuul 0b86e9e148 Merge "Remove old insecure-ci-registry01 from our inventory" 2023-08-17 04:38:58 +00:00
Zuul 03ca4627a1 Merge "Replace ze10-ze12" 2023-07-15 23:50:42 +00:00
Zuul deb37417b8 Merge "Replace ze07-ze09" 2023-07-15 23:50:40 +00:00
Zuul 31d10ee05c Merge "Replace ze04-ze06" 2023-07-11 14:54:01 +00:00
Zuul 12dcfd4cd6 Merge "Replace ze01-ze03" 2023-06-28 17:55:56 +00:00
Clark Boylan 7509976b95 Remove old insecure-ci-registry01 from our inventory
This server has been replaced by insecure-ci-registry02. Remove it. This
will allow us to delete the server as well.

Change-Id: I4d9bc6ab90b13655ace2edf4a6fa7c362623c010
2023-06-26 10:43:34 -07:00
Tony Breeds facd7ec2b3 Deploy on jammy
This adds a second registry host. We will remove the other once we've
cut over successfully (should just depend on a DNS update).

Change-Id: Ib6be5ef242ed038c23e0007488f2c21ce10f4fcb
2023-06-23 13:21:48 -07:00
James E. Blair 1e5746a4c8 Replace ze10-ze12
Change-Id: I6fe1d38e7a25ff9ac93037773e0751423621c939
2023-06-07 08:33:39 -07:00
James E. Blair c08ad2139f Replace ze07-ze09
Change-Id: Icaefe7906d857bf35ec98c25c54bba288070547c
2023-06-07 08:33:17 -07:00
James E. Blair c66aca47fa Replace ze04-ze06
Change-Id: I3207bb907c65769d50898f00e466032d54c1f612
2023-06-07 08:32:11 -07:00
James E. Blair 65bf8e1aa6 Replace ze01-ze03
Change-Id: I60b665b0f6ae102c07e318541cae3a25ed0756fd
2023-06-07 08:31:47 -07:00
James E. Blair 6a09102d82 Revert "Replace ze01-ze06"
This reverts commit 2f54b9b117.

There is a problem with openafs.

Change-Id: I658e71d30afe5e1946b40ad57108e3a4888da2eb
2023-06-03 17:50:41 -07:00
James E. Blair 2f54b9b117 Replace ze01-ze06
Change-Id: I07131742b75d2d66e253f92991c5beee15173678
2023-06-03 07:07:02 -07:00
Clark Boylan 4d05eacff3 Remove zp01
It should no longer be used at this point as zp02 has taken over. Clean
up the old unneeded server.

Change-Id: Idf7f72e284d934968bbac25893456dcfc7d701a5
2023-06-01 10:09:06 -07:00
Clark Boylan fb147ec766 Add zp02 zuul-preview server to replace zp01
zp01 is older and is in the list of servers to replace. Replace it with
the newly launched jammy zp02 server.

Change-Id: Ic01433e96328f5775f73a97cdbf2ae2a07c1a6fa
2023-06-01 10:05:46 -07:00
James E. Blair 00da349414 Upgrade zuul mergers to jammy
Change-Id: I569b3d4dfc102a525b6f8c6ceb59e0a472bf7941
2023-05-24 13:17:59 -07:00
Zuul 13f167a46a Merge "Remove old DNS servers" 2023-04-30 22:29:26 +00:00
Ian Wienand 436f956140
Remove old DNS servers
Remove adns1/ns1/ns2 which are no longer in use.  Switch the primary
master to adns02; the secondaries ns03/ns04 will now update from

Change-Id: I700a514dd2b72b2632e8d0668251f52907008d44
2023-04-28 12:03:30 +10:00
Zuul b4a0613a95 Merge "inventory : add Ubuntu Jammy DNS refresh servers" 2023-04-25 19:19:43 +00:00
Ian Wienand e15312222c
inventory : add Ubuntu Jammy DNS refresh servers
This is a new hidden-primary DNS (to replace and
two small namservers to replace ns1/ns2.  ns03 is in RAX/dfw and ns04
is VexxHost/ya-cmq-1.

Change-Id: I1bcbdc21e5677e83e23418ca8953531cc84e96e6
2023-04-19 09:54:35 +10:00
Clark Boylan 4a6a469ff6 Remove etherpad01 from inventory
Once we have migrated the etherpad db to etherpad02, updated dns to
point at etherpad02, and are comfortable we won't need to fallback to
etherpad01 we should remove etherpad01 from inventory. Then the server
can be deleted and we can clean up DNS.

Change-Id: I43ca5d4469821a4542de92bc1e12d4b8e9fbfb2c
2023-04-11 13:28:48 -07:00
Clark Boylan 6d3e72d425 Remove static01 from inventory
This server is already in the emergency file and DNS records for
everything it serves have been moved to static02. When we are happy that
static01 is no longer necessary as a fallback we should land this
change, delete the server, and cleanup DNS.

Change-Id: I3018727daaa4ab59d2f3182fcfdfaa0ca4fe30e1
2023-04-11 13:27:33 -07:00
Clark Boylan 4c81903ccf Add etherpad02 to inventory
This is a new Jammy etherpad server. Landing this change will deploy it
with an empty database. We will schedule a downtime then stop
etherpad01's services, migrate its db to etherpad02, update dns then
will be swapped over.

Note this requires secret vars updates for db passwds which I have

Change-Id: I4e8d1c9c9bffc7b740be33934942128b6939cc12
2023-04-05 08:36:27 -07:00
Clark Boylan 19352a6944 Add static02 to inventory
This is a Jammy replacement host for static01. It was booted with the
same flavor as the old server as the old server seemed happy with its
size. Note this may be our first afs client on jammy but our PPA appears
to have jammy packages so that should be fine.

Change-Id: Ib66e068024a9f2a2a8098934893dd7a730f6af34
2023-04-05 08:36:24 -07:00
Zuul 6690e13151 Merge "Remove gitea01-04 from configuration management" 2023-03-24 17:23:01 +00:00
Ian Wienand be992b3bb6
infra-prod: run job against linaro
We have access to manage the linaro cloud, but we don't want to
completely own the host as it has been configured with kolla-ansible;
so we don't want to take over things like name resolution, iptables
rules, docker installation, etc.

But we would like to manage some parts of it, like rolling out our
root users, some cron jobs, etc.  While we could just log in and do
these things, it doesn't feel very openinfra.

This allows us to have a group "unmanaged" that skips the base jobs.
The base playbook is updated to skip these hosts.

For now, we add a cloud-linaro prod job that just does nothing so we
can validate the whole thing.  When it's working, I plan to add a few
things as discussed above.

Change-Id: Ie8de70cbac7ffb9d727a06a349c3d2a3b3aa0b40
2023-03-15 12:00:25 +11:00
Clark Boylan b327d70f2b Remove gitea01-04 from configuration management
At this point gitea09-14 should be our only production gitea backends
behind haproxy and the only gitea servers replicated to by Gerrit.
Additionally, our gitea DB backups should be moved to gitea09 by our
depends on change. There shouldn't be any other reason to keep these
servers around as long as the new ones are keeping up.

Change-Id: I3d4dfa1682ed1d14c6cd9108fbcbe0bf934b72c7
2023-03-13 08:37:29 -07:00
Clark Boylan fd5f32991b Add gitea13 and gitea14 to inventory
This brings out total of new giteas to 6. We noticed today that load
skyrocketed on the other four new giteas implying that we need more
gitea backends. We think we tracked this down to a bad crawler (doesn't
identify itself as such), but we should be able to handle these
situations more gracefully. Note that gitea14 recycles gitea08's (now
deleted) IP address.

Change-Id: Ia0517eabd507a6e8c92205b894014c05a92380d1
2023-03-08 11:50:36 -08:00
Clark Boylan 15ac7a28aa Remove gitea05-08 from configuration management
At this point these four servers have been replaced by four new Jammy
servers at gitea09-12. They are no longer behind the load
balancer, and Gerrit is not replicating to them. We should remove them
to stop consuming unnecessary resources and avoid any future confusion.

Change-Id: I636b1ed83ed433f3adca2a8b4523335c6a62c702
2023-03-04 10:07:15 -08:00
Clark Boylan 0bc11a6fa4 Add gitea10-12 to our inventory
These servers will replace gitea05-07 and are built on top of Ubuntu
Jammy. Landing this change should deploy a working, but empty, gitea
installation. We will then transplant the brain (db) of gitea01 into
these three new servers so that they know about historical redirects.
Once that is all done we can replicate git content from gerrit to them
and eventually put them behind the production load balancer.

Change-Id: I519564fd16c204ce182bc7cd82d5e638d01a1a6b
2023-03-02 10:18:46 -08:00
Clark Boylan 86a0b80953 Add gitea09 to our inventory
This adds a new Jammy Gitea09 server to our inventory. THis should
deploy Gitea and provision empty repos on it for entries in our projects
list. However, we'll need to do database surgery to carry over redirects
from an older server. It is for this reason we don't add the server to
the Gerrit replication list or our load balancer's pool.

We'll take this a step at a time and add the new server to those other
items when it is ready.

Change-Id: Idac0f250f74d8db4ff8d6d68c1a1c35c28a4660f
2023-02-15 16:27:55 -08:00
Jeremy Stanley f18e06e011 Farewell limestone
The mirror in our Limestone Networks donor environment is now
unreachable, but we ceased using this region years ago due to
persistent networking trouble and the admin hasn't been around for
roughly as long, so it's probably time to go ahead and say goodbye
to it.

Change-Id: Ibad440a3e9e5c210c70c14a34bcfec1fb24e07ce
2023-02-13 23:54:59 +00:00
Ian Wienand 805ed731c0
Remove linaro-us cloud
All references to this cloud have been removed from nodepool, so we
can now remove nb03 and the mirror node.

Change-Id: I4d97f7bbb6392656017b1774b413b58bdb797323
2023-01-25 11:37:05 +11:00
Ian Wienand 8b42a38df2
This is a replacement ARM64 builder for nb03.  Add basic config

Change-Id: I42889d3acfb269687dcc9731c04c8eae9ace2cfa
2023-01-10 15:07:45 +11:00
Ian Wienand 527dcc9d3e
Add linaro mirror node
This is the mirror node for the new linaro cluster

Change-Id: I0c08cede4b60c958237364918f11244c7f192774
2022-12-22 09:25:06 +11:00
Clark Boylan 1d6f499ca1 Cleanup iweb/inap provider content
This provider is going away and the depends-on change should be the last
step to remove it from nodepool. Once that is complete we can stop
trying to manage the mirror there (it will need to be manually shut
down), stop managing our user accounts, and stop writing cloud.yaml that
include these details for inap/iweb on nodepool nodes.

Note we leave the bridge clouds.yaml content in place so that we can
manually clean up the mirror node. We can safely remove that clouds.yaml
content in the future without much impact.

Change-Id: I01338712aeae79aa78e7f61d332a2290093c8a1b
2022-12-12 09:07:28 -08:00
Jeremy Stanley e63bf52a2d Add to the inventory
This will serve our new Mailman v3 mailing list sites once they're

Change-Id: I0c7229eeffcb5896edadf697044cbd026037d903
2022-11-23 02:22:01 +00:00
Ian Wienand 618708b42a
inventory: add host keys
Add the host keys to the inventory.  This will allow us to populate
the known_hosts on the bastion host from system-config

Change-Id: I4863425d5b784d0cdf118e1252414ca78fd24179
2022-11-18 12:55:39 +11:00
Ian Wienand 10f8749239
This host is no longer in use, remove references

Change-Id: I13bf4b7947ffcf61f8064976c54831bf1b3261eb
2022-11-04 08:32:30 +11:00
Zuul 5cd43f0588 Merge "Remove gitea-lb01 and jvb02 from our inventory" 2022-10-25 15:30:16 +00:00
Ian Wienand 740656835d
Add to inventory
This just adds the host to the inventory and cacti/graphite; it is not
used yet.

Change-Id: Ied7208b300da4488c4ffb5e5c4003842cc22121d
2022-10-25 10:45:44 +11:00
Clark Boylan daab4ea6cd Remove gitea-lb01 and jvb02 from our inventory
gitea-lb01 has been replaced by gitea-lb02. Reviewers should double
check the new gitea-lb02 server appears happy to them before approving
this change. Approving this change will be the last step required before
we delete gitea-lb01 entirely.

jvb02 is one of two additional jitsi meet jvb servers (on top of the one
running in the all in one meetpad install) deployed to help scale up our
jitsi meet server. The current October 2022 PTG has shown that while
meetpad has been useful to a small number of team there isn't the
concurrent demand that having extra jvbs like this supports. This means
we can scale back as the PTG is expected to be our largest load on the

Do both of these in the same change as they update the inventory file
which causes all of our infra-prod jobs to run which takes a long time.
Squashing the changes together ensures we turn that around in half the

Change-Id: I5e8b23ba9a8dc389cd7ef7f294ba4b0f8048102b
2022-10-21 07:46:35 -07:00
Clark Boylan 8d4f1c719e Add Jammy gitea-lb02 to our inventory
This adds our first Jammy production server to the mix. We update the
gitea load balancer as it is a fairly simple service which will allow us
to focus on Jammy updates and not various server updates.

We update testing to shift testing to a jammy node as well. We don't
remove gitea-lb01 yet as this will happen after we switch DNS over to
the new server and are happy with it.

Change-Id: I8fb992e23abf9e97756a3cfef996be4c85da9e6f
2022-10-13 13:09:13 -07:00
James E. Blair b127c484c9 Add tracing server to inventory
Change-Id: I31114219a959952a389e0f85699b655a731a7301
2022-09-21 12:13:11 -07:00
Clark Boylan 5cc6c14a6d Remove ethercalc config management
About a month ago we announced [0][1] that this server would be shutdown
and removed on May 31, 2022. Before we can shutdown the server we need
to remove it from config management to prevent Ansible errors. This
change is safe to land now, then on the 31st we can shutdown, snapshot,
and delete the server.


Change-Id: Ic44bed01384845e5b6322eeed02dd0932501cdb3
2022-05-30 12:57:48 -07:00
Jeremy Stanley d185aedd7d Decommission and services
The server is offline now that it no longer
hosts any working services. Remove all configuration for it in
preparation for retiring related Git repositories.

Also roll some related cleanup into this for the already retired
puppet-kibana module.

Change-Id: I3cfcc129983e3641dfbe55d5ecc208c554e97de4
2022-04-29 16:34:51 +00:00
Clark Boylan 4279e20293 Remove configuration management for ELK stack
We indicated to the OpenStack TC that this service would be going away
after the Yoga cycle if no one stepped up to start maintaining it. That
help didn't arrive in the form of OpenDev assistance (there is effort
to use OpenSearch external to OpenDev) and Yoga has released. This means
we are now clear to retire and shutdown this service.

This change attempts to remove our configuration management for these
services so that we can shutdown the servers afterwards. It was a good
run. Sad to see it go but it wasn't sustainable anymore.

Note a follow-up will clean up elastic-recheck which runs on the status

Change-Id: I5f7f73affe7b97c74680d182e68eb4bfebbe23e1
2022-04-18 10:04:06 -07:00
Clark Boylan e2bbf3eb4d Remove health and subunit works from inventory
This was missed in the previous change that removes config management
for these servers. Additionally these servers are offline and in the
emergency file so we should go ahead and remove them from the inventory.

A followup change will remove config management for the subunit workers.

Change-Id: Ic8ecee9f459385fa4de54c5050f0eb8766158368
2022-04-18 08:45:08 -07:00
James E. Blair bd70f7b9d5 Add zuul-lb01 to inventory
Change-Id: I1dd8ee520aba2c5c47801e751b5ed492f0efddd5
2022-03-04 11:17:02 -08:00