This adds a local mariadb container to the gerrit host to hold the
accountPatchReviewDb database. This is inspired by a few things
- since migration to NoteDB, there is only one table left where
Gerrit records what files have been reviewed for a change. This
logically scales with the number of reviews users are doing.
Pulling the stats on this, we can see since the NoteDB upgrade this
went from a very busy database (~300 queries/70 commits per second)
to barely registering one hit per second :
https://imgur.com/a/QGJV7Fw
Thus separating the db to an external host for performance reasons
is not a large concern any more.
- emperically we've done a bad job in keeping the existing hosted db
up-to-date; it's still running mysql 5.1 and we have been hit by
bugs such as the one referenced in-line which silently drops
backups.
- The other gerrit option is to use an on-disk H2 database. This is
certainly an option, however you need special tools to interact
with it for migration, etc. and it's not safe to backup from files
on disk (as opposed to mysqldump). Upstream advice is unclear, and
varies between H2 being a performance bottleneck to this being
ephemeral data that users don't care about. We know how to admin
mariadb/mysql and this allows us to migrate and backup data, so
seems like the best choice.
- we have a pressing need to update the server to a new operating
system. Running the db alongside the gerrit instance minimises
fiddling we have to do manging connections to and migrating the
hosted db systems.
- related to that, we are tending towards more provider independence
for control-plane servers. A hosted database product is not always
provided, so this gives us more flexibility in moving things
around.
- the main concern here is memory usage. "docker stats" reports a
quiescent container, freshly started on a 8GB host:
gerrit-compose_mariadb_1 67.32MiB
After loading a copy of the production table, and then dumping it
back to a file the same container reports:
gerrit-compose_mariadb_1 462.6MiB
The existing remote mysql configuration path remains mostly the same.
We move the gerrit startup into a script rather than a CMD so we can
call it after a "wait for db" script in the mariadb_container case
(this is the reccommeded way to enforce ordering [1]).
Backups of the local container need different dump commands; backups
are relocated to a new file and updated.
Testing is converted to use this rather than a local H2 database.
[1] https://docs.docker.com/compose/startup-order/
Change-Id: Iec981ef3c2e38889f91e9759e66295dbfb499c2e
This enables the new eavesdrop01.opendev.org server in all current
channels. Puppet has been disabled on the old server and we will
manually stop supybot/meetbot and mirgrate logs before this applies.
Change-Id: I4a422bb9589c8a8761191313a656f8377e93422f
This installs our Limnoira/meetbot container and configures it on
eavesdrop01.opendev.org. I have ported the configuration from the old
puppet as best I can (it is very verbose); my procedure was to use the
Limnoira wizard to start a new config file then backport everything
from the old file. I felt this was best to not miss any new options.
This does channel logging (via built-in ChannelLogger plugin, along
with a cron job for logs2html) and runs our fork of meetbot.
It exports the channel logs via HTTP to /irclogs and meetings logs to
/meetings. meetings.opendev.org will proxy to these two locations
when the server is active.
Note this has not ported the channel list; so the bot will not be
listening in our channels.
Change-Id: I9f9a466c271e1a706f9f98f816de0e84047519f1
This host is no longer under puppet control, but should still be a
webserver to export the logs it is still collecting until we finish
moving that to the new server. Restore the match to open*
See I809f9af3e78f566362142790f6c79654ef5b8959
Change-Id: I524c0a7c5cc93313c180eca68b67a0f0582474df
This moves these services to eavesdrop01.opendev.org, a new
Focal-based server to host IRC services.
We have stopped running puppet on eavesdrop01.openstack.org so there
is nothing left for it to do (note the server is still running
meetbot/ptgbot). Remove the commented out puppet run, and remove the
server from puppet groups. Update the host in the Zuul jobs to the
new node.
Change-Id: I809f9af3e78f566362142790f6c79654ef5b8959
This adds a new server to take over from eavesdrop01.openstack.org.
We limit the puppet installs, etc. to the openstack.org server. The
new server is in the group eavesdrop_opendev as we cut over services.
A stub for basic installation is added to the service playbook.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/795004
Change-Id: I88c3059532e4d6ab267fdec5b390daefa5b0c4a1
The openstack-security mailing list is officially closing, and wants
future attempts at posting to end up on openstack-discuss instead:
http://lists.openstack.org/pipermail/openstack-security/2021-June/006077.html
This was also the only remaining user of the notify-impact Gerrit
hook, so we can stop installing/running it.
Change-Id: Id60b781beb072366673b32326e32fd79637c1219
We are trying to replace eavesdrop01.openstack.org
The main landing page serves meeting information which has been moved
to a static site served from AFS at meeting.opendev.org. Redirect
everything to there.
The IRC logs are currently still hosted on eavesdrop01, so while we
work on migrating these, proxy meeting.opendev.org/<irclogs|meetings>
to this server.
Note this will be a no-op until we move the DNS, but we should make
the eavesdrop acme records before merging.
Change-Id: I5c9c23e619dbe930a77f657b5cd6fdd862034301
This site replaces eavesdrop.openstack.org. I think this name makes
more sense.
That is/was being published by jobs directly pushing this onto the
eavesdrop server. Instead, the publishing jobs for irc-meetings now
publish to /afs/openstack.org/project/meetings.opendev.org. This
makes the site available via the static server.
This is actually a production no-op; nothing has changed for the
current publishing. It is still todo to figure out the correct
redirects to keep things working from the existing
eavesdrop.openstack.org and stop the old publishing method.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/794085
Change-Id: Ia582c4cee1f074e78cee32626be86fd5eb1d81bd
This cleans up ask-staging which hasn't been a thing in a log time.
We remove some puppet stubs for nodepool builders (they are all ansible
now).
We also cleanup the inventory file to remove corvustest, lists-dev,
pbx, mirror-update*.openstack.org (is opendev.org now), and sort the
LE list.
Change-Id: I8da025640e16bf6e8aca1eb6ec7799d26bd03f12
This will provision LE certs for openstackid.org. If we are happy with
the results then the child change can be merged to to swap apache over
to using the new cert.
Change-Id: Icc9fdd8a39630323916d1f33d9867f93fc6f2b85
We have decided to decommision the ask.openstack.org server as it is
running EOL Xenial, and its manually purchased certiface is about to
expire. Although it has been deprecated for some time, we feel like
it has been around long-enough as a resource that it is best if we
replace it with a place-holder. The links included here are the same
as the currently shown header explaining the site is read-only.
There's nowhere particularly relevant to redirect the site, so we add
a static file here, and some minimal Ansible to put it in the right
place in a generic way in-case we want to do the same for another
service.
Change-Id: I8a31f8fcf9b3064c0ae58e463a6014dc14b518a7
This provisions the cert then when we are happy with the results we can
land the child change to swap the cert over in apache.
Change-Id: Id8e66102cf26a3b9819d4638b7589f44f6400634
This provisions the cert but doesn't switch apache to it. When we are
happy with the new cert we can land the child change which will flip
apache over to the new cert.
Change-Id: I9cffd26a51317ea569b078b89cc30dc34c7e7747
This runs the LE ansible alongside the ethercalc puppetry to get an LE
cert provision for this service. Once we are happy with the new cert we
can land the followup change to switch to the LE cert.
Note we don't add an altname for the host because that will require
extra DNS records in rax DNS.
Change-Id: I04c062eb994f672283aa30ffcc0c4d45fc8c50f6
This cleans up zuul01 as it should no longer be used at this point. We
also make the inventory groups a bit more clear that all zuul servers
are under the opendev.org domain now.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/790483
Change-Id: I7885fe60028fbd87688f3ae920a24bce4d1a3acd
This zuul02 instance will replace zuul01. There are a few items to
coordinate when doing an actual switch so we haven't removed zuul01 from
inventory here. In particular we need to update gearman server config
values in the zuul cluster and we need to save queues, shutdown zuul01,
then start zuul02's scheduler and restore queues there.
I believe landing this change is safe as we don't appear to start zuul
on new instances by default. Reviewers should double check this.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/791039
Change-Id: I524b456e494124d8293fbe8e1468de40f3800772
This converts our existing puppeted mailman configuration into a set of
ansible roles and a new playbook. We don't try to do anything new and
instead do our best to map from puppet to ansible as closely as
possible. This helps reduce churn and will help us find problems more
quickly if they happen.
Followups will further cleanup the puppetry.
Change-Id: If8cdb1164c9000438d1977d8965a92ca8eebe4df
The Limesurvey service hosted at survey.openstack.org was a beta
which saw limited use. The platform it runs on, Xenial, is now EOL
from Ubuntu/Canonical and in order to upgrade to a newer
distribution release we would need to rewrite all the configuration
management (the version of Puppet supported by newer Ubuntu is not
backward-compatible with what we've been running).
If a similar service becomes interesting to users of our
collaboratory in the future, it will need to be reintroduced with
freshly written configuration management anyway. The old configs and
documentation remain in our Git history should anyone wish to use
them as inspiration.
Change-Id: I59b419cf112d32f20084ab93eb6f2417a7f93fdb
We are doing this so that we can cleanup the private network + floating
IP setup that the existing mirror does. Once this new mirror is up and
happy we can cname to it and then clean up the old mirror and its
networking config. We do this in order to save an IP that the current
private network router is consuming.
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/787628
Change-Id: I50c311087c6c28726e36913c7e081f3b3d0ee049
We have limited ipv4 address space in this cloud. Currently we can do
about 6 IP addresses for test nodes after we account for network
infrastructure and the mirror. By switching these instances to using the
external network directly we can clean up some of the neutron network
infrastructure which we think may free up 2 more IP addresses. That
should get us our originally intended max-servers of 8.
Change-Id: I705ff082ff06ae1c97f4c229a22893e6d87d206d
This updates out inventory to add the new inmotion mirror. This is a
necessary step in bootstrapping this cloud for nodepool usage.
Change-Id: Ie66cdb010c0772310f1cfa8187ca0a2d7f1de1b8
Due to bad copy pasta the new inmotion cloud enrollmant for opendevzuul
was run against osuosl. Fix this to get the right cloud sorted. I will
manually cleanup osuosl.
Change-Id: I56a71734b3e0fd648443ef7d1894ee79cd23077a
This adds the new inmotion cloud to clouds.yaml files and the cloud
launcher config. This cloud is running on an openstack as a service
platform so we have quite a bit of freedom to make changes here within
the resource limitations if necessary.
Change-Id: I2aed6dffde4a1d6e3044c4bd8df4ca60065ae1ea
Add a variable to configure upload-workers for nodepool-builder
daemons.
Reduce our defaults for nb03 to see if we can get more reliable
uploads.
Change-Id: I819bdd262c7118cbde4e6ffdc12aa3ac64569a96
Once we are satisfied that we have disabled the inputs to firehose we
can land this change to stop managing it in config management. Once that
is complete the server can be removed.
Change-Id: I7ebd54f566f8d6f940a921b38139b54a9c4569d8
This handles planet.openstack.org and redirects it to the
opendev.org/openstack/planet-openstack repo, where we will put a
README and the OPML file of the last state as we deprecate this
service.
Change-Id: If141aca5efbdbe60c91ceefaa4e05c98cd0ba5bb
The Oregon State University Open Source Lab (OSUOSL;
https://osuosl.org/) has kindly donated some ARM64 resources. Add
initial cloud config.
Change-Id: I43ed7f0cb0b193db52d9908e39c04e351b3887e3