The OpenEdge cloud has been offline for five months, initially
disabled in I4e46c782a63279d9c18ff4ba2944c15b3027114b, so go ahead
and clean up lingering references. If it is restored later, this can
be reverted fairly easily.
Depends-On: https://review.opendev.org/783989
Depends-On: https://review.opendev.org/783990
Change-Id: I544895003344bc8202363993b52f978e1c07d061
review02.opendev.org is a much larger replacement server for review01
provided by Vexxhost. It is up and running, with gerrit2 volume
attached and DNS entries.
This adds it to the staging group with no replication and a local h2
database configured for initial bringup. There's quite a bit to
consider for full migration, but this will let us start experimenting.
Change-Id: I3638a5c0c7028dcc800ada42431b75395cff0c42
Create a review-staging group so we can bring up a new server but
avoid running the project-management steps on it.
Change-Id: I93d2a36edcd58a48a36031f0692be3273a36f07c
With our increased ability to test in the gate, there's not much use
for review-dev any more. Remove references.
Change-Id: I97e9865e0b655cd157acf9ffa7d067b150e6fc72
When we cleaned up the puppet in
I6b6dfd0f8ef89a5362f64cfbc8016ba5b1a346b3 we renamed the group
s/refstack-docker/refstack/ but didn't move the variables and some
other references too.
Change-Id: Ib07d1e9ede628c43b4d5d94b64ec35c101e11be8
We duplicate the KDC settings over all our kerberos clients. Add
clients to a "kerberos-client" group and set the variables in a group
file.
Change-Id: I25ed5f8c68065060205dfbb634c6558488003a38
This adds a program, zookeeper-statsd, which monitors zookeeper
metrics and reports them to statsd. It also adds a container to
run that program. And it runs the container on each of the
ZooKeeper quorum members. And it updates the graphite host to
allow statsd traffic from quorum members. And it updates the
4-letter-word whitelist to allow the mntr command (which is used
to gather metrics) to be issued.
Change-Id: I298f0b13a05cc615d8496edd4622438507fc5423
This is a follow-on to I60b40897486b29beafc76025790c501b5055313d to
switch the KDC servers to Ansible control and remove any related
puppet configuration.
Change-Id: Ib8f6ec657ca10a3ba648bd154a035fc3d8da4be5
This adds a role and related testing to manage our Kerberos KDC
servers, intended to replace the puppet modules currently performing
this task.
This role automates realm creation, initial setup, key material
distribution and replica host configuration. None of this is intended
to run on the production servers which are already setup with an
active database, and the role should be effectively idempotent in
production.
Note that this does not yet switch the production servers into the new
groups; this can be done in a separate step under controlled
conditions and with related upgrades of the host OS to Focal.
Change-Id: I60b40897486b29beafc76025790c501b5055313d
This got copied from another command that also had this typo.
Also, don't bother backing up the on-disk backups, as we backup
directly via the stream dumps.
Change-Id: Ie200a29eec2b1a0725a8872ab548bcb0f26980e6
openid_endpoint was set to just base site url which is not
correct, it should be https://openstackid.org/accounts/openid2
Change-Id: I6624150f1ab78560347c8f82a13394b164860cad
The production server is trying to send itself to
refstack01.openstack.org, causing cross-site scripting issues. In
production, use the CNAME, but use the FQDN for testing.
Fix up job file matchers while here.
Change-Id: I18a5067ee25c59c5eaa17b7c2d9bd5a942a9173d
This server has been replaced by ze01.opendev.org running Focal. Lets
remove the old ze01.openstack.org from inventory so that we can delete
the server. We will follow this up with a rotation of new focal servers
being put in place.
This also renames the xenial executor in testing to ze12.openstack.org
as that will be the last one to be rotated out in production. We will
remove it from testing at that point as well.
We also remove a completely unused zuul-executor-opendev.yaml group_vars
file to avoid confusion.
Change-Id: Ida9c9a5a11578d32a6de2434a41b5d3c54fb7e0c
This group no longer does anything. This used to deploy a bunch of
keytabs for mirror-update, but that has all moved into
"mirror_update_keytab_*".
Change-Id: I3e2110a621d6946bc4838bfa2f743f0e9db391f3
This should be called "_extra" ... currently it overrides the default
exclude list. This means /var/lxcfs gets incorrectly included in the
backup and makes it error out as it has sockets and weird stuff that
can't be backed up; this is why we are getting failure mail.
Change-Id: Idea70c32b2d42f77fee2b35487d88a8ee982c856
All hosts are now running thier backups via borg to servers in
vexxhost and rax.ord.
For reference, the servers being backed up at this time are:
borg-ask01
borg-ethercalc02
borg-etherpad01
borg-gitea01
borg-lists
borg-review-dev01
borg-review01
borg-storyboard01
borg-translate01
borg-wiki-update-test
borg-zuul01
This removes the old bup backup hosts, the no-longer used ansible
roles for the bup backup server and client roles, and any remaining
bup related configuration.
For simplicity, we will remove any remaining bup cron jobs on the
above servers manually after this merges.
Change-Id: I32554ca857a81ae8a250ce082421a7ede460ea3c
Ifa5f251fdfb8de737ad2ed96491d45294ce23a0c renamed the afs and afsdb
groups to afs-file-server and afs-db-server, but didn't update the
group files.
Previously the firewall rules were duplicated in the afs/afsdb group;
but now all afs servers are in the afs-server-common group. Rename
afs.yaml->afs-server-common.yaml and remove the now unnecessary
afsdb.yaml.
Remove one of the old group vars files and rename the other to
afs-server-common so we can restore the udp ports they open in our
firewall rules.
Change-Id: I17dd0596660addf061ade31b4450bf040c01ffe8
Both the filesevers and db servers have common key material deployed
by the openafs-server-config role. Put both types of server in a new
group "afs-server-common" so we can define this key material in just
one group file on bridge.
Then separate out the two into afs-<file|db>-server groups for
consistent naming.
Rename afs-admin for consistent naming.
The service file is updated to reflect the new groups.
Change-Id: Ifa5f251fdfb8de737ad2ed96491d45294ce23a0c
Currently this variable is setting several URL's used in the config to
internal http links (port 8000). This bubbles through to the UI which
then can't talk to the API. Emperically, changing these values in the
container config and restarting it makes things work. Update this
variable to make it talk to external https.
Change-Id: If61ec1e0383b98d34d092c55ca0095588487902a
This adds a dockerfile to build an opendevorg/refstack image as well as
the jobs to build and publish it.
Change-Id: Icade6c713fa9bf6ab508fd4d8d65debada2ddb30
As noted inline, a recent mysql client update has broken the
"--all-databases" flag, at least for the client version and very old
server version we use.
Emperically, dumping individual databases still works with this
client. Switch this to stream the db directly into borg.
Ignore the old backups and remove the bup backup while we are here,
since this is all borg now.
Change-Id: I5fe762a003ce2c2ba4830367be87598f67f7e763
Despite be deprecated, the ask server is our 3rd biggest backup. Even
though the site is R/O we're still backing up the fresh rotations of
the gzipped backups every day.
To reduce the incremental space requirements, move to our plain-text
streaming for the db backup. This just needs a file dropped in /etc;
see the backup-borg role README documentation. We do this in puppet
to avoid complexity adding this deprecated service to ansible. This
then excludes the on-disk db backup dir.
Drop the bup backups while we are here.
Change-Id: Icfd81aca58b9a0dc3a3b74de04c1b00f03160327
Add facility to borg-backup role to run a command and save the output
of it to a separate archive file during the backup process.
This is mostly useful for database backups. Compressed on-disk logs
are terrible for differential backups because revisions have
essentially no common data. By saving the uncompressed stream
directly from mysqldump, we allow borg the chance to de-duplicate,
saving considerable space on the backup servers.
This is implemented for our ansible-managed servers currently doing
dumps. We also add it to the testinfra.
This also separates the archive names for the filesystem and stream
backup with unique prefixes so they can be pruned separately.
Otherwise we end up keeping only one of the stream or filesystem
backups which isn't the intention. However, due to issues with
--append-only mode we are not issuing prune commands at this time.
Note the updated dump commands are updated slightly, particularly with
"--skip-extended-insert" which was suggested by mordred and
significantly improves incremental diff-ability by being slightly more
verbose but keeping much more of the output stable across dumps.
Change-Id: I500062c1c52c74a567621df9aaa716de804ffae7
Move common setup steps into a openafs-server-config role, and create
openafs-file-server and openafs-db-server roles to manage fileserver
and db servers respectively.
Modify the playbook to run these roles against the AFS servers.
Change-Id: I4e80ad8ffe1d4992e405ea516b8762109758d7eb
With all AFS file-servers upgraded to 1.8, we can move afs01.dfw back
and rename the group to just "afs".
Change-Id: Ib31bde124e01cd07d6ff7eb31679c55728b95222
This starts at migrating OpenAFS server setup to Ansible.
Firstly we split up the groups and explicitly name hosts, as we will
me migrating each one step-by-step. We split out 1.8 hosts into a new
afs-1.8 group; the first host is afs01.ord.openstack.org which already
has openafs 1.8 installed manually.
An openafs-server role is introduced that does the same setup as the
extant puppet.
The AFS job is renamed to infra-prod-afs as the puppet component will
eventually disappear. Otherwise it runs in the same way, but also
runs the openafs-server role for the 1.8 servers.
Once this is merged, we can run it against afs01.ord.openstack.org to
ensure it works and is idempotent. We can then take on upgrading the
other file servers, and work further on the database servers.
Change-Id: I7998af43961999412f58a78214f4b5387713d30e
The old ethercalc01 server has been deleted as have its DNS entries.
Belatedly update cacti to query the new server, and remove an old
unused reference which was at one time disabling the former server.
Change-Id: Ide70c7d03bfff5bd695272c696913dfb3decc525