Add bool to use_upstream_docker conditional
This is an ansible behavior change that's coming in 2.12 but is
currently spewing warnings. The warnings make the log really hard
to read, so just fix it.
Disable group name auto-renaming
If you have group names with non-python identifier characters, it
prevents you from looking it up in jinja like "groups.group-name"
so ansible auto-transforms it so you can do "groups.group_name".
This is a confusing behavior which is going away. However, ansible
is warning everyone who has groups with characters in them as it
has no idea how you might be accessing those group names. Add
a config setting to suppress the warning about -'s in group names.
Change-Id: Ib3262025799af7c3171ed0b079cb1dd969075931
We'll use this to test the checks plugin.
We have to add jgit as a repo because it's a submodule now.
Change-Id: Ic7e9ad0265e136a9ac6b1147998f6eb5ee398180
A few things have changed and we need to fix them in one go.
Use mirror for installing docker for buildset-registry
While, we need to make this more systemic, that's hanging off of the
mirror rework. For now, since we know all of these jobs are debian
based, just set the mirror location.
Replace use of zuul cloner with git clones
You can never be a prophet in your own hometown. This is now broken
because of the git cache rework, so just replace it.
Update libjemalloc library
python:slim is based on buster now, which has libjemalloc2 not
libjemalloc1.
Remove gerrit repo remote for submodules
A recent change to the base jobs to use prepare-workspace-git
broke the gerrit image builds by actually having the origin
remote by /dev/null as intended. This breaks submodules because
for a few of them where we don't have matching stable branches
the submodule relative path behavior is actually exactly what
we want.
Since we don't care about the remote otherwise, remove the
origin remote before doing the submodule update --init so that
the submodule will clone the refs from the zuul prepared repo.
Change-Id: Ieb5b6bc8711fe971ed3445c7c267306ac4616464
Just to eliminate the possibility that a bug in this may be causing
us to erroneously delete blobs.
Change-Id: I048d9ae5ba92984c90f84f231b412050a52fcea6
This is no longer used as read access is provided to unathenticated
users with the recently added JWT support.
Depends-On: https://review.opendev.org/687422
Change-Id: I36fd28710c644be9b07d645c6b0c6092f52a7385
An upcoming change will add JWT authentication to the registry;
prepare for that by establishing a server-side secret for use
in signing the tokens.
Change-Id: Ibaa15dd0c4b0d797f01a1886186fdc021dc990fa
We're trying to get clouds.yaml into /etc/openstack/clouds.yaml.
This should accomplish that. The previous configuration was in
error and wiped out the /etc directory which made things not work
well.
Change-Id: I88e69b05f3e8c1688d24736fa775163fc25a07f0
This change adds a task to the zuul restart playbook to update the zuul
ansible installs on the executors during the restart. This will help
ensure we keep our ansible installs there up to date.
Change-Id: I443e204eaa69dcf0c5622303201549224e893626
This uses the new zuul-registry container image to run the
intermediate registry. The same authentication data and certs
are used. The new registry also writes to the same swift container,
but uses different pseudo-directories so it won't clash with the
current registry. If there are problems, we can switch back easily.
After successful use of the new registry, we can delete the old data.
Change-Id: Ib855fb99c991411293a617b9b238d79a6bfae328
The extant "logrotate_daily" varaible doesn't really do anything and
isn't used. Modify this to be able to set a range of rotation periods
or a size.
Update mirror rsync mirrors to rotate weekly, as often releases run
overnight and it's a pain to reconstruct.
Change-Id: I121dc5f4fe7f226b66d18b9ec39e7e3839be4d40
This adds mirroring of CentOS 8. It is somewhat simpler because the
architecture we're interested in are in the base repos, no need for
altarch.
The current mirror doesn't have a 8/ directory; possibly they require
their own mirroring filter updates? Use an up-to-date mirror for 8
(we can switch 7 too, but leaving alone for now).
Additionally, the altarch mirror we are using appears to have gone
offline for at least a few days. Switch to another one that is in
Texas, which should be close-ish to the DFW servers.
Change-Id: I33d95fa6b2df23fbfdb6745a3079761e228f677b
Looks like leaseweb, kernel.org and others are not properly
syncing the Suse mirrors as they are out of date for Leap15.0 and
missing files (deltainfo.xml.gz) which causes job breakage for
Leap15.0 based jobs.
Revert this to the original mirror from a year ago which is updated
and not broken
Change-Id: Id7184ee973bbabfec3f601fc9200ffac17322558
This change adds a proxy config for quay which should assist
us when gating using images provided by the publically
available registry.
Change-Id: I971705e59724e70bd9d42a6920cf4f883556f673
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This tasks file was removed when we rewrote the gitea management into
python. Unfortunately our gitea rename project tasks rely on this tasks
file to create new orgs if they are needed to rename a project.
Restore this file so that we can rename projects in this manner. Note we
move it into playbooks/ and rename it to gitea-rename-setup-org.yaml to
make it clear that the gitea management role doesn't use this set of
tasks, only the gitea project rename playbook does.
Change-Id: I782f6e56cad18bdcbf9504d51af15caa950a5752
Kolla uses this to build hacluster images.
Direct usage is causing timeouts recently.
I changed the formatting to make it more readable
and slapped a comment note for maintainers.
Change-Id: I68d7155718c0ae0744198ca96aca1a207bab7ed6
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Use latest bazel
It seems 0.27 is now too old. This is what happens when I go on vacation
apparently.
Add in a hack to override the bazelversion. We'll remove this once
https://gerrit-review.googlesource.com/c/gerrit/+/237495 lands and
has been merged up.
Change-Id: Ib7a6d33ce8bf8498fd5cd09b25087dc09acb8df4
As described inline, this should make our mirror pulses more robust
against timeouts.
This is probably ripe for turning into more of a library situation for
all the other "vos release" calls too. But one thing at a time ... I
think we test with this for a while to see if stability returns.
Change-Id: I041a290053e4e8ceba80785598a5945e5adcf6f1
Setting this to system-config allows us to run the base tests as 3rd
party ci for projects like testinfra.
Change-Id: I2d15df154dcdc7c5da6c3326fbecec2146201164
Randomising the time of this job should help avoid a thundering herd
of I/O intensive operations in the gitea environment.
Change-Id: I035f7781a397665357b6d039b989ab9fe6a46b8a
We are debugging why the fedora mirror has started to take 8+ hours to
release. There is suspicion that rsync is updating the metadata of a
lot of files, causing the incremental update to be more of a full
release.
If you read the man page carefully:
At the single -v level of verbosity, this does not mention when a
file gets its attributes changed. If you ask for an itemized list of
changed attributes (either --itemize-changes or adding "%i" to the
--out-format setting), the output (on the client) increases to
mention all items that are changed in any way
So we need "-i" to see these updates
Change-Id: Ia9cff1e126404bc5a3346e758a15314834aef409
We're having trouble with this volume; this makes the logging more
helpful for humans trying to establish the order of things happening.
It turns down the tracing, puts timestamps on the logs messages, makes
the messages more descriptive, and makes sure to timestamp the output
of the vos commands so we can see how long each step took.
Change-Id: Ia832b7ef86350efcbbe7d3b957e820a8a39df3e1
We had some extra bazel options that don't seem to be necessary
anymore now that we are using upstream bazel options appropriately.
Retry the build a couple of times if it goes south, inside of the
build image. This should allow re-use of the cache the second time,
and if there is a temporary error, it should pick up and move
forward.
Change-Id: I5f304acb21fd3a4d40701fc0414ae0c424c838e5
We don't need things like efi configs or pxeboot configs or ISOs.
Exclude these items to further reduce the size of the fedora mirror.
Change-Id: I93003b2f48d79dae627026e2c7af748ea9a9e34d
The ssh config file is /.ssh/config (not ssh_config)
We are accepting the ed25519 key, not the ecdsa key, so fix that in
the known_hosts stanza.
Change-Id: If3a42a7872f5d5e7a2bf9c3b5184fb14d43e6a1a
This introduces two new roles for managing the backup-server and hosts
that we wish to back up.
Firstly the "backup" role runs on hosts we wish to backup. This
generates and configures a separate ssh key for running bup and
installs the appropriate cron job to run the backup daily.
The "backup-server" job runs on the backup server (or, indeed
servers). It creates users for each backup host, accepts the remote
keys mentioned above and initalises bup. It is then ready to receive
backups from the remote hosts.
This eliminates a fairly long-standing requirement for manual setup of
the backup server users and keys; this section is removed from the
documentation.
testinfra coverage is added.
Change-Id: I9bf74df351e056791ed817180436617048224d2c
The fedora mirror is our largest mirror (850GB about twice as big as the
next mirror). Much of this size is due to the fedora atomic images we
mirror.
On further investigation I notice that we are mirroring ppc images (for
which we do not have cpus to run them on), image for fedora 25 and 36
which are quite EOL'd, and our exclusion of the raw.xz and vagrant
images is failing.
Update the rsync excludes to ensure we don't mirror any of these images
we don't need.
Change-Id: I86856cb4e51b0e687aac45a1f014f87c5141318f
pubmirror1.math.uh.edu is currently offline and listed as an altonly.
pubmirror2 seems to work fine so switch to it.
Change-Id: I2562f8686146d17d4fad3997b9be22361fa05fca
This can be used in an apache vhost later, but should be fine to
merge now.
Depends-On: https://review.opendev.org/673902
Change-Id: Ic2cb7585433351ec1bdabd88915fa1ca07da44e7
We don't want nodepool to use floating IPs in the fn cloud as it is an
ipv6 only cloud. We explicitly tell it there is no fip source and that
the tenant network routes ipv6 externally. This config is based on the
limestone configuration which is a similar cloud network wise.
Change-Id: I4a27a22a5beb9c5fc9d3e16cd2ca5b41aecbb46f
Networking got weird on the previous host so we rebuilt this one going
back to networking we expect to work (FIPs and all that). This updates
the inventory so that we configure the host properly.
Change-Id: I0dcdbc9efdd330d66b57da0b01d23dd3d747f79b
Add the gitea07.opendev.org and gitea08.opendev.org servers into the
haproxy pools now that they've been seeded with current data. Remove
the create repos task disable list entries for them as well.
Change-Id: I69390e6a32b01cc1713839f326fa930c376282af
Add the gitea05.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 05 to 07 and 08, and remove 07 and 08 from the
Ansible inventory and comment them out in the haproxy pools in
preparation for replacement.
To the casual observer it may appear gitea06 is being skipped, but
it was replaced first out of sequence due to filesystem corruption
during the PTG. The increased performance of the 75% of the nodes
which have already been replaced means we can get by doing the final
25% at the same time (so two servers at once).
Change-Id: Ia49157c16582b7ed0dbef3eb9d07bf7f1d4450b9