Over a few upgrades, we've managed to break some of the default avatar
logos you see when browsing code on opendev.org.
After investigating ways to fix this up, we established that there
isn't an exposed API for setting these, but we can do a simple query
to point to logo files on disk. This implements that.
One caveat is that the logos should be PNG files; particiularly we
note that SVG files don't work reliably because they don't get served
with the image/svg+xml mime-type.
Change-Id: Ie6799de2fb27e09f936c488258dc1bd1c638c370
Gitea 1.16 enabled clone filters by default. Unfortunately pip passes
--filter=blob:none when fetching git resources and the new gitea support
for filters breaks against that filter. We are working around this by
restoring the 1.15 behavior of not supporting filters and this change
will test the behavior is as expected.
Change-Id: I13d57e3cc7e135058ff320b3bd9bea76fb178064
Gitea 1.16 added partial clone support, but the clone filters pip
tries to apply (--filter=blob:none) don't work well when combined
with older cgit clients and lead to errors like "Server does not
allow request for unadvertised object" or "protocol error: bad pack
header".
Explicitly disable this feature server-side for now, so that clients
will fall back to making full clones.
Change-Id: Ia86394d5176c28567bf67b60578aadde6629c775
Depends-On: https://review.opendev.org/834196
None of the services we operate rely on openstackid.org any longer,
so we can drop our monitoring of its cert expiration safely (which
is currently complaining). We're already monitoring its successor,
id.openinfra.dev.
Change-Id: I059ef0492f05137fa542c819b64427bd9ef0eb0c
openEuler yum mirror in Russia is down. This patch change the
rsync url to the official HongKong one.
This patch also fix the openEuler mirror url nit.
Change-Id: Ifb930e34fd7f16f77ba55bc489e5389c641139de
Gitea 1.16.4 is now available. Note that this update includes the
changes from 1.16.0-1.16.3 as well since we are upgrading from
1.15.x. The changelog can be found at:
https://github.com/go-gitea/gitea/blob/v1.16.4/CHANGELOG.md
In particular this calls out:
https://github.com/go-gitea/gitea/pull/17846
as a potentially breaking change that may impact our use of ssh. We
attempt to update our Dockerfile to use the correct gitea command script
to address this but we should likely test replication before landing
this update.
The changelog is quite large and I haven't been able to fully examine it
for impacts. Reviewers are encouraged to look it over and find items we
should address. Additionally once this is reliably building we should
hold a node and inspect it directly.
Change-Id: I0bf7400d43583a8e8b54581225c70cba53007876
Because "." is a field separator for graphite, we're incorrectly
nesting the results.
A better idea seems to be to store these stats under the job name.
That's going to be more helpful when looking up in Zuul build results
anyway.
Follow-on to I90dfb7a25cb5ab08403c89ef59ea21972cf2aae2
Change-Id: Icbb57fd23d8b90f52bc7a0ea5fa80f389ab3892e
We used to track the runtime with the old cron-based system
(I299c0ab5dc3dea4841e560d8fb95b8f3e7df89f2) and had a dashboard view,
which was often helpful to see at a glance what might be going wrong.
Restore this for Zuul CD by simply sending the nested-Ansible task
time-delta and status to graphite. bridge.openstack.org is still
allowed to send stats to graphite from this prior work, so no ports
need to be opened.
Change-Id: I90dfb7a25cb5ab08403c89ef59ea21972cf2aae2
As found in Ie5d55b2a2d96a78b34d23cc6fbac62900a23fc37, the default for
this is to issue "OPTIONS /" which is kind of a weird request. The
Zuul hosts currently seem to return the main page content in response
to a OPTIONS request, which probably isn't right.
Make this more robust by just using "HEAD /" request.
Change-Id: Ibbd32ae744af9c33aedd087a8146195844814b3f
Now that we can confirm this hasn't broken for gitea01, set check on
all the remaining server lines as well.
Change-Id: I11f1f15210dafed66e1209329ddf7f3838592881
Apparently the check-ssl option only modifies check behavior, but
does not actually turn it on. The check option also needs to be set
in order to activate checks of the server. See §5.2 of the haproxy
docs for details:
https://git.haproxy.org/?p=haproxy-2.5.git;a=blob;f=doc/configuration.txt;h=e3949d1eebe171920c451b4cad1d5fcd07d0bfb5;hb=HEAD#l14396
Turn it on for all of our balance_zuul_https server entries.
Also set this on the gitea01 server entry in balance_git_https, so
we can make sure it's still seen as "up" once this change takes
effect. A follow-up change will turn it on for the other
balance_git_https servers out of an abundance of caution around that
service.
Change-Id: I4018507f6e0ee1b5c30139de301e09b3ec6fc494
This was running on all group var updates but we only need to run it
when refstack group vars update. Change the file requirements to match
the refstack.yaml group file to address this.
Change-Id: Id5ed4b65c1ed6566696fea9a33db27e9318af1a6
Switch the port 80 and 443 endpoints over to doing http checks instead
of tcp checks. This ensures that both apache and the zuul-web backend
are functional before balancing to them.
The fingergw remains a tcp check.
Change-Id: Iabe2d7822c9ef7e4514b9a0eb627f15b93ad48e2
This introduces and "Open Infrastructure" page which is designed for a
moderately experienced developer with some understanding of Zuul,
Ansible and basic Linux admin skills to have an entrypoint to
navigating the system-config and related repositories.
It is designed to re-enforce the idea of open infrastructure, and
explain how development, testing and production come together at a
level high enough to be understood, but with links or descriptions of
specific places in the code to get started.
It moves a little of what was in the sysadmin page into this, and
leaves that page as more low-level descriptions of various tasks.
Change-Id: I60a9299df455b98ad549ac0075a59d381722bc06
This plugin was updated to accomodate the ${hash} substition in gerrit
gitweb weblinks. We now need this updated version to build Gerrit
successfully but there is no tag for it yet. Just use the branch to
address this.
Change-Id: I4b0fd4ac845cc4289f78aacfa536db4185f12d38
We have discovered that it is possible for a gitea repository to be come
corrupted. Since gitea is not the source of truth the easiest way to
handle this is to replace the repo with a new empty repository and have
Gerrit replicate back to it. This adds documentation that walks through
the process of doing this.
Change-Id: Ief990adaaf3cbb3c748bc9ee6ceb466a1104915a
Change I5b9f9dd53eb896bb542652e8175c570877842584 introduced this tee
to capture and encrypt the logs. However, we should make sure to fail
if the ansible runs fail. Switch on pipefail, which will exit with an
error if the earlier parts of the pipeline fail. Also make sure we
run under bash.
Change-Id: I2c4cb9aec3d4f8bb5bb93e2d2c20168dc64e78cb
We've been told these resources are going away. Trying to remove them
gracefully from nodepool. Once that is done we can remove our configs
here.
Depends-On: https://review.opendev.org/c/openstack/project-config/+/831398
Change-Id: I396ca49ab33c09622dd398012528fe7172c39fe8
The dependent change enables the "detect-ref" option of hound, which
looks at the remote origin HEAD and indexes on that. That should
allow indexing of our mixed repos that have a mix of "master" and
"main".
Add cirros to the test, which should exercise this path, and take some
screenshosts because this a js/react app and just a "curl" doesn't
help.
Change-Id: I1850577c63566b594f9730f5b8f0bc10b07ff7e4
Depends-On: https://review.opendev.org/c/opendev/jeepyb/+/830919
These were added when we faced significant memory pressure on the old
server. That is no longer a problem and there is an issue with the
specification that breaks file compression due to destination files
already existing. It seems like the log specification is only able to
rotate once then it cannot keep moving files aside because they already
exist as eg jvm_gc.log.0.gz. This results in annoying errors in the
Gerrit error_log.
Note that it doesn't appear sufficient to remove this log specification
we also need to move the existing jvm_gc.log* files aside or delete
them. This was tested on a held zuul node and I stopped gerrit, updated
the docker-compose file, moved the files aside, then started gerrit and
that got rid of the startup errors in error_log. Merely updating
docker-compose resulted in the same errors on startup.
Change-Id: Ied1464c57b2e8331b9bdf7cbc9ad74f92dea2dfd
The enterprise-wg and product-wg lists were deleted from the
openstack site per the announcement[*] on 2022-02-01, but I
neglected to push a change to remove them from our configuration
management, so Ansible helpfully recreated them for me. Clean this
up so I can re-remove the lists once and for all.
[*] http://lists.openinfra.dev/pipermail/foundation/2022-February/003048.html
Change-Id: Iddcb5cbac68d426e0ad13dd41541ad1371366bb1
We have validated that the log encryption/export path is working, so
turn it on for all prod jobs.
Change-Id: Ic04d5b6e716dffedc925cb799e3630027183d890