17628 Commits

Author SHA1 Message Date
Jeremy Stanley
7308220484 Switch IPv4 rejects from host-prohibit to admin
When generically rejecting connections, we'd prefer to signal to
users clearly that it's the firewall rejecting them. For IPv4 we
previously emitted generic ICMP "no route to host" responses, but
this tends to make it look incorrectly like a routing failure.
Switch to flagging our error responses as "administratively
prohibited" which is more accurate and less confusing. We're also
already using icmp6-adm-prohibited for the v6 rules, so this makes
our v4 ruleset more consistent.

Note that the iptables-extensions(8) manpage indicates "Using
icmp-admin-prohibited with kernels that do not support it will
result in a plain DROP instead of REJECT" but all our kernels should
have support for it these days so this isn't a concern.

Change-Id: Id423f3ec03d0c3c4e40ddef34c38f97167b173f6
2021-09-21 18:19:06 +00:00
Clark Boylan
522afa85cc GC/pack gitea repos every other day
Previously we were doing this weekly. Gerrit does this daily. "Split"
the difference and do gitea every other day.

We have noticed that replication to gitea can be slow at times. One idea
is that the less packed repos on the gitea side may make negotiating the
updates slower. Pack more often to see if this helps.

Change-Id: I8961007dce3e448bfdbf1c5f3e8dfc5ec8eb82fb
2021-09-21 08:32:15 -07:00
Zuul
fc75255ea9 Merge "gerrit: copy theme plugin from plugins/" 2021-09-21 15:13:09 +00:00
Zuul
02705bf73a Merge "lodgeit: use logo from system-config assets" 2021-09-21 14:28:18 +00:00
Zuul
3791aa78c7 Merge "docs: use logo from assets" 2021-09-20 19:42:57 +00:00
Zuul
856ee196f5 Merge "gitea: cleanup logo assets" 2021-09-20 19:42:54 +00:00
Zuul
57d2e67a5c Merge "Run daily backups of nodepool zk image data" 2021-09-20 17:34:28 +00:00
Ian Wienand
f19e41e893 gerrit: host logo in static files
Instead of using the opendev.org/... logo file, host a copy from
gerrit's static location and use that.  This isolates us from changes
to the way gitea serves its static assets.

Change-Id: I8ffb47e636a59e5ecc3919cc7a16d93de3eae08d
2021-09-17 12:35:12 +10:00
Ian Wienand
227839e8ad gerrit: copy static files directly into container image
Copy static files directly into the container image instead of
managing them dynamically with Ansible.

Change-Id: I0ebe40ad2a97e87b00137af7c93a3ffa84929a2e
2021-09-17 12:35:12 +10:00
Ian Wienand
e8c456926c lodgeit: use logo from system-config assets
This currently uses a file served from gitea's staic assets; to
isolate us from changes to gitea's file layout switch this to use the
canonical file directly from system-config/assets.

Change-Id: Ibf67040af2b0a18261621a120ee26c78020e3ace
2021-09-17 12:35:12 +10:00
Ian Wienand
791d09be46 gerrit: copy theme plugin from plugins/
This is a minor refactor to make a follow-on that copies static files
more logical.

Change-Id: Id31106a875926dd4eca31972b1fc52a1cd0b67bd
2021-09-17 12:35:12 +10:00
Ian Wienand
1d5f5a7657 docs: use logo from assets
Symlink the docs logo to the canonical assets location.  It looks like
it does the sensible thing and de-references the source symlink when
building, as doc/build/html/_static/opendev.svg ends up as the actual
file, not a symlink.

Change-Id: I4409c8e20601bdcb9e387d028b5df13c90d1ffa0
2021-09-17 12:35:12 +10:00
Ian Wienand
8f8100ed28 gitea: cleanup logo assets
I can not find anywhere we use the logo-*.png files currently in the
gitea public directory.  Remove these as they're all trivially
re-created via the SVG if we find we want them.

Similarly I don't see anywhere we use "opendev-icon.svg"; it is the
same as "logo.svg".  This is removed.

Move the logo with text "opendev.svg" into the assets bundle (only
user of this I can find is paste.opendev.org).

Also move in the favicon.<png/svg> files

Note all these are copied back into the gitea container at the same
location so nothing will break.  However we wish to have the
identified external users not rely on files served as gitea assets as
the new version of gitea will move them.

Change-Id: I4f6c64b4042a3f0a17ce4ee59ee8bd0d61648bcc
2021-09-17 12:35:07 +10:00
Clark Boylan
a1cf5b3f6f Run daily backups of nodepool zk image data
This does local backups of the nodepool zk image image data to
/var/log/nodepool on the nodepool-builders. These hosts don't get
offsite backups but we run mutliple redundant servers. This data isn't
critical as we can start from scratch, but may be useful if we don't
want to go through all that trouble.

Change-Id: I7d150df9c0d9566ef2d32167cea535e29822cfa2
2021-09-16 14:12:08 -07:00
Clark Boylan
f1bcb6a586 Set a gerrit replication timeout of 15 minutes
We are seeing that replication tasks occasionally sit around forever and
have had to take manual intervention. One theory is that this is related
to networking between the gerrit server and the gitea servers. We don't
set maxRetries which means replication should be retried infinitely
which means if we hit the timeout we should try again. 15 minutes was
sort of arbitrarily chosen as ~twice the time it takes to clone a large
repo like nova.

Change-Id: Iec2536ad149a2e625a1f0107b9fcee3079493607
2021-09-15 16:25:04 -07:00
Clark Boylan
88a6a202e9 Build our gerrit images on Bullseye
This is a bit of spring cleaning. Previously we based on images on
Buster but Bullseye exists now so give it a go.

Change-Id: Icc3d79b361e41df2f2f063993fd206ab7d992f75
2021-09-15 13:45:31 -07:00
Clark Boylan
17ba858b19 Update our gitea images to bullseye
To do this we also update jinja-init to bullseye and gitea seems to be
the only user of this image. The impact of this should be fairly self
contained to gitea.

Note this update isn't urgent, but good hygiene. We should coordinate
this update with the 1.15.x gitea upgrade and do them in such a sequence
that we can identify problems easily if they pop up.

Change-Id: Ia0075416a1d8a067cfecd26c03f8db9641cbcb89
2021-09-15 10:49:42 -07:00
Clark Boylan
6c115cf29e Add support for Ubuntu Focal to our mailman ansible
This switch testing of lists.openstack.org to Focal and we make a CGI
env var update to accomodate newer mailman.

Specifically newer mailman's CGI scripts filter env vars that it will
pass through. We were setting MAILMAN_SITE_DIR to vhost our mailman
installs with apache2, but that doesn't pass the filter and is removed.
HOST is passed through so we update our scripts, apache vhost configs,
exim, and init scripts to use the HOST env var instead.

Change-Id: I5c8c70c219669e37b7b75a61001a2b7f7bb0bb6c
2021-09-13 09:10:00 -07:00
Zuul
4db138edfd Merge "testinfra: refactor screenshot taking" 2021-09-07 17:05:31 +00:00
Zuul
7a90a92ebb Merge "gitea: add some screenshots to testing" 2021-09-07 16:57:45 +00:00
Zuul
669300d7e2 Merge "gitea: use assets bundle" 2021-09-07 16:57:41 +00:00
Zuul
7a9c9475a7 Merge "Add assets and a related docker image/bundle" 2021-09-07 15:47:23 +00:00
Ian Wienand
9ba398dee9 testinfra: refactor screenshot taking
Reduce the screenshots to a single utility function to avoid copying a
lot of boilerplate.

Change-Id: Iad1c7afa4e9ea9a4ddaca5e62751795e60bc2980
2021-09-07 12:54:26 +10:00
Ian Wienand
1dde7628e8 gitea: add some screenshots to testing
Change-Id: Id13fdd8ffbca1b0cd19858419d68f012e33f3ba8
2021-09-07 08:59:46 +10:00
Ian Wienand
e772abaf96 gitea: use assets bundle
This uses the opendev assets bundle image created with
I3166679bde6d771276289b9d32e7e4407957b2f8.

The mount options require using BuildKit, hence the Dockerfile update.

Otherwise conceptually it's fairly simple; copy in the files from the
opendevorg/assets image rather than the file-system.

Change-Id: I36bdc76471eec5380a676ebcdd885a88d3985976
2021-09-06 15:07:36 +10:00
Zuul
300982c6c1 Merge "Accomodate zuul's new key management system" 2021-09-01 15:56:40 +00:00
Zuul
da558e10c3 Merge "Introduce iweb cloud configs" 2021-08-31 23:38:46 +00:00
Mathieu Gagné
d518f8c02c Introduce iweb cloud configs
INAP mtl01 region is now owned by iWeb. This updates the cloud launcher
to use the new name and instructs the mirror in this cloud to provision
ssl certs for the old inap and new iweb names as well as updating
clouds.yaml files.

Change-Id: I1256a2e24df1c79dea06716ae4dfbcfe119c13f8
2021-08-31 13:42:22 -07:00
Ian Wienand
25cdc97950 Add assets and a related docker image/bundle
Move some common assets into a top-level assets/ directory.  Services
can reference these assets via

 https://opendev.org/opendev/system-config/raw/branch/master/assets/<file>

in <img> tags, etc.

Some services want to embed these into their images, but we wish to
only keep one canonical copy.  For this, add a Dockerfile and jobs
that creates a simple bundle of assets in opendevorg/assets.  This can
be referenced in other builds; the new BuildKit bind-mount is
particularly useful for this
(c.f. I36bdc76471eec5380a676ebcdd885a88d3985976).

Change-Id: I3931566eb86a0618705d276445fa0a5f659692ea
2021-09-01 06:15:43 +10:00
Jeremy Stanley
2fbf6d9e7a Stop managing OpenStackID servers
The Open Infrastructure Foundation's developers who maintain the
OpenStackID software are taking over management of the site itself,
and have deployed it on new servers. DNS records have already been
updated to the new IP address, so it's time to clean up our end in
preparation for deleting the old servers we've been running.

OpenStackID is still used by some services we run, like RefStack and
Zanata, and we're still hosting the OpenStackID Git repository and
documentation, so this does not get rid of all references to it.

Change-Id: I1d625d5204f1e9e3a85ba9605465f6ebb9433021
2021-08-31 19:53:13 +00:00
Zuul
861f4698aa Merge "Switch robots.txt test on gitea to proxy port" 2021-08-30 02:04:50 +00:00
Monty Taylor
8dbf0a3d82 Produce both buster and bullseye container images
This will allow us to roll out consumption forward in a
methodical manner.

This reverts commit 45caec4d43900bc66fb0b8c219c6dcc3180ca8aa.

Note the weird ARG definitions are there for a reason:
https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact

Change-Id: I81174ac035164695f1c27c9662f25335b78c2e64
2021-08-28 16:46:29 -07:00
Zuul
0f3b1f649b Merge "Pin base and builder images to buster" 2021-08-27 17:11:01 +00:00
Zuul
6969f4a033 Merge "Test lists.kc.io on focal" 2021-08-27 16:00:48 +00:00
James E. Blair
45caec4d43 Pin base and builder images to buster
There's some more work before our consumers can switch to bullseye.
To make this process more tractable, revert the recent backport
addition, and specify that we want bullseye images from upstream.

That gets us back to where we were at the start of this.  Next,
we can start building 2x images of python-base/builder and tag
them with bullseye or buster.  Then the consumers can specify
which tags, then start switching.

Revert "Add backports repos to base and builder images"

This reverts commit b217e38904da3ccab6eb96251376f1635ee55d21.

Revert "Update matrix-eavesdrop for bullseye"

This reverts commit fc38c6975367c09d003474ebd7bfefc465459a06.

Change-Id: Id21681342fe5268296128c1b09436a80c46e3169
2021-08-27 08:03:38 -07:00
Zuul
84dca1ecf4 Merge "Update OpenSUSE mirror source" 2021-08-26 22:57:58 +00:00
Zuul
a6beb03ceb Merge "Update matrix-eavesdrop for bullseye" 2021-08-26 22:34:09 +00:00
Zuul
aaeebdda18 Merge "Add backports repos to base and builder images" 2021-08-26 20:37:44 +00:00
Monty Taylor
fc38c69753 Update matrix-eavesdrop for bullseye
Bullseye means we don't need the backports repos. Also, the upstream
images have bullseye-backports in them already now.

Depends-On: https://review.opendev.org/c/opendev/system-config/+/800318
Change-Id: I3813068c21d06d9b182fe81efcf2e636b2170c4a
2021-08-26 15:11:37 -05:00
Monty Taylor
b217e38904 Add backports repos to base and builder images
These don't install anything by default, but allow people
to easily opt-in to a backport package if they need to without
lots of extra lines of boilerplate.

The base python image is on bullseye now instead of buster. That
means the libffi version is 7 not 6.

Change-Id: I0e0c2669d838fb622422f696f73e96e409157270
2021-08-26 11:58:59 -05:00
Zuul
b6e9467bb1 Merge "Retire openstack-i18n-de mailing list" 2021-08-26 06:50:47 +00:00
Jeremy Stanley
cefd919686 Update OpenSUSE mirror source
The rsync mirror we were relying on ended up incomplete on a recent
sync, causing all OpenSUSE 15 jobs to fail updating the package
lists. Switch to an alternative that seems to have all the same
things for which we used the previous one.

Change-Id: I661bdbfcbc766966793cd64d7f21201879d3dbaa
2021-08-25 20:10:59 +00:00
Zuul
74713e4d31 Merge "borg-backup: randomise time on a per-server basis" 2021-08-24 20:22:08 +00:00
Zuul
dac6ae68b9 Merge "Restrict generic inventory matchers to inventory/base" 2021-08-24 19:30:51 +00:00
Zuul
821c9a40f3 Merge "Preserve zuul executor SIGTERM behavior" 2021-08-23 23:34:32 +00:00
Clark Boylan
43e1dce889 Preserve zuul executor SIGTERM behavior
There is a change (the depends on) to modify how zuul executors handle
SIGTERM. Update our executor config to preserve the old behavior of
stopping the instance immediately rather than doing a graceful stop.

If we need to we can still request graceful stops directly using the
graceful stop command.

Depends-On: https://review.opendev.org/c/zuul/zuul/+/804464/
Change-Id: I76a2646a13a71d190be265354de18468bc93184c
2021-08-23 10:03:41 -07:00
Zuul
6d49588b59 Merge "Test a gerrit 3.2 -> 3.3 upgrade" 2021-08-23 17:01:55 +00:00
Zuul
c74d9fc54d Merge "Update Gerrit images to most recent releases" 2021-08-23 16:58:40 +00:00
Andreas Jaeger
afb343f4bf Retire openstack-i18n-de mailing list
The mailing list is dead, the last post was about the current owner
leaving:
http://lists.openstack.org/pipermail/openstack-i18n-de/2020-September/000165.html

Let's retire it.

Change-Id: Iad93a65e8f68397ba3f83d8114c30ae78a930e24
2021-08-23 15:54:10 +02:00
Zuul
e76f2308b3 Merge "Update gerritbot-matrix version to fix a message typo" 2021-08-21 20:28:46 +00:00