145 Commits

Author SHA1 Message Date
Clark Boylan
4dc053ff0f Stop building python3.9 container images
Drop the python3.9 container images (python-builder, python-base,
uwsgi-base). At this point nothing is building off of these images and
we should be using python3.11 for anything new.

Note that python3.10 can be cleaned up once zuul-registry and
openstackclient stop building on top of python3.10. Everything else
appears to be python3.11 at this point.

Change-Id: Id30b616ec336a5599766fb808f55e228da686439
2023-10-16 13:52:07 -07:00
Zuul
51003247f8 Merge "Update Gerrit to 3.7.5" 2023-10-11 23:31:19 +00:00
James E. Blair
e85ab6f746 Add a debug python base image
This adds a python-base:3.11-bookworm-debug image, which is built
on the normal python:3.11-bookworm upstream image instead of the
slim upstream image.  The normal image includes debug symbols for
the python interpreter which is compiled during its build phase,
so this is the best way to get an opendev python-base image with
debug symbols.

Change-Id: I1d89ac947cd3bea8a468f3ee022fb4cc93bece1f
2023-10-10 08:30:32 -07:00
Zuul
cac37a7a3c Merge "Update gerrit image to bookworm" 2023-10-09 16:19:47 +00:00
Clark Boylan
00c098bcad Bump refstack to Bookworm and python3.11
Refstack doesn't rely on much in the base operating system as far as I
can tell. That said refstack seems to test with python3.10 and not 3.11
so this may not work, but our testing should give us a good idea. Bump
these things up as we are updating all the services we can in order to
eventually cleanup image builds for old debian and old python.

Change-Id: Id39027691484e8f81bd097c174f0a4a1e81463af
2023-09-19 10:22:56 -07:00
Clark Boylan
da41905b79 Update Gerrit to 3.7.5
Gerrit just released new point versions for all supported branches
including an update of 3.7 to 3.7.5. We also bump our 3.8 image up to
3.8.2 (note this isn't used in production, only for testing of the
upgrade to 3.8 currently).

Release notes can be found here:

  https://www.gerritcodereview.com/3.7.html#375
  https://www.gerritcodereview.com/3.8.html#382

The updates look minor for us, but there are some bug fixes so worth
updating. We might want to land this change as well as the bookworm
update together in order to do a single short gerrit downtime to get all
of these updates deployed in production.

Change-Id: Ib4ccfe12db94d032fc4743a7aafdf90735aecfa3
2023-09-15 15:39:34 -07:00
Clark Boylan
2888408833 Convert haproxy-statsd to bookworm base image
We keep the same python 3.11 version we had on bullseye but switch the
rest of the userspace to bookworm. Since the python version doesn't
change this should have minimal impact.

Change-Id: I59ad8c2a92159f51d567dd0212e2ab8bec1b45b1
2023-09-14 13:36:18 -07:00
James E. Blair
fec1277185 Update gerrit image to bookworm
And upgrade to Python 3.11, and JDK to 17.

Change-Id: I7c9415e6706141db6cd9cab056a439da81469def
2023-09-08 08:28:22 -07:00
Zuul
af6c43bce3 Merge "Update jinja-init image to bookworm" 2023-08-30 16:13:58 +00:00
Zuul
52b865a9eb Merge "Update zookeeper-statsd image to bookworm" 2023-08-25 21:21:16 +00:00
James E. Blair
55abbf33bf Update hound image to bookworm
Change-Id: I51b8630b27149aa96c63d13d6fae240c5c81cd96
2023-08-24 11:31:46 -07:00
James E. Blair
d39162a79a Update zookeeper-statsd image to bookworm
And upgrade to Python 3.11.

Change-Id: I863aabd21e0518879604b187509aa9b46b15bace
2023-08-24 11:19:23 -07:00
James E. Blair
1354a1e6da Update jinja-init image to bookworm
And upgrade to python 3.11.

Change-Id: I47a99263b93be027ffe251fce24d677d07f519d7
2023-08-24 11:18:11 -07:00
Zuul
da29540162 Merge "Update docker-images/python* to remove '.*'" 2023-07-26 22:04:04 +00:00
Zuul
727724af17 Merge "Add $self to the list of files for python images." 2023-07-26 22:04:02 +00:00
Zuul
10244654c1 Merge "Update ircbot and matrix-eavesdrop to python-3.11" 2023-07-26 22:00:01 +00:00
Zuul
3d14237dac Merge "Update Gerrit images to current versions" 2023-07-25 21:32:58 +00:00
Tony Breeds
e2ec0d18fb Update docker-images/python* to remove '.*'
In "Add a uwsgi-base container image"
(08a807/Iaf6d31a3d119f6b7e87b54cda1969a9994110dad)  The uWSGI containers
added an explicit, but redundant, trailing '.*' for the files settings
of the uWSGI containers.  This change is largely cosmetic but for
style consistency let's remove them.

Change-Id: I04162384fbf6a09cffc1aec76777dd6e21752a0e
2023-07-17 17:40:37 +10:00
Tony Breeds
5ed6598c6b Add $self to the list of files for python images.
With the merging of [1] we observed that the images built
(python-3.{9,10,11}-bookworm) were not then promoted.  This was fixed
in [2] as the assumption is that the promotion jobs didn't trigger
because the files modified in [1] don't match the needed files.  This
change adds the zuul configuration file to the list of relevant files.

We're not 100% sure that this will work but it's worth a try.

[1] https://review.opendev.org/c/opendev/system-config/+/888102
[2] https://review.opendev.org/c/opendev/system-config/+/888633

Change-Id: I958a0af6fbad80f8152438aaee6b7268432f64d5
2023-07-17 17:40:37 +10:00
Tony Breeds
3b2016039c Update ircbot and matrix-eavesdrop to python-3.11
Change-Id: I813f62d06212b09638baa7ef0be0de1ad9ff7d83
2023-07-17 17:40:37 +10:00
Clark Boylan
c02fe3019d Update Gerrit images to current versions
Gerrit 3.8 has been released for a bit. Update our 3.8 image builds to
pull the latest 3.8.1 version for testing.

Gerrit 3.7.4 has also been released. We update our 3.7 images to 3.7.4.
Note this is our production image and this update will affect prod
unlike the 3.8.0 update. Changelog is here:

  https://www.gerritcodereview.com/3.7.html#374

The only breaking change is related to prometheus metrics which we
aren't using (yet).

Change-Id: I52854ed097e295c070e642808e9dae793dc8a716
2023-07-16 16:20:27 -07:00
Tony Breeds
31c3aa52b9 Update accessbot,ircboot and matrix-eavesdrop to bookworm container
Change-Id: Idb096e21f57df3db1acc3d3afe3881e2e540f965
2023-07-12 14:47:14 +10:00
Tony Breeds
b3fb6641ad Add Debian bookworm based python images
uWSGI: libffi7 has been retired with bullseye so update to libffi8

Change-Id: I3e7e05b0e0148d73e7f0087f456a3f8de10e0c87
2023-07-12 12:18:19 +10:00
Tony Breeds
69095ccaa1 Update python container provides
The Current python*bullseye container provides lack the distribution
name.  This prevents zuul from correctly identifying and using new
images in speculative builds.

Fix that.

Change-Id: Iad2d84f217f4b807bf967ef28c32b2c1d05f16c5
2023-07-12 12:17:08 +10:00
Zuul
a72693015c Merge "Revert "Move system-config irc bots into quay.io"" 2023-05-25 17:27:34 +00:00
Clark Boylan
8ccab5e967 Revert "Move system-config irc bots into quay.io"
This reverts commit 5842ce3ea61db14ce450b8ec9a19eb744f58fb2f.

OpenDev is moving back to docker.io in order to keep functional
speculative container image testing.

Change-Id: I748772eb92ddb608df8e10b84ce814ae17af9e18
2023-05-24 13:17:54 -07:00
Clark Boylan
d24f648db8 Revert "Migrate statsd sidecar container images to quay.io"
This reverts commit 4a762a6a8a9bc6c4364e8a4b3588326e4fc40982.
A partial revert of 13cfceaea43f1ddc1dc4431da843e9d92e73e426 is also
performed to swap out zookeeper-statsd without affecting other images.

We perform this revert because Docker cannot do speculative gating of
images hosted anywhere but on docker.io. Speculative testing of
container images is an important feature that we wish to keep so revert
until we can stop relying on Docker.

Change-Id: I7ceafdb7cf1dfd4812ea8f12f273f01045ca89a2
2023-05-24 11:39:46 -07:00
Clark Boylan
b9ab9ba76b Revert "Move assets image to quay.io"
This reverts commit 7f88fc7d4aa332ba51ed45f2689d15819ed19692.

We found a deficiency in Docker when working with images hosted outside
of docker.io which necessitates this revert.

Change-Id: I8b16fe3e5297cab04685e0ceb6b8277a2754b3fb
2023-05-24 10:22:27 -07:00
Clark Boylan
7e984fdb03 Revert "Move python builder/base images to quay.io"
This reverts commit b3a861c0e00b409496ded5c774ac3cb2382f319b.

We found deficiencies in Docker when operating against images hosted
outside of docker.io. We are reverting as a result.

Change-Id: Ic690fc7ff9eb5b5ed9c1162d1f4b97edf6f2cd6c
2023-05-24 10:17:49 -07:00
Clark Boylan
f15d8c1486 Rebuild Gerrit imges for Gerrit 3.7.3.
This release has been made but release notes are not yet published for
it. We can see the draft release notes here: [0].

[0] https://gerrit-review.googlesource.com/c/homepage/+/373554/5/pages/site/releases/3.7.md

Change-Id: I6294bbcbc066c2d170eb7ab154585b1c1246016a
2023-05-12 08:23:45 -07:00
Zuul
0c146b59f4 Merge "Move system-config irc bots into quay.io" 2023-05-08 21:49:12 +00:00
Clark Boylan
5842ce3ea6 Move system-config irc bots into quay.io
This moves accessbot, ircbot, and matrix-eavesdrop to quay.io. We also
update the base image location to source from quay.io as these images
have moved.

Change-Id: I03fdfd382e42542f1da25188deabe017d0cbd772
2023-05-05 19:16:06 -07:00
Clark Boylan
4a762a6a8a Migrate statsd sidecar container images to quay.io
The migration of the zookeeper-statsd image itself already happened as
it was our test case. That said I found a small typo that needs fixing
(and will clean up the extra image that was created later). We also
update zookeeper-statsd to pull the base python image from quay.io

The real fun happens with haproxy-statsd. We update the image to pull
the python base image from quay.io and set it up to publish to quay.io
itself.

Change-Id: Ie0c119ac309837baef850236f7d5ba6b9da97ce0
2023-05-05 19:13:24 -07:00
Clark Boylan
7f88fc7d4a Move assets image to quay.io
We are moving our images to quay.io. The assets image is used by other
image builds so we want to treat this like a base image and update it
early in the process. This will reduce overall churn in the process.

Change-Id: Id9630243f60b280647c493abaccf0ece3d56d40c
2023-05-05 19:13:24 -07:00
Clark Boylan
b3a861c0e0 Move python builder/base images to quay.io
We are moving our images to quay.io. This makes the move for our base
python images. Note once we move these images we should attempt to
quickly update any of the images that depend on these images. This will
minimize the time period where we might be pulling outdated base images.

Change-Id: I03c8053345abf6c94a8e904917f53caa5693ce89
2023-05-05 19:13:17 -07:00
Zuul
0f3a60ca64 Merge "Base jobs for quay.io image publishing" 2023-05-01 19:04:35 +00:00
Clark Boylan
13cfceaea4 Base jobs for quay.io image publishing
This adds new jobs for publishing our images to quay.io. We also add a
secret with the necessary items (password for images pushes, API token
for repo creation). Then we update the zookeeper-statsd image as the
first image to get moved to quay.io. This should exercise things and
ensure it is working for us.

Note we don't try to update all of the images at once as we have many
images managed in this one repository and debugging every one of them
after a single merge just seems like pain. Instead, we'll start with one
image and go from there.

Depends-On: https://review.opendev.org/c/opendev/base-jobs/+/881522
Change-Id: I5198942f184df72543185a0ea88d50f9dc2d9421
2023-05-01 10:05:17 -07:00
Zuul
fd55aa261d Merge "Update accessbot to Use Python 3.11 Base Images" 2023-04-30 23:24:52 +00:00
Gene Kuo
792d3f611f Build haproxy-statsd with Python 3.11 Base Images
This commits update haproxy-statsd Dockerfile to use Python 3.11 images
instead of older 3.9 one. It's part of the effort to update Python
version to 3.10 or later.

Change-Id: I65d469d18d1cf10d7f404b12afdeb5e6d8580bd6
2023-04-27 12:13:54 +09:00
Zuul
f957ebdbb5 Merge "Update Hound to Use Python 3.11 Base Images" 2023-04-27 02:55:13 +00:00
Clark Boylan
40390f996e Add Gerrit 3.8 image builds and test jobs
Gerrit 3.8 isn't officially released yet, but they have relaesed their
third release candidate. Seems like a good time to start building a 3.8
image and testing it.

Change-Id: I11bcd35379cecde135f471a840b03937166b6663
2023-04-26 13:33:14 -07:00
Clark Boylan
cee951e6a9 Remove Gerrit 3.6 image builds and test jobs
We've been running Gerrit 3.7 for some time now and seem very unlikely
to revert at this point. Clean up the Gerrit 3.6 image builds as we
don't need them anymore.

THis change also comments out the 3.6 -> 3.8 upgrade job. Followup
changes will add 3.8 image builds and test the 3.7 -> 3.8 upgrade
process.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/881595
Change-Id: I759b34e48dcede7ffaa66c83da01b81c4fed4b4f
2023-04-26 10:52:54 -07:00
Gene Kuo
9f86500913 Update Hound to Use Python 3.11 Base Images
This commits update hound Dockerfile to use Python 3.11 images instead
of older 3.9 one. It's part of the effort to update Python version to
3.10 or later.

Change-Id: Idf5076709c55d3327c69c6e8160a8348b6640ca7
2023-04-26 19:53:37 +09:00
Gene Kuo
faf0c90fe6 Update accessbot to Use Python 3.11 Base Images
This commits update accessbot Dockerfile to use Python 3.11 images
instead of older 3.9 one. It's part of the effort to update Python
version to 3.10 or later.

Change-Id: I983ca73237b8a11ae3885505f549d8b0b16c144f
2023-04-25 12:29:38 +09:00
Clark Boylan
ca4fff2dce
Update Gerrit 3.7 to 3.7.2
This is mostly a bookkeeping update, but it reflects that upstream made
a release to fix the related changes issue.

  https://www.gerritcodereview.com/3.7.html#372

Change-Id: Iecbda339d0af4cc625df7be3925010ce643808bd
2023-03-28 07:48:01 +11:00
Ian Wienand
31d71e6379
gerrit images : trigger rebuild
Trigger a rebuild from the stable-3.7 branch head to pickup

 https://gerrit-review.googlesource.com/c/gerrit/+/364214

which fixes the redirection of the "submitted together" links when you
have multiple changes stacked; upstream bug report

 https://bugs.chromium.org/p/gerrit/issues/detail?id=16765

Change

 https://gerrit-review.googlesource.com/c/gerrit/+/362878

has merged into stable-3.6 and stable-3.7 branches, and requires a
corresponding update to the download-commands plugin

 https://gerrit-review.googlesource.com/c/plugins/download-commands/+/362879

Since there are no stable releases yet, the plugin hasn't made new
tags.  We switch this to master until we have stable tags.

Change-Id: I93715081a802894420003caf9d01c1d349d80650
2023-03-21 13:55:39 +11:00
Clark Boylan
ca630db7b2 Update gerrit image builds for 3.6.4 and 3.7.1 tags
We actually just build 3.6.4 and 3.7.1 images using 3.6.3 and 3.7.0
versions of plugins. We also deployed the resulting 3.6.4 image to prod.
Luckily for us 3.6.3 plugin tags match 3.6.4 and 3.7.0 tags match 3.7.1.

There was one exception which we were alredy checking out master for in
order to fix build errors. Now that there is a new reviewnotes tag for
3.6.4 we can drop the master checkout for that plugin and use the tag.

TL;DR this should be a noop for our images but is nice bookkeeping to
avoid surprises.

Change-Id: Ica8b3605ecb0f3f93747b904c39913800d1eee4e
2023-03-02 14:04:52 -08:00
Clark Boylan
c95726ed1e
Build Gerrit on top of our python-base images
This is done for a number of reasons. First it will allow us to update
the python version used in the images as we can have a 3.10 builder and
base images (but not a 3.10 openjdk:11 image). Second it will allow us
to easily switch to openjdk 17 by simply updating the package we install
and some paths for the jdk location.

The goal here is to have more control over the images so that we can do
things like change python and java versions when we want to.

Depends-On: https://review.opendev.org/c/opendev/jeepyb/+/870873
Change-Id: I7ea2658caf71336d582c01be17a91759e9ac2043
2023-02-01 11:10:42 +11:00
Clark Boylan
c5c3e6e575 Fix Gerrit 3.6 image build
This checks out master of the reviewnotes plugin which includes fixes
for flogger compile time log string checks brought in by a recent Bazel
update. Unfortauntely no 3.6.x tag includes these fixes yet. Judging by
the change log this should be safe though.

Change-Id: I71d4e7732a49236af9f8f558743c19f3dad9818e
2023-01-19 13:28:38 -08:00
Clark Boylan
34a20bb39c Add Gerrit 3.7 images
This runs system-config-run-review-3.7 but does not add the upgrade test
yet.

Change-Id: I120081b3e30f639a4242fcca7b93def162c240db
2023-01-13 08:40:51 -08:00