Commit Graph

102 Commits

Author SHA1 Message Date
Monty Taylor
763aa927a7 Add python3-distutils to gerrit-base
We need this to exist so that we can install pip.

Change-Id: I83181c1e9b5282ef0cba6d1b9193a71967564a43
2020-03-03 11:41:26 -06:00
Zuul
7fbcb1c561 Merge "Upgrade gitea to 1.11.1" 2020-02-25 23:17:02 +00:00
Monty Taylor
385e89d21e Upgrade gitea to 1.11.1
Gitea build needs npm now, so add the installation.

Change-Id: I250dbedf5fb0b30f5f80d23ff67f0b445ce3eb9d
2020-02-25 15:10:51 -06:00
Zuul
7fe8a64cdc Merge "Build gerrit images with bazelisk" 2020-02-24 18:20:48 +00:00
Monty Taylor
a8e1d1496d Build gerrit images with bazelisk
We need to use bazelisk to build gerrit so that we can properly
track bazel versions in the job. Use the roles developed for
gerrit-review to do that, then simplify the dockerfile to have
it simply copy the war into the target image.

Also add polymer-bridges.

Depends-On: https://review.opendev.org/709256
Change-Id: I7c13df51d3b8c117bcc9aab9caad59687471d622
2020-02-21 17:32:01 -06:00
Zuul
2402b208b2 Merge "Upgrade gitea to 1.10.3" 2020-02-20 17:06:18 +00:00
Monty Taylor
37a47fb259 Add curl to python-builder image
It's pretty common to use curl to grab things.

Change-Id: Ifdf69b066cc6642a06ce77a36065e867be0fe12d
2020-02-13 09:17:59 -06:00
Monty Taylor
acd0307fc5 Upgrade gitea to 1.10.3
The latest from the 1.10 series is 1.10.3.

Change-Id: I294b5f31ef6fc30b36c79e521554f03af33d4be9
2020-02-04 13:13:51 -06:00
Tobias Henkel
e35fcde591
Upgrade pip in python-builder and base
There was an issue in pip that prevented correctly caching locally
built wheels [1]. This has been fixed in recent pip versions so
upgrade pip in both images so image caching works correctly. This is
needed to unbreak nodepool images that fail to install the locally
built netifaces package.

[1] https://github.com/pypa/pip/issues/6852

Change-Id: Ibbe12bcc53253a80d0bafa3d09a20c49a3a2b784
2020-01-22 15:47:58 +01:00
Zuul
3927db22c4 Merge "Add job dependencies to haproxy-statsd" 2020-01-17 00:11:11 +00:00
Monty Taylor
99c3fbff78 Add job dependencies to haproxy-statsd
haproxy-statsd uses opendevorg/python-base already. Add that to its
job dependencies and make sure it triggers on updates to python-base.

Update the FROM line to be fully qualified.

Change-Id: I9c8e8094f5570bf44076915610cd1be6d95ed326
2020-01-17 06:50:19 +08:00
Zuul
69f9f64ebc Merge "Bump gitea version to 1.10.2" 2020-01-07 20:59:52 +00:00
Zuul
564a9d2a2a Merge "Base gitea-init on opendevorg/python-base" 2020-01-06 15:41:11 +00:00
Monty Taylor
6689ee968e Bump gitea version to 1.10.2
Change-Id: If29f2a4bdb95819433cde4cf9f4b5315222de4b0
2020-01-06 09:05:13 -06:00
Monty Taylor
5de73f6c36 Use explicit image paths
To make it clear that docker hub is but one of many possible registries,
update our usage of FROM and image: lines to include docker.io in the
path.

There are a few other FROM lines for the gitea images which are handled
in a separate stack.

Change-Id: I6fafd5f659ad19de6951574afc9a6b6a4cf184df
2019-12-17 08:13:34 -05:00
Monty Taylor
c6c8ed75be Update bazel to version 1.2.0
Upstream has updated.

Change-Id: I07af5f101270b6ec735b6c7e29599d5dae0e2a86
2019-12-17 08:13:33 -05:00
Zuul
5497d9365c Merge "Add some details on where to find things to homepage" 2019-12-05 09:54:53 +00:00
Zuul
99a0fed381 Merge "Split bindep and wheel invocations" 2019-12-04 17:41:22 +00:00
Zuul
3e7b9e9736 Merge "python-builder: install sibling packages" 2019-12-04 17:34:01 +00:00
Zuul
93067cb679 Merge "Update gitea to v1.10.0" 2019-12-04 16:39:46 +00:00
Monty Taylor
b3e5f0cab0 Update gitea to v1.10.0
1.10 introduces a PASSWORD_COMPLEXITY setting with a default value
of lower,upper,digit,spec - which requires passwords to have an
upper, lower, digit and special character. Our example password does
not have this, so set the PASSWORD_COMPLEXITY setting. We could
alternately leave it at the default and ensure that our passwords
meet the spec.

The sshd_config file is templated now, so we can set the listen port
via env var.

Change-Id: I6e4b595eabb9c6885d78fff1109ea9f602e89ef7
2019-12-04 07:54:49 -06:00
Monty Taylor
5e12438e0d Split bindep and wheel invocations
We need to run bindep before installing git, because otherwise if
a project needs git in its bindep, it won't show up because it'll
be on the build host.

Split the function in two and call them before and after the git
installation.

Change-Id: I316b1bc643eb9293500b31e676361eec7060701d
2019-12-03 11:13:56 -06:00
Ian Wienand
bd66a7cb1b python-builder: install sibling packages
In the dependent change, the docker roles will add sibling packages to
the .zuul-siblings directory of the checked-out source.

Refactor the "assemble" script to handle this.  Essentially we build
the wheel for "." and then iterate over ZUUL_SIBLINGS subdirectories
(set in a --build-arg by the role in dependent change) to also build
the sibling packages.  Note we concatenate the bindep.txt files, so
that we end up with the complete package list required by the main
code and its dependencies.

"install-from-bindep" now installs all the wheels, using --force to
make sure we re-install the speculatively built packages.

This means that a single Dockerfile works under Zuul when
ZUUL_SIBLINGS is set, pointing to Zuul's checkouts; but it also works
stand-alone -- in this case ZUUL_SIBLINGS is empty and we just install
from upstream as usual.

Depends-On: https://review.opendev.org/696987
Change-Id: I4943ae723b06b0ad808e7c7f20788109e21aa8bf
2019-12-03 20:51:32 +11:00
Zuul
02ef233529 Merge "gitea: Use 1.9.6" 2019-11-21 19:43:50 +00:00
Ian Wienand
9f6df6deaf gitea: Use 1.9.6
We are seeing issues with hanging git connections discussed in [1].
It is suggested to upgrade to gitea 1.9.6; do that.

[1] https://github.com/go-gitea/gitea/issues/9006

Change-Id: Ibbbe73b5487d3d01a8d7ba23ecca16c2264973ca
2019-11-19 13:48:42 +11:00
Monty Taylor
fe8daeca5b Update bazel to 1.1.0
Upstream gerrit now needs at least 1.1.0

Change-Id: I783eb712e4153df4ccba30cbf92f44221141f559
2019-11-16 10:49:52 -08:00
Zara
1d07e57ab8 Add some details on where to find things to homepage
The homepage mentions a lot of technologies that OpenDev use, but
doesn't link to any of the running instances. This commit
adds links to review.opendev.org, etherpad.openstack.org and
the configuration for the opendev homepage itself, so that
it's easier to find things and to experience the technologies
it refers to.

Change-Id: Ia041ebbc558539955238bb4fdb4da868bf6f1dd8
2019-11-15 10:16:09 +00:00
Thierry Carrez
16c06894ef Fix recently-introduced "Proposed changes" link
I'm bad at Gitea templates, so the recently-introduced "proposed
changes" tab is active-selected (while it should never be) and the link
is missing the repository name.

This should fix it...

Change-Id: I02adc8ebd012adc233a37223480d14517c7f3c98
2019-11-08 04:18:02 +01:00
Thierry Carrez
67408f6ead [opendev][gitea] Add a link to open changes
Gitea is quickly becoming the public face of Opendev, however it can
be difficult for visitors to understand how to propose changes (or
access already-proposed changes), and then assume everything on opendev
is read-only (which is the exact opposite of what we want to convey).

In the spirit of further integrating Opendev tooling, add a link to
on every repository to open proposed changes on Gerrit.

NB: the link is not I18n-ilized since there is no simple way to add a
new string there, and I did not want to use teh "Pull requests"
terminology.

Change-Id: I851a1e7d25556194947198a8f5534542d167c7f8
2019-11-07 01:51:15 +00:00
Zuul
47f2721386 Merge "Update gitea to 1.9.5" 2019-11-06 00:32:57 +00:00
Monty Taylor
a4643fdef6 Update gitea to 1.9.5
This reverts commit c53c87621f.

Change-Id: I616ad7c631b72ebd704cce1ab4a6d5f1ca188285
2019-11-02 09:15:05 +09:00
Monty Taylor
3b48ad0def Fix gerrit execution path
Java is in /usr/local in these base images. Also, combining
ENTRYPOINT and CMD with [] syntax seems to lead to the whole
thing with [ getting passed to the entrypoint - which leads to
errors like:

  /bin/sh: 1: [/usr/bin/java,: not found

Change-Id: I7c1ebdff58d6590724eaf5d429437a5c8c25fe22
2019-10-26 09:24:04 +09:00
Monty Taylor
c53c87621f Revert "Update gitea to 1.9.4"
Also Revert "Update gitea build to golang 1.13 on buster-slim"

1.9.4 has a bug where doing a get as an unauthenticated user
results in a 500:

http://paste.openstack.org/show/785534/

A pull request has been submitted upstream:

https://github.com/go-gitea/gitea/pull/8653

This reverts commit 1993d985d0.
This reverts commit cedb272a9b.

Change-Id: I75c117d0dc851f7b3c389a19ad0c8e233886b250
2019-10-24 09:17:32 +09:00
Monty Taylor
ccaf54c866 Base gitea-init on opendevorg/python-base
So that we can keep one python base image for our python things,
base jinja-init and gitea-init on python-base. Also, tie jinja-init
to python-base in the dependency graph and gitea-init to jinja-init.
This way if python-base updates, we'll rebuild our python images.

Update FROM lines to use full paths to images.

Change-Id: I554bf07fa8e458e443729cf4b8f40d7ceeaafa04
2019-10-23 17:14:02 +09:00
Monty Taylor
1993d985d0 Update gitea build to golang 1.13 on buster-slim
buster is the new debian release, use it. And use the
slim image, rather than the full, because we can.

Upstream is now building with 1.13 - follow suit.

Update the FROM lines to use explicit paths to images.

Change-Id: I42f7cea1365b8bb0af56861f38107cbdffd130b0
2019-10-23 17:13:21 +09:00
Monty Taylor
cedb272a9b Update gitea to 1.9.4
This is the latest release from upstream. We're not
currently running with any additional patches, although
we did submit one for the atomic updating of repo counts. That
patch will be in 1.10.

There is only one change to the templates we've customized
since the ref we're running from. That is included in the
repo/header.tmpl.

Change-Id: Id426ce6072e127a08810b9fbb109445d36bef2d9
2019-10-23 17:13:20 +09:00
Monty Taylor
4b4dcf37b1 Pin python images to 3.7
rehash may have an issue with python 3.8 internals.

Change-Id: I1a9c6150d74f6a646446b65a7456c73dc104fd9e
2019-10-22 03:37:23 +09:00
David Shrewsbury
0ae912cc04 No-op change to rebuild python-builder
Apparently, an update to the python-base image caused a new image
to be uploaded to dockerhub. Because python-builder was not updated,
it is running a slightly older base image which we believe is causing
issues with python wheel packaging due to possible different python
versions.

Change-Id: I38948882131e30c7358a970292621b0280b75aac
2019-10-21 11:22:50 -04:00
Monty Taylor
9ab25e89a9 Several updates because the world is a dark place
A few things have changed and we need to fix them in one go.

Use mirror for installing docker for buildset-registry

While, we need to make this more systemic, that's hanging off of the
mirror rework. For now, since we know all of these jobs are debian
based, just set the mirror location.

Replace use of zuul cloner with git clones

You can never be a prophet in your own hometown. This is now broken
because of the git cache rework, so just replace it.

Update libjemalloc library

python:slim is based on buster now, which has libjemalloc2 not
libjemalloc1.

Remove gerrit repo remote for submodules

A recent change to the base jobs to use prepare-workspace-git
broke the gerrit image builds by actually having the origin
remote by /dev/null as intended. This breaks submodules because
for a few of them where we don't have matching stable branches
the submodule relative path behavior is actually exactly what
we want.

Since we don't care about the remote otherwise, remove the
origin remote before doing the submodule update --init so that
the submodule will clone the refs from the zuul prepared repo.

Change-Id: Ieb5b6bc8711fe971ed3445c7c267306ac4616464
2019-10-19 07:51:29 +09:00
Monty Taylor
5a31c91623 Expose volume for tmp dir, add dirs for hooks and static
Our existing config management puts files into hooks and static.
We can bind mount those in, but having the mount-points in the
image is useful.

The tmp dir is important for gerrit to be able to write
plugin expansions and javaamelody data.

Change-Id: Idd917c268ed7bdead412620dfe3ca842736b7463
2019-10-16 14:57:46 +09:00
Monty Taylor
d2e7da959d Use bazel 1.0.0
1.0.0 has released, which is what gerrit wants now. Use it.

Depends-On: https://review.opendev.org/688555
Change-Id: I6cd76b8cfda3656d6105f9fe96b82a388809375f
2019-10-14 16:25:30 -04:00
Monty Taylor
d9fb5b3faf Install jeepyb into the gerrit images
We need jeepyb installed because the content of the gerrit hook scripts
we install is done via jeepyb commands. Use python-builder so that we
can just install the jeepyb wheel.

Should we maybe transition these hooks into being zuul jobs?

Depends-On: https://review.opendev.org/683146/
Change-Id: I8899885b05d1e9f48b3f354ca22b360b54d455a3
2019-09-19 15:17:14 +02:00
Monty Taylor
940e31f043 Fix gerrit java invocation
We need the right path to java, and we also need to tell it to run
daemon.

Change-Id: I7dfbc5a1c054953b34a02327e6bba50c486110d1
2019-09-19 14:16:30 +02:00
Monty Taylor
072fcca06f Fix files matcher and bazel for gerrit base image
Use latest bazel

It seems 0.27 is now too old. This is what happens when I go on vacation
apparently.

Add in a hack to override the bazelversion. We'll remove this once
https://gerrit-review.googlesource.com/c/gerrit/+/237495 lands and
has been merged up.

Change-Id: Ib7a6d33ce8bf8498fd5cd09b25087dc09acb8df4
2019-09-16 21:20:18 +02:00
Monty Taylor
9bb1c73139 Split out bazel builder and gerrit base image
There is a bunch of duplication which needs to be redone almost never.
Split those into their own images so we can run them once and reuse them.

Change-Id: I923d4bff96dae75eb52a1c271fa52d5ae79933a0
2019-08-26 11:26:23 +02:00
Monty Taylor
3a26ed621a Build gerrit with bazel 0.27.1 instead
0.27 is the first semi-stable release.

Change-Id: I7314c6500841ed13fdc10dc951761f5fde4b28af
2019-08-26 11:26:23 +02:00
Monty Taylor
56ceaf1c40 Remove the extra bazel options
We had some extra bazel options that don't seem to be necessary
anymore now that we are using upstream bazel options appropriately.

Retry the build a couple of times if it goes south, inside of the
build image. This should allow re-use of the cache the second time,
and if there is a temporary error, it should pick up and move
forward.

Change-Id: I5f304acb21fd3a4d40701fc0414ae0c424c838e5
2019-08-26 11:26:19 +02:00
Monty Taylor
80981a52c8 Update bazel flags to be more inline with upstream
During the Gerrit Hackathon, we learned some things about setting
bazel options. Use the ones recommended in upstream docs rather than
these. The outcome should be largely the same.

Change-Id: I32b4c567488f0739fb80f69dc881b9837803575c
2019-08-25 11:34:58 +02:00
Clark Boylan
05e0ffdebc Collect gitea sshd logs
Currently we don't have any logs from our gitea sshd processes because
sshd logs to syslog by default and /dev/log isn't in our containers. You
can ask sshd nicely to log to stderr instead with the -e flag which
docker will pick up and store for us.

Update the sshd command to include -e then use testinfra to check we
collect logs and they are accssible from docker.

Change-Id: Ib7d6d405554c3c30be410bc08c6fee7d4363b096
2019-08-06 13:42:25 -07:00
Monty Taylor
2a46202b9f Build gerrit images for 2.16 and 3.0 as well
Our goal is upgrading to 3.0. To do that we need to upgrade to 2.15, then
to 2.16, then to 3.0. Build all of the images so that we can do that.

2.16 and 3.0 also use bazel, so just use one copy of the Dockerfile for
all three and let zuul check out the repos to the right versions.

Depends-On: https://review.opendev.org/673147
Depends-On: https://review.opendev.org/672320
Change-Id: I35bd278e0c70c871fa44d005c60a987d1d8e3cdc
2019-07-27 11:34:42 -04:00