534 Commits

Author SHA1 Message Date
Monty Taylor
bde5860f57 Parameterize manage-projects logging output
Change-Id: If4c456957fdce308ccafee0dfad8b627a85ea796
2020-03-30 13:53:39 -05:00
Zuul
a00499bf2f Merge "Really bindmount acls" 2020-03-29 15:38:56 +00:00
Monty Taylor
a2cd7d22c7 Really bindmount acls
There is a gerrit in the path.

Change-Id: I6b1a292e35a2de95de63bb90813cb6154573d8ce
2020-03-29 09:10:26 -05:00
Zuul
f986742ab9 Merge "Bind-mount acls dir to /home/gerrit2" 2020-03-28 21:36:59 +00:00
Zuul
7375566fd0 Merge "Use gerrit_vhost_name in projects.ini" 2020-03-28 20:30:44 +00:00
Monty Taylor
1a4e781173 Bind-mount acls dir to /home/gerrit2
Turns out our config has a bunch of hardcoded /home/gerrit2/acls
entries in it. That doesn't work if we're just pointing the
config file at /opt/project-config/gerrit/acls.

Change-Id: I387e446501e17a3bdd807807d5ef6b69b53abde5
2020-03-28 12:58:58 -05:00
Monty Taylor
1be28fddf3 Use gerrit_vhost_name in projects.ini
We use this to make the .gitreview file too, so our thought that
we could just use localhost was a little misguided.

Change-Id: I501b10b2003c7e04ca1ac345d14fa33916b3e60b
2020-03-28 12:06:24 -05:00
Monty Taylor
cc1929d075 Add an image pull to manage-projects playbook
When we run manage-projects, we always want the latest image.

Add a task file to the gerrit role so that it can have access
to role default variables. Then call it from the playbook
with tasks_from.

Change-Id: I08bd10539d7f08e8759f1c22d66dea18821c9d42
2020-03-27 15:58:41 -05:00
Zuul
50391bcfad Merge "Run manage-projects on gerrit servers" 2020-03-27 18:34:14 +00:00
Zuul
66097bcfea Merge "Use our jitsi-meet image for meetpad" 2020-03-27 17:47:00 +00:00
James E. Blair
6ee1cfa736 Use our jitsi-meet image for meetpad
Change-Id: I0be6adf3852c7ab475ab6b4cfc8e53e207c382d1
2020-03-27 09:43:41 -07:00
Zuul
ce3a064133 Merge "Add meetpad server" 2020-03-27 14:44:30 +00:00
Monty Taylor
5d79086350 Run manage-projects on gerrit servers
We have a wrapper script now that works on the gerrit servers.
Run it after we run the gitea repo creation.

Remove the -it since this is not an interactive run. Add net=host
so that localhost maps. And set the server to localhost.

Don't run manage-projects in gate jobs for run-review yet, because
we don't have a functioning enough gerrit. We can follow up with
that.

Change-Id: I8c0eb5d7973815ae21d4d5bc1be8d4bdfe56d7fc
2020-03-25 17:50:52 -05:00
James E. Blair
8b093dacd5 Add meetpad server
Depends-On: https://review.opendev.org/714189
Change-Id: I5863aaa805a18f9085ee01c3205b0f9ad602922d
2020-03-25 07:44:24 -07:00
Monty Taylor
2d208f0df9 Actually write out the projects.ini config file
We have a template for it in tree, but we're not, you know,
templating it.

Change-Id: Ic6b3ecc62cdfc7c486a912ba4863c7c4c7734e0a
2020-03-24 14:39:12 -05:00
Zuul
c8b6312430 Merge "Install utility scripts for running jeepyb commands" 2020-03-23 14:34:51 +00:00
Zuul
cbdecbd696 Merge "Use dev subdir on review-dev for project-config things" 2020-03-23 14:34:49 +00:00
Monty Taylor
e6d98f0181 Install utility scripts for running jeepyb commands
jeepyb is installed in the gerrit image because of hook scripts.
To run manage-projects and track-upstream, make wrapper scripts
in /usr/local that runs the commands from in the container image
bind-mounting the appropriate dirs and files into the container.

Change-Id: I7ef1f00e69d4c310d69d83c80ca210e8f340878d
2020-03-22 10:39:47 -05:00
Zuul
3af9548401 Merge "Update CLA link" 2020-03-21 18:24:50 +00:00
Zuul
06e0155cb8 Merge "Update infra-manual location" 2020-03-21 18:02:17 +00:00
Andreas Jaeger
4293e228b7 Update CLA link
link to contributor guide, the OpenDev manual does not talk about CLA
anymore.

Change-Id: Ia414513fc1645e0eec462f95af0d9247eb4c0a99
2020-03-21 18:17:57 +01:00
Monty Taylor
f934851ad8 Set ulimits for gerrit
We set core size to 0 and number of files to twice the setting of
core.packedGitOpenFiles. Make gerrit_packed_git_open_files a role
variable so we can jinja the 2x transformation.

Change-Id: I05dbfe1f091e7014c47b0f1ba79f99b51d1a6c30
2020-03-21 11:41:00 -05:00
Monty Taylor
906e6a72a1 Use dev subdir on review-dev for project-config things
This is in anticipation of running manage-projects, which wants to
use the dev subdir of project-config when run on dev, but we're
currently not doing anything with that. Point at root or dev as
appropriate. Then we can do a similar thing - probably just
bind-mounting the root or the dev when we docker run for
manage-projects.

Change-Id: Ia13bbb0bf8dbe1f7e7c0f378ba9b41bef3ecd5c1
2020-03-21 11:34:28 -05:00
Zuul
6ceb12fe99 Merge "Base 2.13 image on gerrit-base" 2020-03-21 16:21:05 +00:00
Zuul
a31bae50a3 Merge "Add a new docs.airshipit.org vhost on static01" 2020-03-20 22:07:40 +00:00
Monty Taylor
63d8f7af48 Base 2.13 image on gerrit-base
We install jeepyb and launchpadlib in gerrit-base. Those are
important. We also need to add cgi for gitweb.

The gerrit init command does two things that we don't actually
want it to do at runtime. It extracts the plugins into the
plugins dir, and it downloads the right database library.

We can extract the plugins for it during image creation, and
then we can also download the plugin it would have downloaded.

We can also download the mysql library for it:

https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-2.13/gerrit-pgm/src/main/resources/com/google/gerrit/pgm/init/libraries.config

Finally, we tell it to not download or expand anything during
init, because we're running in a container and next time we run
the process that dir isn't going to be there.

Our gerrit integration tests don't depend on our gerrit image builds.
Put in image depends between run-review and gerrit builds.

We also need to depend directly on opendev-buildset-registry.

Add java.security.egd setting to java invocation

This tells java to be secure.

https://stackoverflow.com/questions/58991966/what-java-security-egd-option-is-for

Add support for setting heap limit properly

The gerrit init script does this based on the value in
container.javaOptions. We could, but then we'd have to
run an entrypoint script. Instead, set the value via
the JAVA_OPTIONS env var setting based on a value from
ansible.

Finally, make gerrit-master image build non-voting

It looks like there might be a real issue, but debugging that
is not important for us at this moment.

Depends-On: https://review.opendev.org/714216
Change-Id: I01e94c10f470fb3c8ddfce7b0e201357e5050679
2020-03-20 16:37:18 -05:00
Andreas Jaeger
2c0b82e5e8 Update infra-manual location
The infra-manual now lives on docs.opendev.org, update links.

New location is: https://docs.opendev.org/opendev/infra-manual/latest

Change-Id: I7716c68cbff4f3a640d7161f59cfc034a7ccca52
2020-03-20 22:03:09 +01:00
Jeremy Stanley
abcae98b8e Add a new docs.airshipit.org vhost on static01
The Airship project is continuously publishing documentation to AFS,
so serve that volume with a corresponding vhost on the static01
server. Also add it to the list of volumes for periodic vos release.

Change-Id: I718963533d9e8596d44d451b5e930314d699fa28
Depends-On: https://review.opendev.org/706599
2020-03-20 19:09:13 +00:00
Monty Taylor
9b999423b7 Add an use a utility service for gerrit commands
We have a need to run ad-hoc commands with the various things
mounted, such as the gerrit init, as well as our friends like
manage-projects. Make a second "service" that doesn't do anything,
but allows us to run "docker-compose run".

Change-Id: Ic9bafe85b90af413c891b969273437be9df6a79f
2020-03-20 09:51:29 -05:00
Zuul
0a98e2cf5f Merge "Add restTokenPrivateKey" 2020-03-20 12:18:53 +00:00
Zuul
f0f022a0c2 Merge "Copy the contents of static and etc" 2020-03-20 10:39:06 +00:00
Monty Taylor
6cc2d0c176 Copy the contents of static and etc
We don't want to copy static and etc into the static and etc dirs
creating static/static and etc/etc.

It's possible this could also be done with trailing /'s?

Change-Id: I054158a677261743ed67107894e207f7b615ea7f
2020-03-20 03:25:02 +00:00
Monty Taylor
dd4323b294 Add idleTimeout to sshd config section
Puppet was adding this. Let's add it.

Change-Id: I5d34ff79f6684caef9922a6653c05708ca21cb55
2020-03-20 03:24:01 +00:00
Monty Taylor
e51e289425 Add restTokenPrivateKey
We don't have this on review-dev, so it was missed.

Change-Id: I459266ac6766c204891152c161e80f3cdfc9a295
2020-03-20 14:22:04 +11:00
Monty Taylor
d3c8c1077b Switch to running gerrit via ansible+containers
This should be mostly a no-op - but we will need to do a shutdown
in emergency mode.

Tell the gerrit role to not run compose up when run as part of
remote_puppet_git.

Change-Id: Id45376c2697656a12afeacf317b6f26c85c08dad
2020-03-19 17:21:39 -05:00
Monty Taylor
2e5b95a7f3 Add flag to allow skipping docker-compose up
For our rollout, we need to be able to run this without actually
running the up.

Also, split out startup tasks so that we can run them from a
dedicated start playbook by themselves.

Change-Id: I08d994e496fbd8d5adbfa1ce344b0ae52f46535c
2020-03-19 15:51:33 -05:00
Zuul
316efe74b1 Merge "Remove Fedora 29 mirroring" 2020-03-19 02:00:57 +00:00
Zuul
b0f81dc7b9 Merge "Update git.starlingx/git.airship redirects" 2020-03-19 01:21:21 +00:00
Zuul
51a5f5488f Merge "Update git.zuul-ci.org redirects" 2020-03-19 01:21:20 +00:00
Zuul
3709d0a39b Merge "Restart gitea more cleaner" 2020-03-18 23:56:10 +00:00
Monty Taylor
77fb33f5d5 Restart gitea more cleaner
Gerrit replication plugin is good about retrying replication if its
connectivity to the remote fails. It however thinks everything is happy
if it can connect and push even when gitea-web isn't running.

Make the whole replication system happier by stopping gitea-ssh before
other services and starting it after other services. This way gerrit
should fail to replicate until gitea is ready for it to ssh in.

Change-Id: I3440d8dd8a01a3aaf5d18c9c2ca48e7ead63856f
2020-03-18 15:40:40 -07:00
Zuul
dc1848d804 Merge "Replace nb01.opendev.org with nb04" 2020-03-18 21:57:28 +00:00
Zuul
69ae0766db Merge "Remove the Rackspace Cloud repo" 2020-03-18 21:14:21 +00:00
Ian Wienand
ba5d9547c8 Replace nb01.opendev.org with nb04
We are starting over with the container nodepool host, and also moving
it to "nb04" to avoid any possibility of conflicting with the
short-hostname of nb01.openstack.org.

Story: #2007407
Task: #39064

Depends-On: https://review.opendev.org/713575
Depends-On: https://review.opendev.org/713571

Change-Id: I18ab9834ad4da201774e0abef56f618cd7839d36
2020-03-19 07:41:56 +11:00
Andreas Jaeger
eecf3e71fc Update git.starlingx/git.airship redirects
After the big OpenDev rename, these repos got renamed again. Update the
redirects for git.airshipit.org and git.starlingx.io to point to the
current location.

Update test_static.py for this, change the test repo since
airship-in-a-bottle was first renamed to in-a-bottle and later to
airship-in-a-bottle.

Change-Id: I71b786cd528aac9ae68464618db02e22cd4c0b5b
2020-03-18 18:39:48 +01:00
Andreas Jaeger
a6480bcefb Update git.zuul-ci.org redirects
zuul and nodepool now life in opendev, avoid double redirects and
redirect directly to final location.

Change-Id: Ia55d76b24f07ec64cb55055955c4549f3706a95b
2020-03-18 18:28:42 +01:00
Monty Taylor
06be60bc08 Drop version specifier for nodepool-builder compose
We don't actually need version 3. Mark it as version 2 to keep it
inline with everything else. In general we should only increase
past v2 if we need a specific feature.

Change-Id: Ie243da369ddec30e0eca4805434d572e12c40491
2020-03-17 13:11:25 -05:00
Monty Taylor
74b822f352 Use version 2 for docker-compose for review
Also - update the test job to use xenial nodes, since that's what
our gerrit servers are. docker-compose in xenial is too old for
version 3 file format - but we didn't get an error in the test
becuase we're running on bionic.

Change-Id: I855c3df241932098c95de84cf9f92383b335702f
2020-03-17 13:02:20 -05:00
Zuul
87db9b6ac6 Merge "nodepool-builder: put container configs in /etc" 2020-03-17 17:50:12 +00:00
Zuul
38220dbecf Merge "Added updates dir for Fedora 31 release" 2020-03-17 08:37:03 +00:00