We used to replicate every openstack/* project to GitHub,
through a global replication at Gerrit-level. Now that the job for
granular replication is in place, we can stop the global replication,
so that only active/official repositories are synced.
Depends-On: https://review.opendev.org/724310
Change-Id: Ibba02e626e33aba9779f771d5ae49920bac86b19
We get deprecation warnings from ansible about use
of python2 on xenial hosts. Rather than setting
ansible_python_interpreter to python3 on a host by
host basis, set it globally to python3.
Set it to python for the one host that's too old,
refstack.openstack.org, which is running on trusty
which only has python3.4.
Change-Id: I4965d950c13efad80d72912911bc7099e9da1659
We need to override this because it's erroneously picking up
python2 which on etherpad does not have everything it needs.
Change-Id: Ic127f505740f9ac66abb8754356e04acde66dd85
Zuul is publishing lovely container images, so we should
go ahead and start using them.
We can't use containers for zuul-executor because of the
docker->bubblewrap->AFS issue, so install from pip there.
Don't start any of the containers by default, which should
let us safely roll this out and then do a rolling restart.
For things (like web or mergers) where it's safe to do so,
a followup change will swap the flag.
Change-Id: I37dcce3a67477ad3b2c36f2fd3657af18bc25c40
By default gerrit replication pushes +refs/*:refs/*, which includes
refs/changes. For large repositories that potentially means hundreds
of thousands of references.
Per-repo git mirroring does not push refs/changes, so when it runs it
ends up deleting those references, which can take a long time, blocking
the executor.
To fix that, we should:
- stop pushing refs/changes to GitHub (this change)
- delete refs/changes on GitHub repositories, asynchronously
- enable per-repo replication
- disable Gerrit-wide replication
NB: it is unclear if Gerrit replication would start deleting the
extraneous references on remote GitHub repositories once this
merges. If this is the case, since replication is limited to a
single thread (default value for 'threads') and is not happening
in an executor, this should not have negative impact, beyond
potentially delaying GitHub mirroring.
Change-Id: I94f69c889c9b4418ef81b3b2ca436ba99696ba72
We use project-config for gerrit, gitea and nodepool config. That's
cool, because can clone that from zuul too and make sure that each
prod run we're doing runs with the contents of the patch in question.
Introduce a flag file that can be touched in /home/zuulcd that will
block zuul from running prod playbooks. By default, if the file is
there, zuul will wait for an hour before giving up.
Rename zuulcd to zuul
To better align prod and test, name the zuul user zuul.
Change-Id: I83c38c9c430218059579f3763e02d6b9f40c7b89
We run some utility scripts which ssh to ourselves, but we aren't
setting host keys for them. We should fix that.
Change-Id: I2aa5d5e65b15c5c151767377dbc5ead1e442b3ce
Upstream likes building the settings file into the image, but that's
less exciting, let's bind-mount ours in.
Depends-On: https://review.opendev.org/717491/
Change-Id: Ia1894d884ef2a84e1282345b77fe07bf8898f367
jeepyb is installed in the gerrit image because of hook scripts.
To run manage-projects and track-upstream, make wrapper scripts
in /usr/local that runs the commands from in the container image
bind-mounting the appropriate dirs and files into the container.
Change-Id: I7ef1f00e69d4c310d69d83c80ca210e8f340878d
This is in anticipation of running manage-projects, which wants to
use the dev subdir of project-config when run on dev, but we're
currently not doing anything with that. Point at root or dev as
appropriate. Then we can do a similar thing - probably just
bind-mounting the root or the dev when we docker run for
manage-projects.
Change-Id: Ia13bbb0bf8dbe1f7e7c0f378ba9b41bef3ecd5c1
We install jeepyb and launchpadlib in gerrit-base. Those are
important. We also need to add cgi for gitweb.
The gerrit init command does two things that we don't actually
want it to do at runtime. It extracts the plugins into the
plugins dir, and it downloads the right database library.
We can extract the plugins for it during image creation, and
then we can also download the plugin it would have downloaded.
We can also download the mysql library for it:
https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-2.13/gerrit-pgm/src/main/resources/com/google/gerrit/pgm/init/libraries.config
Finally, we tell it to not download or expand anything during
init, because we're running in a container and next time we run
the process that dir isn't going to be there.
Our gerrit integration tests don't depend on our gerrit image builds.
Put in image depends between run-review and gerrit builds.
We also need to depend directly on opendev-buildset-registry.
Add java.security.egd setting to java invocation
This tells java to be secure.
https://stackoverflow.com/questions/58991966/what-java-security-egd-option-is-for
Add support for setting heap limit properly
The gerrit init script does this based on the value in
container.javaOptions. We could, but then we'd have to
run an entrypoint script. Instead, set the value via
the JAVA_OPTIONS env var setting based on a value from
ansible.
Finally, make gerrit-master image build non-voting
It looks like there might be a real issue, but debugging that
is not important for us at this moment.
Depends-On: https://review.opendev.org/714216
Change-Id: I01e94c10f470fb3c8ddfce7b0e201357e5050679
We are starting over with the container nodepool host, and also moving
it to "nb04" to avoid any possibility of conflicting with the
short-hostname of nb01.openstack.org.
Story: #2007407
Task: #39064
Depends-On: https://review.opendev.org/713575
Depends-On: https://review.opendev.org/713571
Change-Id: I18ab9834ad4da201774e0abef56f618cd7839d36
Start installing Let's Encrypt SSL certs on static01 for a new
docs.airshipit.org it will be hosting.
Change-Id: Ia5089515cd02d78267827840521cbee4f71ceb0b
This adds the Open Edge (formerly Fortnebula) CI mirror.
Change-Id: I1ccf2a602f8a41e00bc64a9516a326cc07d9b254
Depends-On: https://review.opendev.org/711787
Sister change for Ia5caff34d3fafaffc459e7572a4eef6bd94422ea and
removing earlier references to the mirror server in preparation for
building and adding the new one.
Change-Id: I7d506be85326835d5e77a0c9c461f2d457b1dfd3
This site was never used nor published, it can be killed according to QA
PTL.
codesearch returns no matches for it in any docs.
Keep the occurence in manifests/static.pp, this will get deleted
as part of https://review.opendev.org/710388.
Change-Id: I3c0d3b567a3eccb959dc903f169197e4581f1e13
This is a slight divergence from the accepted spec, where we were
going to implement these redirects via a new haproxy instance
(I961456d44a56f2334d3c94ef27e408f27409cd65). We've decided it's
easier to keep them on static.opendev.org
The following sites are configured to redirect to whatever they are
redirecting to now on static.opendev.org:
* devstack.org
* www.devstack.org
* ci.openstack.org
* cinder.openstack.org
* glance.openstack.org
* horizon.openstack.org
* keystone.openstack.org
* nova.openstack.org
* qa.openstack.org
* summit.openstack.org
* swift.openstack.org
As a bonus, they all get a https instance too, which they didn't have
before.
testinfra coverage should be total for this change. I have created
the _acme-challange CNAME records for all the above.
Story: #2006598
Task: #38881
Change-Id: I3f1fc108e7bb1c9500ad4d1a51df13bb4ae00cb9
This creates the redirect sites
git.airshipit.org
git.openstack.org
git.starlingx.io
git.zuul-ci.org
The htaccess rules are put into the main configuration file to avoid
having to create a directory and manage another file. We use a macro
to duplicate the rules and retain the old semantics of the http site
redirecting directly (as opposed to doing a extra 301 to
https://git.openstack.org first). This required adding "/" to the "^"
matches as it now runs in VirtualHost context; no functional change is
intended over the old sites.
This will require _acme-challenge CNAMEs to acme.opendev.org before
being merged.
testinfra is updated to exercise some redirects matching against the
results of the extant sites.
Change-Id: Iaa9d5dc2af3f5f8abc11c2312e4308b50f5fcd2b
files.openstack.org serves a view of /afs/openstack.org/, which is the
same as static.opendev.org. Add a serveralias for it and certificate.
Make static.openstack.org be consistent with opendev by showing the
same thing.
Change-Id: I4c492e3b02554a7c736c015790bd4cd5bb435a43
While the service is review.opendev.org, the server is actually
review01.openstack.org. The ansible inventory in production knows
it that way, as does the Nova in RAX DFW. Update the host_vars
entry and the zuul jobs so that it matches (And so that LE certs
apply)
Change-Id: I4c762c57f6826f2c5f9ed5c9cb0ae02644570c3d
This creates sites to serve
developer.openstack.org
docs.openstack.org
docs.opendev.org
docs.starlingx.io
which are all just static directories underneath /afs/openstack.org/.
This is currently done by files02.openstack.org, but will be better
served in the future by consolidating in ansible configuration on
static.opendev.org.
The following dns entries need to be made before merging to ensure the
certificates are provisioned
_acme-challenge.developer.openstack.org
_acme-challenge.docs.openstack.org
_acme-challenge.docs.opendev.org
_acme-challenge.docs.starlingx.io
Once done, we can merge and then cut-over the main DNS entries as we
like.
Since there are some follow-ons, I have not removed the puppet
configuration from files02.openstack.org. I think it's best we
migrate everything away from that and remove it in one lot.
Change-Id: I459a36f823a8868e6cc09e2b0d85f2fe05d69002
This adds the site to publish from
/afs/openstack.org/project/releases.openstack.org
Change-Id: Ia91deb9a51441ac9974137ed39fc5a185689a11c
Task: #37724
Story: #2006598
We have LE dns entries for review.o.o, but we're not actually
requesting the cert. Go ahead and request it - it'll make the
apache config easier to sort out.
Get the openstack.org certs for review-dev while we're at it.
Change-Id: I91d06c97993ba37204bd1fc326ae823e1b9c0c1a
Depends-On: https://review.opendev.org/707267
Depends-On: https://review.opendev.org/707255
This adds a mirror to the new airship citycloud region. Add the host to
the inventory and add necessary host vars for LE setup.
Depends-On: https://review.opendev.org/706573
Change-Id: I33cefe914911b4f5ce5e09e0329ba48e039ede64
Add these hosts to static.opendev.org, serving from AFS. Note that
tarballs.openstack.org just redirects to static.opendev.org/openstack.
This should have no effect currently, it will only become live when we
switch DNS.
For more details see the thread at:
http://lists.openstack.org/pipermail/openstack-infra/2020-January/006584.html
Change-Id: Ie56fac17ffaa91ee55be986de636485a58125a02
Add a new review-dev server on the opendev domain with LE support
enabled.
Depends-On: https://review.opendev.org/705661
Change-Id: Ie32124cd617e9986602301f230e83bb138524fdf
Add this host for serving content from AFS.
The
_acme-challenge.governance.openstack.org
_acme-challenge.security.openstack.org
CNAMES should be in place for creating the certificates (added with
Ie1b92f06b71aa6069fe831b26ba1cc272ce4562c).
Also add a cert for the base server (static.opendev.org) since we
added the DNS entries for it.
Change-Id: I55e0ac7487b02f9a816ac486ed01b73f82b391a5
Story: #2006598
Task: #37757
Depends-On: https://review.opendev.org/704469
This adds a new handler to restart the zuul registry to pick up the new
cert. We may want to consider updating zuul registry to accept a reload
of ssl config without restarting the service.
Depends-On: https://review.opendev.org/702050
Change-Id: I23f6bea68285bc7cb0d12224235eaa16f0d07986
This name/host doesn't actually exist so don't try issuing a cert for
it. Instead only issue a cert for zuul.opendev.org.
Change-Id: I6c8eaa9280c3d6f070b8a1c79d850ee42e0e8d50