This job installs all of the vhosts necessary for our static hosting as
well as compiling afs kernel modules. Then it tests that they function.
Unfortunately in some cases half an hour just isn't enough time. Double
the default timeout to an hour to work around this.
Change-Id: Ief457aafa503af6c5bad83a1198b6c699d2d4983
We previously had two manually issued certs (one each for opendev.org
and openstack.org) but now have a single cert with all the appropriate
names in it automatically issued by LE. Use this new cert before the old
one expires.
Change-Id: I635d2bfd820fe138ee951833dd66f157b2b7c097
This site was never used nor published, it can be killed according to QA
PTL.
codesearch returns no matches for it in any docs.
Keep the occurence in manifests/static.pp, this will get deleted
as part of https://review.opendev.org/710388.
Change-Id: I3c0d3b567a3eccb959dc903f169197e4581f1e13
There is a bug, or misfeature, in acme.sh using dns manual mode where
it will not renew the certificate when new domains are added to an
existing certificate. It appears to generate the TXT record requests
correctly, but then when we renew the certificate it thinks it is not
time and skips it. This is filed upstream with [1] however we can
work around it, and generally be better anyway.
For each letsencrypt host, during certificate request we build up the
"acme_txt_required" key which is a list of TXT record tuples.
Currently we keep the challenge domain in the first entry, which is
not useful (all our hosts have the same challenge domain,
amce.opendev.org). Modify this to be the certificate key from the
host config. To be clear; when a host has
letsencrypt_certs:
hostname-cert-main:
hostname.opendev.org
altname.opendev.org
hostname-cert-secondary:
secondary.opendev.org
secondaryalt.opendev.org
acme_txt_required when renewing all certs will end up looking like:
[
(hostname-cert-main, <txt1>), (hostname-cert-main, <txt2>),
(hostname-cert-secondary, <txt3>), (hostname-cert-secondary, <txt3>>)
]
In the certificate creation path, we walk "acme_txt_required" and take
the unique 0-value entries; this gives us the list of keys in
"letsencrypt_certs" which were actually updated.
We can then force renewal for these certs, because we know they
changed in some way that requires reissuing them (within renewal time,
or new domains).
This isn't just a work-around, it is generically better too.
Previously if any cert on host required an update, we would try to
update them all. This would be a no-op; acme.sh would just skip doing
anything; but now we don't even have to call into the renewal if we
know nothing has changed.
[1] https://github.com/acmesh-official/acme.sh/issues/2763
Change-Id: I1e82c64217d46d7e1acc0111dff4db2f0062c42a
The content for many projects has moved but the legacy redirects were
not updated, update to current location.
Change-Id: I7030ad35378085b0c45429c272dc24f00d33b2d2
This is a slight divergence from the accepted spec, where we were
going to implement these redirects via a new haproxy instance
(I961456d44a56f2334d3c94ef27e408f27409cd65). We've decided it's
easier to keep them on static.opendev.org
The following sites are configured to redirect to whatever they are
redirecting to now on static.opendev.org:
* devstack.org
* www.devstack.org
* ci.openstack.org
* cinder.openstack.org
* glance.openstack.org
* horizon.openstack.org
* keystone.openstack.org
* nova.openstack.org
* qa.openstack.org
* summit.openstack.org
* swift.openstack.org
As a bonus, they all get a https instance too, which they didn't have
before.
testinfra coverage should be total for this change. I have created
the _acme-challange CNAME records for all the above.
Story: #2006598
Task: #38881
Change-Id: I3f1fc108e7bb1c9500ad4d1a51df13bb4ae00cb9
When converting this from a htaccess file to run in the virtualhost
context, one instance of '^cgit' -> '^/cgit' was missed. Fix it, and
add a coverage test for it to testinfra.
Change-Id: Icc1dae6dce232e69c5cd1cf98b594f562c60d3f2
* Remove a stray trailing ' from the key
* update the key url to use https
* fix the log path to scrape
Change-Id: I580b63f08147494a937d44f4f6637947221c8937
The documentation is wrong, this needs to be just "host:" (fixed
upstream with
cf4882e0c0
but not released yet).
Change-Id: I5110e6795fa8b5729ddc87da6aef7c9ac7ed39a4
This creates the redirect sites
git.airshipit.org
git.openstack.org
git.starlingx.io
git.zuul-ci.org
The htaccess rules are put into the main configuration file to avoid
having to create a directory and manage another file. We use a macro
to duplicate the rules and retain the old semantics of the http site
redirecting directly (as opposed to doing a extra 301 to
https://git.openstack.org first). This required adding "/" to the "^"
matches as it now runs in VirtualHost context; no functional change is
intended over the old sites.
This will require _acme-challenge CNAMEs to acme.opendev.org before
being merged.
testinfra is updated to exercise some redirects matching against the
results of the extant sites.
Change-Id: Iaa9d5dc2af3f5f8abc11c2312e4308b50f5fcd2b
This only needs to run on the executor, specify a blank nodeset.
Add the static.opendev.org host key after adding the host.
Change-Id: Iedde486ce8f3e9b415991830121fb87ba192afc6
files.openstack.org serves a view of /afs/openstack.org/, which is the
same as static.opendev.org. Add a serveralias for it and certificate.
Make static.openstack.org be consistent with opendev by showing the
same thing.
Change-Id: I4c492e3b02554a7c736c015790bd4cd5bb435a43
This moves the creation of a zuul user with the Zuul per-project key
for system-config to a separate role from the static role, so it can
be reused on other hosts.
Change-Id: Ice605b70a2c42d9b85090406216253fec0820f50
This is an alternative to Iccf24a72cf82592bae8c699f9f857aa54fc74f10
which removes the 404 scraping tool. It creates a zuul user and
enables login via the system-config per-project ssh key, and then runs
the 404 scraping script against it periodically.
Change-Id: I30467d791a7877b5469b173926216615eb57d035
The source pattern for the tarballs.openstack.org openstackid
redirect incorrectly included an openstack parent directory. Remove
it, an also make the regex more properly differentiate lack of a
trailing "/" character from a directory name containing openstackid
as a substring (not that there is one, but this serves as a safer
template for future additions).
Change-Id: I705d849d1c10cf91391181aeef72a9f4b495d520
While the service is review.opendev.org, the server is actually
review01.openstack.org. The ansible inventory in production knows
it that way, as does the Nova in RAX DFW. Update the host_vars
entry and the zuul jobs so that it matches (And so that LE certs
apply)
Change-Id: I4c762c57f6826f2c5f9ed5c9cb0ae02644570c3d
* migrated user storage to idp
* created users crud
* created groups crud
* migrated from eloquent to doctrine orm
* reafactoring
Change-Id: I766bbb75c0e65f504880e8c59951f63494a1e13f
Signed-off-by: smarcet <smarcet@gmail.com>
The blanket redirect from tarballs.openstack.org to
tarballs.opendev.org/openstack is only relevant for projects
publishing from the openstack namespace, since our new publication
jobs put content into Git-namespace-specific trees. Start a game of
Whack-a-Mole with the (hopefully few) projects in this situation.
Change-Id: I5d54532ef512df449d62299391d853d30862b7f2
Removed all variables related to Silverstripe
Dependency
Change-Id: Ib5e6834686c4952dd8e7220a31abe71a9278e397
Signed-off-by: smarcet <smarcet@gmail.com>
This reverts commit c25e91f49632d8e187f35807f250567446bd5102.
This script parses the Apache logs and writes out a local count of the
404 data to files.openstack.org, and then exports it via
files.openstack.org.
As part of the spec [1] we're trying to remove publishing from local
volumes, in general.
Since this is not widely used, there is only one link to it, it's not
discoverable from the landing page of files.openstack.org (which just
shows the afs directory listing), it has a very long latency making it
not that useful for debugging and grepping the logs there have been no
accesses in the past 2 weeks (as far back as logs go) I propose we
remove it.
If we want to retain this, we should publish the output alongside the
docs AFS volume. That could certainly be done by distributing the
docs keytab to the host and having it write out in a similar cron job.
Another option could be to setup a keypair for remote login and keep
that as a secret in Zuul, and do the same from a periodic job
(complicated by apache logs being root only, so needs some sudo magic
or similar). Or, we could figure out an altogether better, privacy
respecting client analytics solution.
[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/retire-static.html
Depends-On: https://review.opendev.org/709036
Change-Id: Iccf24a72cf82592bae8c699f9f857aa54fc74f10
We need to use bazelisk to build gerrit so that we can properly
track bazel versions in the job. Use the roles developed for
gerrit-review to do that, then simplify the dockerfile to have
it simply copy the war into the target image.
Also add polymer-bridges.
Depends-On: https://review.opendev.org/709256
Change-Id: I7c13df51d3b8c117bcc9aab9caad59687471d622
