Some jobs are using Fedora 29 release, which is deprecated.
Create new mirror with release 31 and wait for changing job
rules.
Change-Id: I418f4c815d337c2b1edd0df1aa065536d992cb62
This will give us a nice link to the goaccess reports on the zuul
dashboard build pages.
Move ansible-lint config into config file
As of 4.2.0 we can configure ansible-lint with a config file. It's
also apparently now smart enough to only find ansible yaml. Let's
see how that goes.
Add a fake zuul_return module
This should let us fake out ansible-lint without having to install
all of zuul.
Change-Id: Ib233eb577a8ca2aabfe3a49b2cd823dd4a00bd82
This switches all log file input to stdin using a subshell to cat all
the related log files. This makes the command a bit more consistent with
itself.
Change-Id: I99a8b0326f2a27c8497d616501fd9bed48678da8
This adds a simple periodic job that runs against static.opendev.org and
produces opendev reports. To start we don't publish these reports
publicly so that we can double check their contents first. Assuming that
all goes well we'll be able to apply this to all of our
static.opendev.org hosted sites and publish these reports through the
zuul logging system.
Change-Id: I66705808d435c16bf0da6989296c896099964aaa
Story: 2007387
This adds the Open Edge (formerly Fortnebula) CI mirror.
Change-Id: I1ccf2a602f8a41e00bc64a9516a326cc07d9b254
Depends-On: https://review.opendev.org/711787
Sister change for Ia5caff34d3fafaffc459e7572a4eef6bd94422ea and
removing earlier references to the mirror server in preparation for
building and adding the new one.
Change-Id: I7d506be85326835d5e77a0c9c461f2d457b1dfd3
This adds a simple role to install Zookeeper.
Add an option to nodepool-base to use this role to install Zookeeper.
Use this in the nodepool-builder gate testing where we are just
validating that the nodepool-builder container starts and is ready to
accept connections. It needs a zookeeper to talk to, even though it
is not going to do anything.
Change-Id: I4ae89a51e454be4ee53ad4e04407162aaa8d9f9a
This is a start at ansible-deployed nodepool environments.
We rename the minimal-nodepool element to nodepool-base-legacy, and
keep running that for the old nodes.
The groups are updated so that only the .openstack.org hosts will run
puppet. Essentially they should remain unchanged.
We start a nodepool-base element that will replace the current
puppet-<openstackci|nodepool> deployment parts. For step one, this
grabs project-config and links in the elements and config file.
A testing host is added for gate testing which should trigger these
roles. This will build into a full deployment test of the builder
container.
Change-Id: If0eb9f02763535bf200062c51a8a0f8793b1e1aa
Depends-On: https://review.opendev.org/#/c/710700/
This has got me a number of times; I think we can tell in review if a
task firing in response to a "changed" is best in a handler or not.
Remove existing noqa flags
Change-Id: I80ad631f978eeeb9903abe230a95f23f5709d20e
When testing our system-conf configuration we don't actually add zuul to
the docker group. This means the zuul user cannot access the docker
socket. This then breaks docker container log collection. Address this
by becoming root when collecting logs.
Change-Id: Ic0232f7ef458cdd07fb0853f97f2dc22ce137c71
Currently we don't set a contact email with our accounts. This is an
optional feature, but would be helpful for things like [1] where we
would be notified of certificates affected by bugs, etc.
Setup the email address in the acme.sh config which will apply with
any new accounts created. To update all the existing hosts, we see if
the account email is added/modified in the config *and* if we have
existing account details; if so we need a manual update call.
For anyone who might be poking here, we also add a note on sharing an
account based on some broadly agreed upon discussion in IRC.
[1] https://community.letsencrypt.org/t/revoking-certain-certificates-on-march-4/114864
Change-Id: Ib4dc3e179010419a1b18f355d13b62c6cc4bc7e8
This site was never used nor published, it can be killed according to QA
PTL.
codesearch returns no matches for it in any docs.
Keep the occurence in manifests/static.pp, this will get deleted
as part of https://review.opendev.org/710388.
Change-Id: I3c0d3b567a3eccb959dc903f169197e4581f1e13
There is a bug, or misfeature, in acme.sh using dns manual mode where
it will not renew the certificate when new domains are added to an
existing certificate. It appears to generate the TXT record requests
correctly, but then when we renew the certificate it thinks it is not
time and skips it. This is filed upstream with [1] however we can
work around it, and generally be better anyway.
For each letsencrypt host, during certificate request we build up the
"acme_txt_required" key which is a list of TXT record tuples.
Currently we keep the challenge domain in the first entry, which is
not useful (all our hosts have the same challenge domain,
amce.opendev.org). Modify this to be the certificate key from the
host config. To be clear; when a host has
letsencrypt_certs:
hostname-cert-main:
hostname.opendev.org
altname.opendev.org
hostname-cert-secondary:
secondary.opendev.org
secondaryalt.opendev.org
acme_txt_required when renewing all certs will end up looking like:
[
(hostname-cert-main, <txt1>), (hostname-cert-main, <txt2>),
(hostname-cert-secondary, <txt3>), (hostname-cert-secondary, <txt3>>)
]
In the certificate creation path, we walk "acme_txt_required" and take
the unique 0-value entries; this gives us the list of keys in
"letsencrypt_certs" which were actually updated.
We can then force renewal for these certs, because we know they
changed in some way that requires reissuing them (within renewal time,
or new domains).
This isn't just a work-around, it is generically better too.
Previously if any cert on host required an update, we would try to
update them all. This would be a no-op; acme.sh would just skip doing
anything; but now we don't even have to call into the renewal if we
know nothing has changed.
[1] https://github.com/acmesh-official/acme.sh/issues/2763
Change-Id: I1e82c64217d46d7e1acc0111dff4db2f0062c42a
The content for many projects has moved but the legacy redirects were
not updated, update to current location.
Change-Id: I7030ad35378085b0c45429c272dc24f00d33b2d2
This is a slight divergence from the accepted spec, where we were
going to implement these redirects via a new haproxy instance
(I961456d44a56f2334d3c94ef27e408f27409cd65). We've decided it's
easier to keep them on static.opendev.org
The following sites are configured to redirect to whatever they are
redirecting to now on static.opendev.org:
* devstack.org
* www.devstack.org
* ci.openstack.org
* cinder.openstack.org
* glance.openstack.org
* horizon.openstack.org
* keystone.openstack.org
* nova.openstack.org
* qa.openstack.org
* summit.openstack.org
* swift.openstack.org
As a bonus, they all get a https instance too, which they didn't have
before.
testinfra coverage should be total for this change. I have created
the _acme-challange CNAME records for all the above.
Story: #2006598
Task: #38881
Change-Id: I3f1fc108e7bb1c9500ad4d1a51df13bb4ae00cb9
When converting this from a htaccess file to run in the virtualhost
context, one instance of '^cgit' -> '^/cgit' was missed. Fix it, and
add a coverage test for it to testinfra.
Change-Id: Icc1dae6dce232e69c5cd1cf98b594f562c60d3f2
* Remove a stray trailing ' from the key
* update the key url to use https
* fix the log path to scrape
Change-Id: I580b63f08147494a937d44f4f6637947221c8937
The documentation is wrong, this needs to be just "host:" (fixed
upstream with
cf4882e0c0
but not released yet).
Change-Id: I5110e6795fa8b5729ddc87da6aef7c9ac7ed39a4
This creates the redirect sites
git.airshipit.org
git.openstack.org
git.starlingx.io
git.zuul-ci.org
The htaccess rules are put into the main configuration file to avoid
having to create a directory and manage another file. We use a macro
to duplicate the rules and retain the old semantics of the http site
redirecting directly (as opposed to doing a extra 301 to
https://git.openstack.org first). This required adding "/" to the "^"
matches as it now runs in VirtualHost context; no functional change is
intended over the old sites.
This will require _acme-challenge CNAMEs to acme.opendev.org before
being merged.
testinfra is updated to exercise some redirects matching against the
results of the extant sites.
Change-Id: Iaa9d5dc2af3f5f8abc11c2312e4308b50f5fcd2b
This only needs to run on the executor, specify a blank nodeset.
Add the static.opendev.org host key after adding the host.
Change-Id: Iedde486ce8f3e9b415991830121fb87ba192afc6
files.openstack.org serves a view of /afs/openstack.org/, which is the
same as static.opendev.org. Add a serveralias for it and certificate.
Make static.openstack.org be consistent with opendev by showing the
same thing.
Change-Id: I4c492e3b02554a7c736c015790bd4cd5bb435a43
This moves the creation of a zuul user with the Zuul per-project key
for system-config to a separate role from the static role, so it can
be reused on other hosts.
Change-Id: Ice605b70a2c42d9b85090406216253fec0820f50
This is an alternative to Iccf24a72cf82592bae8c699f9f857aa54fc74f10
which removes the 404 scraping tool. It creates a zuul user and
enables login via the system-config per-project ssh key, and then runs
the 404 scraping script against it periodically.
Change-Id: I30467d791a7877b5469b173926216615eb57d035
The source pattern for the tarballs.openstack.org openstackid
redirect incorrectly included an openstack parent directory. Remove
it, an also make the regex more properly differentiate lack of a
trailing "/" character from a directory name containing openstackid
as a substring (not that there is one, but this serves as a safer
template for future additions).
Change-Id: I705d849d1c10cf91391181aeef72a9f4b495d520