This change adds a proxy config for quay which should assist
us when gating using images provided by the publically
available registry.
Change-Id: I971705e59724e70bd9d42a6920cf4f883556f673
Signed-off-by: Kevin Carter <kecarter@redhat.com>
There have been several Web sites added to files.o.o which missed
getting x509 SSL/TLS certificate checking added through our
certcheck cron job. Add those now so we know in advance whether
they're at risk of expiration.
Change-Id: I3eda77f165348e510d43344b172cf5b56ce2b003
Docker hosts report back mounts in container directories via snmp
storage queries
# php -q /usr/share/cacti/cli/add_graphs.php --host-id=585 --snmp-field=hrStorageDescr --list-snmp-values
Known values for hrStorageDescr for host 585: (name)
...
/var/lib/docker/containers/05ed2dc...
/var/lib/docker/containers/7cebed4...
/var/lib/docker/containers/f452861...
Because these can keep changing, hosts just end up getting more and
more invalid graphs in their results (see gitea0X hosts in cacti at
the moment).
Filter out docker directories from the query
Change-Id: Ia1db628975e7a67ad531438ef85735abae1ce652
Full replication is very costly and makes gerrit restarts expensive
these days. Turn off replicate_on_startup.
Depends-On: https://review.opendev.org/678486
Change-Id: I31d81821c645697e72a8702c60e2482156e01bb0
Because of a limitation in GnuPG we need to have the Jessie archive
signing key in the list of VerifyRelease key IDs for the Debian
reprepro mirror. Also some suites (currently buster-backports) are
signed by a subkey of an archive signing key, so add the "+" suffix
to all these key IDs indicating subkey signatures are also allowed.
As always, Debian signing keys are published and available here:
https://ftp-master.debian.org/keys.html
Change-Id: Iedce38318718a18ace7b2c638755a7d7d4dcd69b
When a job is killed by zuul due to failure like DISK_FULL, a different
message ends up in as a comment.
<li>job-name
finger://ze09.openstack.org/8b6d...6f : DISK_FULL in 2h 59m 50s</li>
This adds another pattern that recognize these messages as failures,
regardess the case (DISK_FULL in this case).
Change-Id: Ib17f05a043430362b02a2826d69572f6b2dbd64a
Needed-By: https://review.opendev.org/#/c/631509/
The buster-backports suite on Debian mirrors is not signed by the
old jessie signing key we have set to verify in reprepro, but also
we're not mirroring Debian 8/jessie any longer anyway. Replace that
list with the 9/stretch and 10/buster signing keys and switch to
longer key IDs which match the names used for them in the Puppet
manifest. Also add Puppetry and keyfile for the buster keys so that
they will be installed accordingly. The official list of keys can be
found here: https://ftp-master.debian.org/keys.html
Change-Id: Ia193f040b2b707329948955eb091a186eabf8096
This rsync'd mirror is now being managed by the opendev mirror update
server. Remove it from the older openstack server to avoid a conflict in
excludes around sclo repo.
Currently we have opendev adding sclo and openstack removing it.
Change-Id: I599ee7d0fab8c5e2a060aff86bce20f1f8d4f54b
Zuul has hit a scenario where a git repo update was unable to talk to
gerrit via ssh because it had reached its per user connection limit [0].
This then led to some openstack job failing [1].
The default limit (which we were using) is 64 connection per user.
Apparently this is not quite enough for a busy zuul? Increase this by
50% up to 96.
[0] http://paste.openstack.org/show/754741/
[1] http://lists.openstack.org/pipermail/release-job-failures/2019-July/001193.html
Change-Id: Ibeca2208485608f3b61aa716184165342bfcc3c9
This is a follow on to I67870f6d439af2d2a63a5048ef52cecff3e75275 to do
the same for files.openstack.org (as
http://files.openstack.org/mirror/logs/ is a handy central place to
point people at)
Change-Id: I07c707d45ab3e3c6f87460b3346efd7026467c56
Apply the exclusion for trusted CI comments to the hide function's
conditional case as well as the toggle function's.
Change-Id: Ia4e5ec22a097a8b8cb564c237fd0aa48ab6f8724
When filtering CI system comments, don't hide those from Zuul, our
gating CI system. It is important to see these comments as not all
results may match the patterns used to expose them as rows in the CI
table. Rename the "Toggle CI" button to "Toggle Extra CI" so that
the name remains accurate without being too verbose.
Change-Id: Id0cd8429ee5ce914aebbbc4a24bef9ebf675e21c
This used to be mirrored, however there were issues when upstream
dropped the PC1 repositories a few months back. The puppet openstack
jobs are still trying to leverage this mirror but it does not exist in
some regions because it was disabled on the afs content. This change
fixes the reprepo configuration to still pull down puppet5/6 for xenial
and strech and add the symlink back to the mirrors.
Change-Id: I71ad5afe086a503d75a365543ad8869e35ef873b
This change adds a proxy config for registry.access.redhat which should
assist us when gating using images provided by the publically available
registry.
Change-Id: Ica7477d63659610de852d305a63f3e78d0dd8c4f
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This move was prompted by wishing to expose the mirror update logs for
the rsync updates so that debugging problems does not require a root
user (note: not actually done in this change; will be a follow-on).
Rather than start hacking at puppet, the rsync mirror scripts make a
nice delination point for starting an Ansible-first/Bionic update.
Most magic is included in the scripts, so there is not much more to do
than copy them. The host uses the existing kerberos and openafs roles
and copies the key material into place (to be added before merge).
Note the scripts are removed from the extant puppet so we don't have
two updates happening simultaneously. This will also require a manual
clean to remove the cron jobs as a once-off when merging.
The other part of mirror-update is the reprepro based scripts for the
various debuntu repositories. They are left as future work for now.
Testing is added to ensure dependencies and scripts are all in place.
Change-Id: I525ac18b55f0e11b0a541b51fa97ee5d6512bf70
This reverts commit b3ce1c52dc7ca455ffd94ea07d8a4fb1b6905fa8.
It removed the AFS mirror at the same time it added the proxy,
but jobs don't know to look for the proxy since it's on a
totally different TCP port.
Change-Id: I87cc03eb3322bd7b093dd6fe798aadb48f319805
This removes groups.openstack.org as this service was shut down. Add new
opendev services behind ssl.
Change-Id: I14c667c8fbde07c3a52778bc2c5e93abf8f053a4
Previously we evaluated the vhost templates before setting
ssl_cert_file_ and ssl_key_file_ and ssl_chain_file_. This made erb
unhappy because those are the three variables we use to set paths in the
vhost. Fix this by moving the vhost after the ssl file vars are set.
Change-Id: I4ba62521c9e7da104f8799d016cbcf0214cbdfc1
To deal with puppet scoping fun we evaluate the template for our
files.o.o website vhosts in the context of the website define and not in
the context of httpd::vhost.
Change-Id: I90bb881eb6ad78cede3a8a2548e1dfcf24e1160b
As a follow-on to I0e110ef51c8ed301fd8280ae7fc039e3b01db92c; this
dropped the /centos/ from the base mirror, add it back.
Also switch the mirror to the only one on the altarch-mirrors page
that is in US/TX, which from the name is in Dallas, which must be
pretty close to rax.dfw where the update server lives.
Change-Id: If4d71865f4328e73a26c7b38300767ed6b790579
CentOS keeps non-x86 architectures in /altarch/ directory (contrary to
/centos/ one for x86-64). We have aarch64 (arm64) machines in infra and
they fail due to lack of CentOS altarch mirror.
List of wanted alternative architectures is controlled by ALTARCHS
variable (aarch64 and ppc64le enabled). As CentOS has several other
architectures too they are listed in ALTARCHS_IGNORED so we do not fetch
them.
Current CentOS mirror lands in same /mirror/centos/7/ directory. Altarch
mirrors goes to /mirror/centos/altarch/7/ one.
Change-Id: I0e110ef51c8ed301fd8280ae7fc039e3b01db92c
This way we can send a single email that our users can see if subscribed
to this list instead of sending emails to all of their discuss lists.
Change-Id: I3b978a3c4e7888f14e3986628cb29a6c86bbcf61
The yum-puppetlabs mirror exceeded its 100GB quota as of April 26.
Rather than increase the quota, start excluding packages for old
platforms we don't provide like RHEL5-6 and Fedora F20-27. We could
probably get even more aggressive with it, but this get the
utilization back under 50% which is plenty of headroom for now.
Change-Id: I9665b3a2a89f991f9433fe7f45bc1bb0e0c7632b
It seems the openSUSE build process can leave artifacts behind,
in the form of .~tmp~ files in the mirror. I assume these are
wrongfully present.
This is a problem, as those ~tmp~ files prevent syncing the
repositories.
While it's most likely that openSUSE files will be cleaned in the
source repos, should this problem arise in the future, it's also
more robust to skip the syncing of those files.
This has the extra benefit of temporarily unblock mirroring of
openSUSE Leap 15.1 in infra, as of today.
Change-Id: I0124b992483cfda9f97960b43bddf94efa008030
Build a container image with the haproxy-statsd script, and run that
along with the haproxy container.
Change-Id: I18be70d339df613bf9a72e115e80a6da876111e0
It doesn't seem like this is used anymore. Let's remove it before
we update the rest of this, so that we don't have to, you know,
update abandoned things.
Change-Id: I1c3708021046a428da82eaa843961091915ba4af