We're seeing nodes IPv6 nodes occasionally come up with no working
IPv4 connectivity to the Internet. Declare that we require IPv4
configuration and egress connectivity so these nodes will be
discarded before they cause jobs expecting to be able to reach
v4-only resources to fail.
Change-Id: I5ebf008998b62f044d0a686de8a2337851ef3cf8
Depends-On: https://review.opendev.org/763065
Similar to ORD region Ieb856cadcdeb9f6e099b1ce2711cc0d5e1e61f5a
Depends-On: https://review.opendev.org/760494
Change-Id: I53b3fbd3191c2c4748fd01d1e480fc307be31557
This converts RAX to use the internal address for mirror access
Depends-On: https://review.opendev.org/759971
Change-Id: Ieb856cadcdeb9f6e099b1ce2711cc0d5e1e61f5a
The OVH mirrors are being rebuilt with opendev.org domain versions,
so add them to the region list in the mirror map.
Change-Id: Ic12c1ac924ffb676ee284a82a169defeb42513fc
New clouds aren't getting openstack.org mirrors but instead opendev.org
names. We have to override the mirror name based on region and cloud in
order for jobs to properly find the mirror.
Change-Id: I4ed2f55416f1898746c414b2c443bc9bd5a100fe
We have enough quota for 44 8-node hosts; give ourselves a little
headroom. The mirror host is up and responding.
Story: #2007195
Task: #38357
Change-Id: Ib6797c5bcc90d492c4a47e173b93b3867548879f
This reverts commit b025023dd2439dedfca1f6fc095cdf737f481f8f.
This option is now disabled by default. Stop setting it explicitly.
Change-Id: Ida64b575cedae46dae94c32c1edcba30a0d4a9e8
We set stage_compress_logs to false to stop compressing files during the
stage-output role. We are planning to make this the default in zuul-jobs
but this allows us to test it in opendev first.
Change-Id: I5e959912f4e2e9df264b0434f7a73f9f5a439102
The mirror.mtl01.inap.opendev.org server is setup and running, this
region should be using it.
Story: 2006762
Task: 37268
Change-Id: Ie9015c7615c26abe42c6500e3258756ad05da500
This reverts commit 3e5df9727f246f6504fc41bf5d9354b6fec67b2b.
We have been testing with the kernel referenced in the original for
several days and have enough data. There is an open issue with oops
in fscache which we need to resolve for this to be stable.
Change-Id: I691e2bd676816884f732601229c98b578efc46b3
This is running afs-next branch @
579cb95f6d3a8f7db0ee361d6e8ed0c448c7a244. This contains a number of
AFS fixes proposed for 5.3 series which we would like to sign-off as
tested in our environment.
In particular, we want to montior for messages like:
kAFS: Volume 536871010 'mirror.opensuse' is offline
across vos release of the mirrors. There should be changes to
fallover to alterntive servers better during release to avoid such
errors.
Prior reported fscache issues [1] are still are known problem and
being worked on. We'll have to monitor the server; we can revert this
change after we have had sufficient testing if there are issues.
See
https://etherpad.openstack.org/p/opendev-mirror-afs
[1] https://www.redhat.com/archives/linux-cachefs/2019-June/msg00011.html
Change-Id: I5d696907729fedd60783b13aac6537fb75c975fc
I did not add openstack.org DNS records for the mirror in this cloud
region so we need to use the opendev name instead.
Change-Id: I4bcc80ca8bfec799f15c06e13c667f121896f8c5
We have some new changes to try that need to be rolled out. In the
mean time, they have a tendency to hard crash in fscahe which needs
very active monitoring which isn't happening right now.
Change-Id: Ia12096af302f61e437e2b65c900c34da1b2a9d59
This is our second kafs based mirror (rax.IAD being the first), chosen
to be geographically remote to the AFS servers. It is currently
manually configured with 5.2-rc7 kernel, which has all the current
kAFS fixes incorporated.
Switch it in for initial testing
Change-Id: I83c73a8cff9728336b6208cab04e7c97c4c4ad86
The mirror.ord.rax.opendev.org host is running, with OpenAFS 1.8.3.
This is similar to rax.dfw which has been running without issue since
I78d938eaaec993ac64c7e701a506a108f5c7fb07.
Switch it in.
Depends-On: https://review.opendev.org/663852
Change-Id: I5d9d9d5f2d54c6023e3d08988579d667714affb5
This reverts commit d8b7f2da286f88a4bb687897f6f514384871aa6e.
The IAD server is now running 5.1.9 stable kernel, so we can try again
for a little and see what we get. We also have some debugging plans
(capturing /proc/<pid>/stack for apache) if it doesn't quite work.
Change-Id: I250596015d9d0fa4f37112c05c48a70957968ac1
This reverts commit 79cad08fffd85d6c705e7eb2e9162b3d1f67e1de.
The kafs running mirror.iad.rax.opendev.org ended up not servicing
requests in a timely manner. The server had a load >130 and apache was
using at least one full cpu.
Change-Id: I362c53d67a0e1330d02cb1ee6b2bd10bfcc276f6
We have identified issues with openafs meaning the DFW opendev.org
mirror is unstable. The IAD mirror is running a (currently
hand-installed) version of kafs to provide the mirror directories (the
host is currently in the emergency file).
Switch in IAD for testing kafs, and switch out DFW which is giving
occasional errors to jobs.
Change-Id: I5115030cd323db459b248e21413b68f0ad0aafa8
Brown bag fix for errors in
I631e3e3592a9be5c7da642cf34eec6d9758bf161 which are causing job
failures.
Change-Id: I06dcb09a4825a8e39d2dc000e7425649aea7e6a8
We now have an opendev.org mirror in rax.dfw region (see
I26b9ed1e21e2111f48bc7ecc384880c274eed213 and related changes).
Switch this in for initial testing. As mentioned, this can grow as we
bring new mirrors online, but will eventually be removed once all
opendev.org mirrors are active.
Change-Id: I631e3e3592a9be5c7da642cf34eec6d9758bf161
This is backwards compat for openstack, not an actual thing
we want to do generally.
Depends-On: https://review.opendev.org/656969
Change-Id: I107127df8bb683fbefee328c45d81c1a759f69a2
Remove using old location, we can switch now to the new location
everywhere.
Note: We can only merge this after new images are build with the new
location build in.
This needs a recheck once change 1 is in since project-config is
not self-testing. So, we can remove jenkins/data/bindep-fallback.txt in
this change.
Change-Id: I08f23e96defb152946ade1490628f37f0215300d
Depends-On: https://review.openstack.org/543141
Depends-On: https://review.openstack.org/543142
We have a todo list item to pull these from site-variables, get that
done. With this, our base pre playbook is pretty copyable if someone
wanted to.
Change-Id: Icfd1d416a9f4520775b6e33c4055dbd223600cbb
The bindep role is set up to allow setting a bindep command and fallback
file via variables, and to do the right thing if they're not there.
Add bindep_command and bindep_fallback to our site-variables so that we
can remove copy-pasta references from jobs.
Change-Id: I94dfe8e6aab9c2da1283590be5e26f7d8c35dcad
We actually want to be using the cloud name, not provider name when
building up our fqdn for mirrors.
Change-Id: I35e846dc2e7162a6113e67dc93d30979ccf6ee50
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This moves our mirror configuration out from unittests in zuul-jobs
into our base job. This also creates our site-varaibles file for our
zuul-executors.
Change-Id: I9f495e1568efcbcb74260579ca26ef040eecd748
Depends-On: I4a0749b64a71e551e4fbea5b416b46d2d6433d0b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
