When we added 9-stream with I2da3a5e8a45598c6b3ded132ea61b86b4480e262
we had some components pointing to the mirrors and some not. This is
basically because the OpenDev mirrors choose to mirror these parts of
the distribution. It was an oversight to do things like this, as this
is nominally a generic configuration tool.
This proposes that we put each component behind a variable which can
be enabled or disabled. This way users of the role can switch
on-or-off whatever their local mirrors carry, and fall-back to the
generic mirror metalink in other cases.
I imagine that with future distros like this we would have everything
disabled by default in zuul-jobs, and leave it up to sites to
explicitly opt-in the various components. We could announce this via
our usual deprecation path and do similar for 9-stream; but for now
this keeps the defaults as it is currently written.
Change-Id: Ibfd88c137855c22b960099cc8d582b241c3b1db6
This enables deb-src repos on Debian hosts with a flag. The flag
currently defaults to True to maintain backward compatible behavior with
the hardcoded deb-src repos that configure-mirrors previously set. We
intend on flipping this default to False after announcing this change.
The reasons for this are that source package repos are rarely needed
by CI systems, but these repos can consume large amounts of disk in
mirrors. To make it easier to people deploying mirrors we want to avoid
enabling source repos by default.
Change-Id: I7e9cd0ec1e3184c3c0561bbb7d3069feaf5f1ba5
I74b9de7092f182c942a58ac7a46b9fbd791889de hit a common ansible gotcha
where it likes to strip the trailing newline after a {% endif %}.
This has resulted in invalid lines in our sources.list.
Unfortunately we miss this because it still exits with 0. Add a
simple test looking for warning output.
Change-Id: I46d393a5e67d10a52c4dcca803176ff368a4b5bd
Out-of-the-box installs of centos-8 do not enable the PowerTools
or HighAvailability repositories. Debian/Ubuntu do not enable
the backports repository by default.
Having these repos enabled by default in CI has led to merging
broken code for OpenStack deployment tooling which is attempting
to also manage the presence/absence of these repositories. It
is challenging to remove these repositories on the running node
because the repo URL (rather than just the name) is required as
input to the apt_repository and yum_repository ansible modules.
This patch adds a role default variable to configure-mirrors to
allow a job to opt out of these extra repositories. The default
is set to 'True' to allow existing jobs to work as before.
Change-Id: I74b9de7092f182c942a58ac7a46b9fbd791889de
Starting with Debian 11 (bullseye), security packages are in
bullseye-security as opposed to older releases like buster/updates.
List the last several stable releases in hopes nobody is trying to
use this role to configure platforms older than Debian 8 (jessie,
the current "oldoldstable").
A followup change demonstrates this works in the test-base-roles
job, but because the job matrices have to be updated in one fell
swoop, and many of those jobs won't work without this change already
merged (due to protected use in our base job), it's not tested
directly within this change.
Change-Id: I2d7712cbfd037a65b9025980a6c0cccd917f8947
With CentOS8.3 release repo files are now
renamed to CentOS-Linux*, update in zuul-jobs
too to sync, also update repo names as per
latest CentOS8 minor release.
The change is needed now as dib images are updated
to reflect new repo files post [1].
[1] https://review.opendev.org/c/openstack/diskimage-builder/+/765963
Change-Id: I7ed9bd582043717ae5bae303ea9b32db7f73008e
zypper wants to autodetect these values and it appears that upstream
changes repo types because we are having errors now. Dropping this
config manually on a held node seems to fix things.
Change-Id: I8ad28da7164d9a0955f43d4864ba24c14f0bd4a3
The failovermethod option was removed in dnf and produces eye catching
warnings. Clean this up to clean up job output by configure-mirrors.
More details at: https://bugzilla.redhat.com/show_bug.cgi?id=1653831
Change-Id: Id646835e0fa9e5e99cde57382b3148caeb55e8bd
High Availability packages have been added as part of core CentOS
repositories in 8.1.
This patch add this repo and enables it as it provides corosync and
pacemaker based HA solution and some other packages required for it.
Change-Id: Idbddd81f251c1ade97892128e52f9214420bead7
There is mirror_fqdn that, if defined, overrides mirrors used for
packages and also mirrors for pypi. This is generally incorrect,
if one wants to use different mirrors for packages vs pypi eggs.
Add pypi_fqdn that defaults to mirror_fqdn, which allows users
to go with a custom pypi_mirror. Make pypi trusting the given
pypi_fqdn value instead of generally unrelated mirror_fqdn.
Change-Id: I12975b57951699351cfc0d40beaeb7c703651dd0
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
We install pydistutils.cfg to catch legacy cases where people
are doing things with easy_install. That code path is not
desirable and increasingly broken, as support for things like
allow-hosts is dropped upstream.
Stop installing the file. This will mean people using easy_install
won't get the benefit of the mirrors, but they shouldn't be
using easy_install anyway.
Remove pydistutils.cfg if it's there
Make sure we don't have any pydistutils.cfg around.
Change-Id: I24a05f456f87bd4cb57ebf89d4245477bf675f97
This keeps each platform's "/etc" template files in a separate
top-level directory.
Additionally, we add a distro major version match to the task import
and rename centos7 (in preparation for centos8).
Change-Id: If65d51a27e30311b1da20522afb6dbce7ee6cf35
debian-security has been mirrored by change[1], so we can use
it now.
[1] https://review.openstack.org/#/c/577316/
Change-Id: I2c88306564b04db3c6e061ab52545c5a63665e53
With the release of fedora-28, the default locations for
fedora-update.repo have changed. We still need to support fedora-27
until fedora-29 is released.
Change-Id: I8eacd659cf18a8dc571aea6531483610fcd91d1a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Today the mirror system in openstack-infra isn't properly setup to
mirror debian-security. Until we can fix this, switch to the upstream
URL.
Change-Id: Icfa7dc874d14de245927d461595f90b9dacc7825
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Tumbleweed as a rolling release model distribution does not
have a sensible release version and is kept distinct from the
classing "release + update" distributions that are called "Leap".
Conditionalize the repository setup for tumbleweed properly,
the base path is different and it does not have a releaseversion
component (the release version is always changing, it is
the snapshot date in the form YYYYMMDD).
Change-Id: I700ba2279dc7aadc7051cad53898e1de9007a08d
CentOS Plus is not being mirrorred by openstack-infra and contains
mostly alternative kernel images which are not expected to be used in
jobs.
Let's reconsider if there is a use case for it in the future but in
the meantime let's remove it.
Change-Id: I9681b3b92c913f45791f610d845a94ccf3a16e48
Under debian, /etc/apt/sources.list.d is where the default list files
are stored. We need to also use that to properly re-write them.
Otherwise, we don't properly configure our AFS mirrors.
Change-Id: I1eb3ac19ba7081d3f4fd9b000e266d86449dee0e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
- Improve the structure to support multiple distributions
- Use one task to set up the different repository files
- Use handler to update the apt cache
Change-Id: I90933164cbca9d230cebeb3e2e1263c7806db840
It make more sense to revert to mirror_fqdn, since it is a FQDN.
Change-Id: I4a0749b64a71e551e4fbea5b416b46d2d6433d0b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
After talking to clarkb, we decided we can drop this in favor of just
using the mirror_host.
Change-Id: Ie4ce336174aa2a825d125f7e0d9a256de139908b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
There is an ansible_managed variable that can be referenced in templates
to add a comment about a file being managed by ansible. Use it.
Change-Id: I60c27cf0eb35dcd4805c3517b3b7e0c7d9a0b3ea
This will be used to replace our current confgure_mirror.sh script
that we today as a ready-script in nodepool.
This only adds support for ubuntu today. As we move forward and
configuration openstack-infra from JJB to ansible, we'll need to add
more distros.
Attach the role to the unittests base job for now. It should ultimately
be attached to the base job, but adding it to the unittests job lets us
test it more easily.
Co-Authored-By: Monty Taylor <mordred@inaugust.com>
Change-Id: I9bfa28c87390c09bb2c4cd0de6ce4c7890f8d81a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>