This patch adds a vhost that exposes /afs/openstack.org/mirror/wheel
to the internet under <hostname>/wheel. It also adds appropriate
rewrite rules.
Change-Id: Id514a76f7759b663b2ea3a3ec4aa2dfe083d6666
This adds a hiera-loaded keytab for the wheel slaves. It may
be used when writing wheels to the AFS share directory.
Note: This will require the creation of a keytab for the
wheel mirror, as well as appropriate folders and access rights
on AFS.
Change-Id: I81c39d420d8ac70def57949ea0d4c323b8797086
This removes the SSH keys from the wheel mirror slaves, and
replaces them with the AFS share. It does not yet add afs
credentials. As we have not yet provisioned our wheel slaves,
no manual updates of the servers should be required.
Hiera keys for the previous approach should be removed.
Change-Id: Ifebf5d53d80e934674704078f7dd675f77aef5aa
Consumed puppet modules may declare these (example: openafs).
This patch wraps the kernel header modules in test conditionals to
avoid conflicts.
Change-Id: I72962ae0d006cfe5f5d7e61046fd4862067b3b44
Update nodepool logging for Fedora 23 image builds. There are also a
lot of other images that have been removed in the mean time.
Depends-On: I6ff6d684e17b0581ae6fbb310af29dd1be0e54d7
Change-Id: I5ffe9098a6bcf0e565ccc5b556b943d02eae26cd
While we perform an initial sync to a new site, updates to the RO
volume are suspended, so switch to the RW volume temporarity.
Change-Id: Ic14fc8c462f3eacfc9faed4b6bae55cd56b5286d
Now that the nodepool builder is running as a separate daemon it needs
its own log config file. Move the auto generated nodepool logging config
stuff over to the new builder logging config as we can manage the main
daemon's logging config by hand trivially now.
Depends-On: I013835621dfbc311a0f7bd7c957b7d4656dfa628
Change-Id: Ic1da30eab949876e5bd6c88e83979bdedc6dd50a
This patch adds two RewriteCond with backreferences which
check for the existence of an AFS-style split directory. If
the directory exists, the rewrite rule will apply the necessary
path changes.
This patch permits passthrough of files such as index.html
at the root directory, while not impacting the folder structure
at sub-directory levels.
This patch will break rather badly if pypi ever publishes a
package named 'index.html'.
Change-Id: I712f2d4e262b9e20378365af8173066522e2e0cd
The script tools/normalize_projects_yaml.py in project-config is run as
periodic job on the proposal slave. It needs the Python yaml module, so
install it.
Change-Id: I113dcfa13da34b3981de8910e6d07d69c5f95dfc
This adds the parent directory /var/www, which is required to
create the directory /var/www/mirror.
Change-Id: I6b8c7494c4bfebd8160c9cd6f75f0cb363bfa839
The full manage-projects run is taking as much as 20 minutes now,
leaving new project creation broken when it trips the timeout.
Double it from 15 to 30 minutes for some added safety margin.
Change-Id: Icc36aca2f7b949347ec891bba6cb533932428171
This patch ensures that the 'extra-index-url' field is available
in the global pip.conf file, so that project-config's
configure_mirrors.sh script can use sed to replace it with the
region appropriate wheel mirror.
Change-Id: Ic9b069313c7a4e105cd086bdc4516c6c5826d53e
This patch adds a vhost that exposes /afs/openstack.org/mirror/pypi/web
to the internet under <hostname>/pypi. It also adds a do-not-follow
robots.txt file.
Change-Id: I3d7629d80fe1e8762344852fab4a942bc476b706
This patch creates a new manifest for unified infra mirrors,
which will act as read-only AFS nodes that host our mirror
data. These mirrors will, once validated and provisioned, replace
the existing pypi mirrors in infra under a new, more generic,
hostname.
This patch is only intended to create the AFS read-only slave.
Apache hosting will be added in subsequent patches.
Change-Id: I9a2bf596cf47bffad5d6a5fd0da3c571fa266013
The wheel-mirror workers share a common set of hiera keys, and so
keeping them in a group together reduces significant duplication in
our hiera files.
Change-Id: I67b717943eb19404c87b2a3b571f681a0a15b7b4
This is a machine that will drive centralized mirror updates for
mirrors that are in AFS.
Change-Id: I33eed90c21a2e57a26349be83ee69e968b18d2f8
Depends-On: I56bb3ce2237be1179724f7a2bdcf9d5b04bdecd2
Depends-On: I704c8bf395d83adea0f9a0db6fd3d3814ad7660a
When we create a new server and install puppet, we also install pip.
That's awesome, except when pip >=8 breaks us. For now, pin it.
Change-Id: Id12f866f577c3ca2405a6049084c3cb0af82fde5
