This patch adds two RewriteCond with backreferences which
check for the existence of an AFS-style split directory. If
the directory exists, the rewrite rule will apply the necessary
path changes.
This patch permits passthrough of files such as index.html
at the root directory, while not impacting the folder structure
at sub-directory levels.
This patch will break rather badly if pypi ever publishes a
package named 'index.html'.
Change-Id: I712f2d4e262b9e20378365af8173066522e2e0cd
The script tools/normalize_projects_yaml.py in project-config is run as
periodic job on the proposal slave. It needs the Python yaml module, so
install it.
Change-Id: I113dcfa13da34b3981de8910e6d07d69c5f95dfc
This adds the parent directory /var/www, which is required to
create the directory /var/www/mirror.
Change-Id: I6b8c7494c4bfebd8160c9cd6f75f0cb363bfa839
The full manage-projects run is taking as much as 20 minutes now,
leaving new project creation broken when it trips the timeout.
Double it from 15 to 30 minutes for some added safety margin.
Change-Id: Icc36aca2f7b949347ec891bba6cb533932428171
This patch ensures that the 'extra-index-url' field is available
in the global pip.conf file, so that project-config's
configure_mirrors.sh script can use sed to replace it with the
region appropriate wheel mirror.
Change-Id: Ic9b069313c7a4e105cd086bdc4516c6c5826d53e
This patch adds a vhost that exposes /afs/openstack.org/mirror/pypi/web
to the internet under <hostname>/pypi. It also adds a do-not-follow
robots.txt file.
Change-Id: I3d7629d80fe1e8762344852fab4a942bc476b706
This patch creates a new manifest for unified infra mirrors,
which will act as read-only AFS nodes that host our mirror
data. These mirrors will, once validated and provisioned, replace
the existing pypi mirrors in infra under a new, more generic,
hostname.
This patch is only intended to create the AFS read-only slave.
Apache hosting will be added in subsequent patches.
Change-Id: I9a2bf596cf47bffad5d6a5fd0da3c571fa266013
The wheel-mirror workers share a common set of hiera keys, and so
keeping them in a group together reduces significant duplication in
our hiera files.
Change-Id: I67b717943eb19404c87b2a3b571f681a0a15b7b4
This is a machine that will drive centralized mirror updates for
mirrors that are in AFS.
Change-Id: I33eed90c21a2e57a26349be83ee69e968b18d2f8
Depends-On: I56bb3ce2237be1179724f7a2bdcf9d5b04bdecd2
Depends-On: I704c8bf395d83adea0f9a0db6fd3d3814ad7660a
When we create a new server and install puppet, we also install pip.
That's awesome, except when pip >=8 breaks us. For now, pin it.
Change-Id: Id12f866f577c3ca2405a6049084c3cb0af82fde5
We already have a dynamic system for managing static group management.
Use it for the disabled group so that the rules for managing the members
are not different.
Also, update the disabled list to match reality.
Also, Update docs because hosts are no longer groups
The upstream OpenStack Inventory in Ansible was fixed to no longer
return each cloud host as its own group unless there are duplicates for
the host in question. This means it's no longer the right thing to do
to put hosts into disabled:children - disabled is just fine.
Change-Id: I95c83ed64801db15ad99a14547895f3520356f99
We have ansible inventory reporting hosts as proper hosts not as groups
now, so we no longer need to list them as children. In fact, this does
not work.
Change-Id: Idd72824023ecaef41bb98fb3a3092122d4397895
The openstack_project::server class declares things we want to be
true of our long-lived servers, so use it instead of
openstack_project::template (which it then indirectly uses).
Change-Id: Ie4bc41ec0b333ad9151e5df458e2762835672e6c
At long last, the day of reckoning is here. Run puppet apply and then
copy the log files back and post them to puppetdb.
Change-Id: I919fea64df0fbb8681e91ac9425b4c43760bb3dd
We don't need to rsync to ourselves. Best case it's a no-op. Worst case
something weird happens and we overwrite ourselves while running.
Change-Id: I890ea487d7a6129b7477b6d17b6a7e3c1904bade
When we do it as a second playbook, the failure to copy updated code
cannot prevent puppet from running.
Change-Id: I94b06988a20da4c0c2cf492485997ec49c3dca13
Depends-On: I22b7a21778d514a0a1ab04a76f03fdc9c58a05b3
