opendev/system-config
Code Issues Proposed changes
22d4ca6227
system-config/doc/source/bandersnatch.rst
James E. Blair 6d9c4eee3a Rename sjc1 mirror to regionone 
Current pypi mirror selection uses the actual cloud region from
nodepool, so the attempt to hide the default region name used by
bluebox does not work here.  In the future, we will alter the
format of pypi mirrors to something like
mirror.<region>.<cloud>.openstack.org, but that's a bigger job.
Today, just go with regionone so we can start using bluebox.

Change-Id: I575d3982a2c335054511492b1e664cfd17ab392a
2015-11-11 09:27:23 -08:00

1.9 KiB
Raw Blame History

title

Bandersnatch

Bandersnatch

A pypi mirror tool

At a Glance

Hosts
Puppet
  • modules/openstack_project/manifests/static.pp
Projects
Documentation
Bugs

Overview

Bandersnatch is a tool we run on the static.openstack.org host to build a complete mirror of pypi.python.org. Cron execs bandersnatch on an interval with logs going to /var/log/bandersnatch.

Stale Packages

There is an issue with pypi.python.org syncing to its CDN occasionally resulting in stale package artifacts. You will notice this in the bandersnatch logs as:

2014-07-11 01:30:04,592 INFO: Syncing package: python-novaclient (serial 1154164)
2014-07-11 01:30:04,592 DEBUG: Getting /pypi/python-novaclient/json (serial 1154164)
2014-07-11 01:30:04,599 DEBUG: Expected PyPI serial 1154164 for request https://pypi.python.org/pypi/python-novaclient/json but got 1154163
2014-07-11 01:30:04,599 ERROR: Stale serial for package python-novaclient
2014-07-11 01:30:04,599 ERROR: Stale serial for python-novaclient (1154164) not updating. Giving up.

The fix for this is to issue a PURGE against the url specified above:

curl -X PURGE https://pypi.python.org/pypi/python-novaclient/json

The next run of bandersnatch will sync the package. Note this PURGE step should be performed automatically by our bandersnatch wrapper script, but can be performed by hand safely if necessary.