opendev/system-config
Code Issues Proposed changes
a62a7ec2ae
system-config/doc/source/bandersnatch.rst
Jeremy Stanley 5ce07c5afe Add vexxhost mirror to documentation and cacti 
Add the new mirror.ca-ymq-1.vexxhost.openstack.org host to our
bandersnatch documentation and also correct a typo in another entry
there. Further, add it to the cacti monitoring list and remove the
entry for the dead HPCloud mirror while we're in the neighborhood.

Change-Id: I4be0dfc0099336773ea94bd940e55c4b0f7d036f
2016-03-02 16:41:02 +00:00

2.1 KiB
Raw Blame History

title

Bandersnatch

Bandersnatch

A pypi mirror tool

At a Glance

Hosts
Puppet
  • modules/openstack_project/manifests/static.pp
Projects
Documentation
Bugs

Overview

Bandersnatch is a tool we run on the static.openstack.org host to build a complete mirror of pypi.python.org. Cron execs bandersnatch on an interval with logs going to /var/log/bandersnatch.

Stale Packages

There is an issue with pypi.python.org syncing to its CDN occasionally resulting in stale package artifacts. You will notice this in the bandersnatch logs as:

2014-07-11 01:30:04,592 INFO: Syncing package: python-novaclient (serial 1154164)
2014-07-11 01:30:04,592 DEBUG: Getting /pypi/python-novaclient/json (serial 1154164)
2014-07-11 01:30:04,599 DEBUG: Expected PyPI serial 1154164 for request https://pypi.python.org/pypi/python-novaclient/json but got 1154163
2014-07-11 01:30:04,599 ERROR: Stale serial for package python-novaclient
2014-07-11 01:30:04,599 ERROR: Stale serial for python-novaclient (1154164) not updating. Giving up.

The fix for this is to issue a PURGE against the url specified above:

curl -X PURGE https://pypi.python.org/pypi/python-novaclient/json

The next run of bandersnatch will sync the package. Note this PURGE step should be performed automatically by our bandersnatch wrapper script, but can be performed by hand safely if necessary.