openstack/project-config
Code Issues Proposed changes

wheel-build : temporary pip with checksum output

Browse Source 
We are currently trying to debug persistent wheel corruption.  So far,
we have been completely unable to replicate the corrupt output outside
the periodic jobs.

A first thought was that it was due to AFS corruption due to multiple
writes (I4f8a2f2c6c8164e7ea207f8e4b286e06df0b13dd), however this does
not appear to be the case.

Experimentally manually running the builds, on infra nodes, under
python2 and python3, in parallel just as done here, does not replicate
the problem.

This patched version of pip will output the sha256 hash for the final
build output of each wheel.  The plan is to correlate that against any
corrupt file that appears.  If the corrupt hash matches any file
produced by pip, then we know the problem is inside pip (and we will
have the exact build situation that it occured in); if the corrupt
file does not match then we must have some sort of issue copying the
files or similar.

Change-Id: I81943ed459bf4e2c77cae42e50af5fc5979682b4
This commit is contained in:
Ian Wienand 2018-10-22 16:04:49 +11:00
parent ea1cc4fc45
commit 8b3532c562
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
roles/build-wheels/files/wheel-build.sh
View File

@ -24,6 +24,12 @@ for BRANCH in master $BRANCHES; do
rm -rf build_env rm -rf build_env
virtualenv -p $PYTHON_VERSION build_env virtualenv -p $PYTHON_VERSION build_env
# NOTE(ianw) 2018-10-22 This is a temporary hack to get some more
# info into the logs to debug corrupt wheels. We should see pip
# stamping sha256 hashes into the logs for each wheel, so we can
# see if the bad output is coming from pip, or somewhere else.
build_env/bin/pip install -e 'git+https://github.com/ianw/pip.git@path-and-hash#egg=pip'
# SHORT_BRANCH is just "master","newton","kilo" etc. because this # SHORT_BRANCH is just "master","newton","kilo" etc. because this
# keeps the output log hierarchy much simpler. # keeps the output log hierarchy much simpler.
SHORT_BRANCH=${BRANCH##origin/} SHORT_BRANCH=${BRANCH##origin/}