opendev/system-config
Code Issues Proposed changes

Use the build tool in assemble instead of setup.py

Browse Source 
Our python builder's assemble script was used as guidance for building
wheels to workaround git's new permission checking behavior. When it was
called out that setup.py is deprecated and using `build` would be better
we decided that using setup.py was fine since all our images currently
use it.

Now that we've cleaned up our old buster images and python3.7 it is the
perfect time to update to modern python build tooling as well. Update
the assembel script to use `build` instead of setup.py.

As far as I can tell `build` maintains the old setup.py behavior of
emitting artifacts to the output directory separate of any dependency
packages that may have been pulled down to build the package. As the old
comment indicates this behavior is desireable.

It may also be worth testing this update against Zuul and other
relatively complicated python packages to ensure we haven't changed
behavior in some unexpected way.

Change-Id: Ib2a7335219842413e507c0593ceed187819d83b3
This commit is contained in:
Clark Boylan
2022-04-20 08:52:34 -07:00
parent 90effa2af0
commit 80ec51e907
1 changed files with 7 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docker/python-builder/scripts/assemble
View File

@@ -44,14 +44,15 @@ function install_wheels {
# Build a wheel so that we have an install target. # Build a wheel so that we have an install target.
# pip install . in the container context with the mounted # pip install . in the container context with the mounted
# source dir gets ... exciting. # source dir gets ... exciting.
# We run sdist first to trigger code generation steps such # The `build` tool builds an sdist first then a wheel from
# as are found in zuul, since the sequencing otherwise # that sdist which is exactly what we want. This triggers code
# happens in a way that makes wheel content copying unhappy. # generation steps such as are found in zuul, since the sequencing
# pip wheel isn't used here because it puts all of the output # otherwise happens in a way that makes wheel content copying unhappy.
# `pip wheel` isn't used here because it puts all of the output
# in the output dir and not the wheel cache, so it's not # in the output dir and not the wheel cache, so it's not
# possible to tell what is the wheel for the project and # possible to tell what is the wheel for the project and
# what is the wheel cache. # what is the wheel cache.
python3 setup.py sdist bdist_wheel -d /output/wheels /tmp/venv/bin/python3 -m build -o /output/wheels ./
# Install everything so that the wheel cache is populated with # Install everything so that the wheel cache is populated with
# transitive depends. If a requirements.txt file exists, install # transitive depends. If a requirements.txt file exists, install
@@ -88,7 +89,7 @@ done
# Use a clean virtualenv for install steps to prevent things from the # Use a clean virtualenv for install steps to prevent things from the
# current environment making us not build a wheel. # current environment making us not build a wheel.
python3 -m venv /tmp/venv python3 -m venv /tmp/venv
/tmp/venv/bin/pip install -U pip wheel /tmp/venv/bin/pip install -U pip wheel build
# If there is an upper-constraints.txt file in the source tree, # If there is an upper-constraints.txt file in the source tree,
# use it in the pip commands. # use it in the pip commands.