12 Commits

Author SHA1 Message Date
Tobias Henkel
e35fcde591
Upgrade pip in python-builder and base
There was an issue in pip that prevented correctly caching locally
built wheels [1]. This has been fixed in recent pip versions so
upgrade pip in both images so image caching works correctly. This is
needed to unbreak nodepool images that fail to install the locally
built netifaces package.

[1] https://github.com/pypa/pip/issues/6852

Change-Id: Ibbe12bcc53253a80d0bafa3d09a20c49a3a2b784
2020-01-22 15:47:58 +01:00
Monty Taylor
5de73f6c36 Use explicit image paths
To make it clear that docker hub is but one of many possible registries,
update our usage of FROM and image: lines to include docker.io in the
path.

There are a few other FROM lines for the gitea images which are handled
in a separate stack.

Change-Id: I6fafd5f659ad19de6951574afc9a6b6a4cf184df
2019-12-17 08:13:34 -05:00
Monty Taylor
5e12438e0d Split bindep and wheel invocations
We need to run bindep before installing git, because otherwise if
a project needs git in its bindep, it won't show up because it'll
be on the build host.

Split the function in two and call them before and after the git
installation.

Change-Id: I316b1bc643eb9293500b31e676361eec7060701d
2019-12-03 11:13:56 -06:00
Ian Wienand
bd66a7cb1b python-builder: install sibling packages
In the dependent change, the docker roles will add sibling packages to
the .zuul-siblings directory of the checked-out source.

Refactor the "assemble" script to handle this.  Essentially we build
the wheel for "." and then iterate over ZUUL_SIBLINGS subdirectories
(set in a --build-arg by the role in dependent change) to also build
the sibling packages.  Note we concatenate the bindep.txt files, so
that we end up with the complete package list required by the main
code and its dependencies.

"install-from-bindep" now installs all the wheels, using --force to
make sure we re-install the speculatively built packages.

This means that a single Dockerfile works under Zuul when
ZUUL_SIBLINGS is set, pointing to Zuul's checkouts; but it also works
stand-alone -- in this case ZUUL_SIBLINGS is empty and we just install
from upstream as usual.

Depends-On: https://review.opendev.org/696987
Change-Id: I4943ae723b06b0ad808e7c7f20788109e21aa8bf
2019-12-03 20:51:32 +11:00
Monty Taylor
4b4dcf37b1 Pin python images to 3.7
rehash may have an issue with python 3.8 internals.

Change-Id: I1a9c6150d74f6a646446b65a7456c73dc104fd9e
2019-10-22 03:37:23 +09:00
David Shrewsbury
0ae912cc04 No-op change to rebuild python-builder
Apparently, an update to the python-base image caused a new image
to be uploaded to dockerhub. Because python-builder was not updated,
it is running a slightly older base image which we believe is causing
issues with python wheel packaging due to possible different python
versions.

Change-Id: I38948882131e30c7358a970292621b0280b75aac
2019-10-21 11:22:50 -04:00
Paul Belanger
2c4428aa57 Use --no-cache-dir with pip
This helps to reduce the image size but not creating /root/.cache folder
for pip related files.

Change-Id: I1162d7d6fd2e4c7dd9cde44c964aec610a4dd6d3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-04-28 16:13:07 -04:00
Paul Belanger
736695553e Clean up apt cache
By removing /var/lib/apt/lists it reduces the image size.

Change-Id: Icbe118a2725700b9e5e8da97d062161c5b9a5d2d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-04-28 16:04:18 -04:00
Monty Taylor
9adc5ce8fe Split python-base into its own Dockerfile
There's no real need to tie these together into a multi-stage
Dockerfile as they don't really share anything. Split them.

Change-Id: Ifd7ccadcd8048eeb57797d60356aec2f9f0d2c80
Depends-On: https://review.openstack.org/641805
2019-03-08 15:49:49 -08:00
Monty Taylor
b78d2f8ed6
Reorder the copy in the python-builder builder image
It really doesn't actually matter, but if we do the apt-get install
before the COPY, it can increase the caching efficiency by a billionth
of a percent.

Depends-On: https://review.openstack.org/632528
Change-Id: I875812c956af1c4739e0986d267a47652bb5e6f3
2019-01-22 19:14:04 +00:00
Monty Taylor
d3c6bbc7af
Install python requirements directly
There are cases when an issue has been fixed upstream in a
dependency but not yet released and it is desirable to install
that version in a built image. pbr does not support such urls
in requirements files because releasing software that depends
on unreleased software is quite dangerous.

Update assemble and install-from-bindep to install -r requirements.txt
before installing the actual software so that image builds with
such requirements lines will work.

Change-Id: Ie17a66bdc4ea0609ce4e66051f659cbd829fdd5d
2019-01-21 20:50:43 +00:00
Monty Taylor
0e1cd6ee85
Add python-builder docker image
This is a builder image intended to make it easy to build
images for projects that use pbr and bindep.

Also, build a python-base image for use with python-builder

pbrx installed dumb-init in a base image and used it as an
entrypoint for the image. This is a good pattern that we should
continue. Make a python-base image that can be used in the FROM
line of consuming final images.

Change-Id: I8da39e680e9c5ca0d8f603a2645e83fc3be87a8b
2019-01-21 20:06:42 +00:00