Commit Graph

10 Commits

Author SHA1 Message Date
Clark Boylan
e9854d8bdc Stop using jemalloc in python base image
Initial testing seems to show that jemalloc is a probable source of our
memory leaks with zuul-scheduler and zuul-web under python3.7 and
python3.8. While this was desireable under python3.5 with zuul-executor
it doesn't seem like we want to enable this universally. For this reason
remove jemalloc from the base image. It can be added into images in
specific places if desireable but isn't a good default.

Change-Id: I671eef4ef27fdde161b52cf0a4187fac10ac8c4e
2020-05-04 16:31:36 -07:00
Monty Taylor
650392c700 Start making 3.8 python images
base and builder are currently pinned to 3.7, which makes it hard
to consider upgrading to python-3.8. To help with that, make
python-3.7 and python-3.8 images, but point latest as 3.7 for the
time being. Then add version-specific provides and FROM lines
so that we can start being deliberate and clear about our version
choice.

Change-Id: Ibf1d846d5c4a005547785124567ce2900e272a7a
2020-03-25 17:45:39 -05:00
Monty Taylor
643623ba8a Disable recommends in python-base and python-builder
We disable these on our VM images, so might as well make our
container images follow suit. Could make them a smidge smaller.
Also do it in the gerrit-base image.

Change-Id: Iba60cf5c7009d57c4910f9e4464aff9231598ad6
2020-03-23 16:06:43 -05:00
Monty Taylor
f875c7a73f Undo debian changes to openssl.cnf for python-base
Debian has decided to be a bit too aggressive in their openssl
defaults. Vendor in a copy of openssl.cnf with the changes in

https://salsa.debian.org/debian/openssl/-/blob/debian/unstable/debian/patches/Set-systemwide-default-settings-for-libssl-users.patch

reverted.

DEFAULT@SECLEVEL=2 breaks API interactions with Rackspace, but
it's not just them. https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_security_level.html
indicates that 1 is defaut and setting higher is problematic.

Change-Id: Ida7e9a557b873c14c0bf474450508f42fe7a5ad2
2020-03-16 10:58:37 -05:00
Tobias Henkel
e35fcde591
Upgrade pip in python-builder and base
There was an issue in pip that prevented correctly caching locally
built wheels [1]. This has been fixed in recent pip versions so
upgrade pip in both images so image caching works correctly. This is
needed to unbreak nodepool images that fail to install the locally
built netifaces package.

[1] https://github.com/pypa/pip/issues/6852

Change-Id: Ibbe12bcc53253a80d0bafa3d09a20c49a3a2b784
2020-01-22 15:47:58 +01:00
Monty Taylor
5de73f6c36 Use explicit image paths
To make it clear that docker hub is but one of many possible registries,
update our usage of FROM and image: lines to include docker.io in the
path.

There are a few other FROM lines for the gitea images which are handled
in a separate stack.

Change-Id: I6fafd5f659ad19de6951574afc9a6b6a4cf184df
2019-12-17 08:13:34 -05:00
Monty Taylor
4b4dcf37b1 Pin python images to 3.7
rehash may have an issue with python 3.8 internals.

Change-Id: I1a9c6150d74f6a646446b65a7456c73dc104fd9e
2019-10-22 03:37:23 +09:00
Monty Taylor
9ab25e89a9 Several updates because the world is a dark place
A few things have changed and we need to fix them in one go.

Use mirror for installing docker for buildset-registry

While, we need to make this more systemic, that's hanging off of the
mirror rework. For now, since we know all of these jobs are debian
based, just set the mirror location.

Replace use of zuul cloner with git clones

You can never be a prophet in your own hometown. This is now broken
because of the git cache rework, so just replace it.

Update libjemalloc library

python:slim is based on buster now, which has libjemalloc2 not
libjemalloc1.

Remove gerrit repo remote for submodules

A recent change to the base jobs to use prepare-workspace-git
broke the gerrit image builds by actually having the origin
remote by /dev/null as intended. This breaks submodules because
for a few of them where we don't have matching stable branches
the submodule relative path behavior is actually exactly what
we want.

Since we don't care about the remote otherwise, remove the
origin remote before doing the submodule update --init so that
the submodule will clone the refs from the zuul prepared repo.

Change-Id: Ieb5b6bc8711fe971ed3445c7c267306ac4616464
2019-10-19 07:51:29 +09:00
Monty Taylor
d7c0be958d Run python with jemalloc in containers
jemalloc is apparently good with python memory according to

  https://zapier.com/engineering/celery-python-jemalloc/

Also, according to these graphs from Tobias:

  https://paste.pics/581cc286226407ab0be400b94951a7d9

it helps with the apparent memory leak seen in python3.6
and python3.7 with zuul.

There doesn't seem to be any downside to it, so install it
and enable it.

Change-Id: I354bc35e5fa1aee90e1c8b6918c5a70dafd4f990
2019-03-08 15:49:51 -08:00
Monty Taylor
9adc5ce8fe Split python-base into its own Dockerfile
There's no real need to tie these together into a multi-stage
Dockerfile as they don't really share anything. Split them.

Change-Id: Ifd7ccadcd8048eeb57797d60356aec2f9f0d2c80
Depends-On: https://review.openstack.org/641805
2019-03-08 15:49:49 -08:00