pip-and-virtualenv : deprecate source for CentOS 8, new variables
As described inline, deprecate the "source" install for CentOS 8. Overwriting the packaged tools has long been a pain-point in our images, and the best outcome is just not to play the game [1]. However, the landscape remains complicated. For example, RHEL/CentOS 8 introduces the separate "platform-python" binary, which seems like the right tool to install platform tools like "glean" (simple-init) with. However, platform-python doesn't have virtualenv (only the inbuilt venv). So that every element doesn't have to hard-code in workarounds for these various layouts, create two new variables DIB_PYTHON_PIP and DIB_PYTHON_VIRTUALENV to just "do the right thing". If you need is "install a pip package" or "create a virtualenv" this should work on all the platforms we support. If you know more specifically what you want (e.g. must be a python3 virtualenv) then nothing stops elements calling that directly (e.g. python3 -m virtualenv create); these are just helper wrappers for base elements that need to be broadly compatible. [1] http://lists.openstack.org/pipermail/openstack-infra/2019-September/006483.html Change-Id: Ia267a60eecfa8f4071dd477d86daebe07e9a7e38
This commit is contained in:
parent
5f3b7cd7b7
commit
18215274d8
@ -4,10 +4,6 @@ pip-and-virtualenv
|
||||
|
||||
This element installs pip and virtualenv in the image.
|
||||
|
||||
.. note:: This element setups and Python 2 and Python 3 environment.
|
||||
This means it will bring in python2 packages, so isn't
|
||||
appropriate if you want a python3 only environment.
|
||||
|
||||
Package install
|
||||
===============
|
||||
|
||||
@ -16,10 +12,12 @@ from distribution packages. In this case, ``pip`` and ``virtualenv``
|
||||
will be installed *only* for the python version identified by
|
||||
``dib-python`` (i.e. the default python for the platform).
|
||||
|
||||
Distribution packages have worked out name-spacing such that only
|
||||
Namespacing of the tools will be up to your distribution. Some
|
||||
distribution packages have worked out name-spacing such that only
|
||||
python2 or python3 owns common scripts like ``/usr/bin/pip`` (on most
|
||||
platforms, ``pip`` refers to python2 pip, and ``pip3`` refers to
|
||||
python3 pip, although some may choose the reverse).
|
||||
python3 pip, although some may choose the reverse). Other platforms
|
||||
have avoided making a decision and require explicit version suffixes.
|
||||
|
||||
To install pip and virtualenv from package::
|
||||
|
||||
@ -28,12 +26,28 @@ To install pip and virtualenv from package::
|
||||
Source install
|
||||
==============
|
||||
|
||||
Source install is the default. If the source installtype is used,
|
||||
``pip`` and ``virtualenv`` are installed from the latest upstream
|
||||
releases.
|
||||
.. note:: For source installs this element setups and Python 2 and
|
||||
Python 3 environments. This means it will bring in python2
|
||||
packages, so isn't appropriate if you want a python3 only
|
||||
environment.
|
||||
|
||||
Source installs from these tools are not name-spaced. It is
|
||||
inconsistent across platforms if the first or last install gets to own
|
||||
.. note:: Source install is considered deprecated for several reasons.
|
||||
Because it makes for a hetrogenous environment between
|
||||
distro packaged tools and upstream it means the final images
|
||||
create bespoke environments that make standarised testing
|
||||
difficult. The tricks used around holding packages to
|
||||
overwrite them cause difficulty for users of images. This
|
||||
also brings in Python 2 unconditonally, something not wanted
|
||||
on modern Python 3 only distributions.
|
||||
|
||||
Source install is the default on most platforms for historical
|
||||
purposes. The current exception(s) are RHEL8 and CentOS 8.
|
||||
|
||||
If the source installtype is used, ``pip`` and ``virtualenv`` are
|
||||
installed from the latest upstream releases.
|
||||
|
||||
Source installs from upstream releases are not name-spaced. It is
|
||||
inconsistent across platforms if the first or last install will own
|
||||
common scripts like ``/usr/bin/pip`` and ``virtualenv``.
|
||||
|
||||
To avoid inconsistency, we firstly install the packaged python 2
|
||||
@ -54,8 +68,25 @@ The system will be left in the following state:
|
||||
Source install is supported on limited platforms. See the code, but
|
||||
this includes Ubuntu and RedHat platforms.
|
||||
|
||||
Using the tools
|
||||
===============
|
||||
Environment Variables
|
||||
=====================
|
||||
|
||||
To simplify the common-case of "install a package" or "create a
|
||||
virtualenv" with the default system Python, the following variables
|
||||
are exported by this element:
|
||||
|
||||
* ``DIB_PYTHON_PIP``
|
||||
* ``DIB_PYTHON_VIRTUALENV``
|
||||
|
||||
This will create/install using the ``dib-python`` version for the
|
||||
platform (i.e. python2 for older distros, python3 for modern distros).
|
||||
Note that on Python 3 platforms it will use the inbuilt ``venv``
|
||||
(rather than the ``virtualenv`` package -- if you absolutely need
|
||||
features only ``virtualenv`` provides you should call it directly in
|
||||
your element; see below).
|
||||
|
||||
Explicit use of the tools
|
||||
=========================
|
||||
|
||||
Due to the essentially unsolvable problem of "who owns the script", it
|
||||
is recommended to *not* call ``pip`` or ``virtualenv`` directly. You
|
||||
|
@ -0,0 +1,33 @@
|
||||
# Due to the many historical problems mixing python2/3 versions and
|
||||
# upgrading packaged system pip/setuptools/virtualenv binaries with
|
||||
# upstream non-packaged versions, we wish to avoid this completely on
|
||||
# modern distros.
|
||||
if [[ $DISTRO_NAME =~ (centos|rhel) && $DIB_RELEASE -ge 8 ]]; then
|
||||
export DIB_INSTALLTYPE_pip_and_virtualenv=${DIB_INSTALLTYPE_pip_and_virtualenv:-package}
|
||||
|
||||
if [[ ${DIB_INSTALLTYPE_pip_and_virtualenv} == "source" ]]; then
|
||||
echo "*** pip-and-virtualenv does not support 'source' install for $DISTRO_NAME/$DIB_RELEASE"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# The default variables setup below are only useful during the phases
|
||||
# that dib-python exists
|
||||
if [[ ! -e /usr/local/bin/dib-python ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# NOTE(ianw): you don't want to call "dib-python -m pip" because that
|
||||
# can leave behind interpreters #!/usr/local/bin/dib-python in
|
||||
# scripts. De-reference the link
|
||||
_dib_python_path=$(readlink /usr/local/bin/dib-python)
|
||||
export DIB_PYTHON_PIP="$_dib_python_path -m pip"
|
||||
# We make an opinionated, but simplifying decision here that on
|
||||
# Python3 platforms, just use venv. There are some corner cases that
|
||||
# the external "virtualenv" package still handles better, but for most
|
||||
# purposes "venv" should be fine.
|
||||
if [[ $DIB_PYTHON_VERSION == 3 ]]; then
|
||||
export DIB_PYTHON_VIRTUALENV="$_dib_python_path -m venv"
|
||||
else
|
||||
export DIB_PYTHON_VIRTUALENV="$_dib_python_path -m virtualenv"
|
||||
fi
|
@ -0,0 +1,10 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
The ``pip-and-virtualenv`` element has added variables ``DIB_PYTHON_PIP``
|
||||
and ``DIB_PYTHON_VIRTUALENV`` to make it more straight forward for other
|
||||
elements to install packages or create virtualenv environment
|
||||
deprecations:
|
||||
- |
|
||||
Source install for ``pip-and-virtualenv`` is deprecated for RHEL
|
||||
8/CentOS 8
|
Loading…
Reference in New Issue
Block a user