7 Commits

Author SHA1 Message Date
Ian Wienand
67f223b53a Partial revert "Ensure wheel exists for build-release-python"; move to ensure-pip
This partially reverts commit
3f961ce202d7d24e2944de09636b35cec9c13bf6.

This alternative installs wheel with the ensure-pip role instead of in
a separate role.  wheel is very closely linked with pip install
operations so this isn't a large overreach of the role.

I suggest this for several reasons; firstly the python-wheel role
doesn't try to install packages, so we end up with mixed system pip
and upstream versions of wheel most of the time.  This is the type of
thing that has proven problematic in the past.  It also installs via
pip --user; something we've already had problems with tox when for
various reasons roles want to run this as non-zuul user.  Using
ensure-pip we keep the packaged versions together.

[1] did try to install wheel with root, but during runtime which
didn't work due to sudo being revoked.  This should work for the
existing build-python-release job, because it already includes
ensure-pip in pre-run via playbooks/python/pre.yaml

I believe our conclusion on the ensure-* roles was that requiring
root/become: for installation is OK, but we should have a no-op path
if the tools are found.  This is consistent with that approach
(i.e. if you want wheel and can't do sudo, you should pre-install it
on your image using whatever you build that with).

This adds a check to the existing "is pip installed" check to also
check if wheel packages are available.  If not we trigger the install
path.

This revealed some issues with RedHat.yaml -- we can always install
Python 3 (packages available for CentOS 7) so remove that check, and
if Ansible is running under Python 2; ensure we install the
dependencies too (not only if it is forced).

Update the documentation to describe that it will enable support for
bdist_wheel, and add a basic sanity test that wheels are produced by
pip.  The existing build-python-release job is kept; although it is
modified to use the playbooks/python/pre.yaml playbook as the build
job does.

Change-Id: I2ab11bb45b6b2a49d54db39195228ab40141185c
[1] https://review.opendev.org/#/c/736001/5/roles/build-python-release/tasks/main.yaml
2020-06-18 12:51:56 +00:00
Sorin Sbarnea
3f961ce202 Ensure wheel exists for build-release-python
We need it to exist for building wheels. It should be
installed in a pre-playbook.

Change-Id: Ieaec3d21911decf11283e82fed97dd26b3a93ffa
2020-06-17 16:59:24 -05:00
vass
ef2f2eacb1 Remove install-* roles
The install-* roles has been renamed to ensure-* but have been kept
to give users time to update their repositories.

Tuesday the 26th of may is four weeks after we announced the deprecation
of the install-* roles and the date we should remove them completely.

Depends-On: https://review.opendev.org/719323
Depends-On: https://review.opendev.org/719324
Depends-On: https://review.opendev.org/719325
Depends-On: https://review.opendev.org/719326
Depends-On: https://review.opendev.org/719333
Depends-On: https://review.opendev.org/719334
Depends-On: https://review.opendev.org/719401
Depends-On: https://review.opendev.org/719402
Depends-On: https://review.opendev.org/719404

Change-Id: I1bc87cf9ef544c06e2f1daf3177bba9d577e332e
2020-05-26 09:34:25 +02:00
Ian Wienand
08bbd16c52 Add ensure-virtualenv
This is part of the efforts to remove pip-and-virtualenv from our base
images [1].

There are some users who specifically require the virtualenv command
(perhaps, like dib, they have some code that uses the activate_this.py
mechanisms it provides wich venv does not).

This installs the virtualenv package for the currently running
distribution.

One of the main maintenance issues of pip-and-virtualenv is that tried
to ensure that "virtualenv" created a Python 2 environment always by
default.  Now that we have Python 3 only distributions like current
Fedora, this is not something we can continue to do (even if we wanted
to, which we don't).

What owns virtualenv and what it produces in our heterogeneous
environment is messy, and I think the best we can do is document it as
done here.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html

Change-Id: I97d8bfb970ed2b5aaa02a0813899014c94994066
2020-04-28 06:09:54 +00:00
Ian Wienand
5068744eff ensure-pip: Add role
This role is intended to ensure that the `pip:` module, and jobs that
may wish to use `pip` from the shell, have the necessary requirements.

It is intended as a partial replacement for the pip-and-virtualenv
element in diskimage-builder, which currently pre-installs pip on our
infra CI images during image build.

We wish to remain broady compatible with this element, but not
replicate some of the more problematic areas of its implementation.

By default, this installs the system packages for pip and setuptools
(the latter being a requirement of the Ansible pip module, which
imports it directly, despite pip itself not requiring it).

In this role, we ensure the libraries for the currently running
ansible_python_interpreter version are installed.

There is provision to provide a flag to install the packages directly
from upstream via get-pip.py, although this is not recommended.

Story: #2007386
Task: #39309

Change-Id: Iac2d518a66caf1b801273225f75a0a748412903c
2020-04-14 10:20:33 +10:00
vass
c0b358fedb Rename install-if-python to ensure-if-python for consistency
The old role will be kept and include ensure-if-python for backwards-compatability.

Change-Id: Ief86799bcd91358d3ce8b6badda272a26ee04002
2020-04-04 15:41:29 -04:00
James E. Blair
4b3adfeea6 Organize documentation by subject area
This splits all of the current job and role documentation into
files by subject area so that jobs and roles are easier for users to
find.

This will require that any future new jobs or roles add a line to the
appropriate area of the documentation, since that can no longer be
done automatically.  A linter check is added to ensure that every
job and role continue to be documented.

After this refactor, we can begin to enhance the documentation pages
so that they include narrative documentation and subsections.

Change-Id: Ia6f0e89b57e3cb0d7d1745206384c946506d7ea0
2019-03-25 14:21:04 -07:00