Create virtualenvs in series to avoid cache race
virtualenv 20.0.24 creates ~/.local/share/virtualenv with the seed packages needed for making virtualenvs per-python version. Creating empty virtualenvs is quick, so run those in sequence to avoid race possibilities. Then, we can still run the installs into the virtualenvs in parallel. We also fix a bug in the console stream functional jobs and install pip with the use of ensure-pip. This is necessary because the virtualenv fix runs the stream functional jobs and the update to the stream functional jobs relies on working docker images. Change-Id: I3dec251d19dd7b3807848a54e6a20a8e89d30a4e
This commit is contained in:
parent
7c0bf04f55
commit
1f7c6fef9c
|
@ -30,8 +30,10 @@ RUN cd /tmp/src/web && yarn install -d && yarn build
|
|||
RUN assemble
|
||||
|
||||
# The wheel install method doesn't run the setup hooks as the source based
|
||||
# installations do so we have to call zuul-manage-ansible here.
|
||||
RUN /output/install-from-bindep && zuul-manage-ansible
|
||||
# installations do so we have to call zuul-manage-ansible here. Remove
|
||||
# /root/.local/share/virtualenv after because it adds wheels into /root
|
||||
# that we don't need after the install step so are a waste of space.
|
||||
RUN /output/install-from-bindep && zuul-manage-ansible && rm -rf /root/.local/share/virtualenv
|
||||
|
||||
RUN mkdir /tmp/openshift-install \
|
||||
&& curl -L $OPENSHIFT_URL -o /tmp/openshift-install/openshift-client.tgz \
|
||||
|
|
|
@ -1,6 +1,8 @@
|
|||
- hosts: controller
|
||||
roles:
|
||||
|
||||
- role: ensure-pip
|
||||
|
||||
- role: bindep
|
||||
bindep_profile: test
|
||||
bindep_dir: src/opendev.org/zuul/zuul
|
||||
|
|
|
@ -49,8 +49,6 @@ class ManagedAnsible:
|
|||
self.install_root = self._ansible_roots[-1]
|
||||
|
||||
def ensure_ansible(self, upgrade=False):
|
||||
self._ensure_venv()
|
||||
|
||||
self.log.info('Installing ansible %s, requirements: %s, '
|
||||
'extra packages: %s',
|
||||
self.version, self._requirements, self.extra_packages)
|
||||
|
@ -76,7 +74,7 @@ class ManagedAnsible:
|
|||
p.stderr.decode()))
|
||||
self.log.debug('Successfully installed packages %s', requirements)
|
||||
|
||||
def _ensure_venv(self):
|
||||
def ensure_venv(self):
|
||||
if self.python_path:
|
||||
self.log.debug(
|
||||
'Virtual environment %s already existing', self.venv_path)
|
||||
|
@ -192,6 +190,13 @@ class AnsibleManager:
|
|||
self.default_version = default_version
|
||||
|
||||
def install(self, upgrade=False):
|
||||
# virtualenv sets up a shared directory of pip seed packages per
|
||||
# python version. If we run virtualenv in parallel we can have one
|
||||
# create the dirs but not be finished filling them with content, a
|
||||
# second notice the dir is there so it just goes on with what it's
|
||||
# done, and thus races leaving us with virtualenvs minus pip.
|
||||
for a in self._supported_versions.values():
|
||||
a.ensure_venv()
|
||||
# Note: With higher number of threads pip seems to have some race
|
||||
# leading to occasional failures during setup of all ansible
|
||||
# environments. Thus we limit the number of workers to reduce the risk
|
||||
|
|
Loading…
Reference in New Issue