translation updates: refactor Python 3.8 workarounds

I think we have quite a tangle of fixes on fixes in the translation
jobs, and I hope this refactor makes things a bit better.

The fundamental problem seems to be that python3.6 can't work with
master requirements.txt.  But we can not move some jobs off bionic
(python 3.6) because of java version contsraints.

The first thing that happened was these jobs got "ensure-python" with
"python_version: 3.8" set to bring in python3.8 on Bionic.  This role
was not bringing in python3.8-venv, so we had no way to create 3.8
virtual environments.  Thus ensure-pip ended up getting modified to
drag this package in (reverted with notes in
372f3af706c796331578ef81c2680e8c4c03c3b6).  This has a fairly easy
solution; let's just make ensure-python bring in the venv packages for
the udpated python -- see dependency
Ie3c03fea82bcec80a897f0905c15f35405a50396.

The git-review install here uses 'ensure_pip_virtualenv_command' to
install git-review in a virtualenv.  git-review still supports bionic,
and does not need to be installed in any speical way.  However, by
overriding the 'ensure_pip_virtualenv_command' for the entire job,
this unrelated install now grew a dependency on python3.8 and created
more confusion over why it was failing.

I believe what we actually want to do on Bionic is just run tox under
python3.8.  So this pulls out all the job-level defintions and
encapsulates them in a single block scoped to run on bionic.  It will
pull in python3.8 packages, and then install tox under python3.8 with
an override of the virtulenv command just for that step.  The jobs
just maintain their node definition and note on why it is required.

The long-term future of these jobs is obviously a question; but I
believe this keeps them running on bionic in as sane a way as possible
for now.

The second part of this is the python3.9 interpreter required on
Focal.  Setting "python_version: 3.9" in the job definition is a bit
confusing, because the job actually requires both Python 3.8 AND 3.9.
This variable is passed to ensure-python, and the naming is a bit
unfortunate because it is so broad (but we can't change that without
zuul-jobs updates).  Here I've pulled out the Python 3.9 install into
a separate focal-gated section in the pre.yaml that explains what is
going on (note that Python 3.8 is just the base python on focal, so
needs no extra installation).  I've also updated the description on
the job node with this info.  There's no need to install the tox
environment any differently on focal; it just uses the base system
python (3.8) which is fine.

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/850957
Change-Id: I547e5eb7faabcd9d3983ae85f3291b1a740bc77c

playbooks/proposal/pre.yaml

@@ -4,10 +4,6 @@
       include_role:
         name: ensure-pip
 
-    - name: Ensure python
-      include_role:
-        name: ensure-python
-
     - name: Install git-review
       pip:
         name: git-review
@@ -45,5 +41,44 @@
     - copy-release-tools-scripts
     - add-sshkey
     - bindep
-    - role: ensure-tox
-      ensure_global_symlinks: true
+
+  tasks:
+    # NOTE: Due to a confluence of problems between requirements
+    # versions, java versions, zanata clients, etc. translation
+    # proposal jobs have to run on bionic.  However, ensure-tox uses
+    # ensure_pip_virtualenv_command, which installs using default
+    # python on the system.  On bionic this is 3.6, which can not deal
+    # with master upper-constraints.txt.
+    #
+    # So we bring in the python 3.8 packages and override ensure-tox
+    # to use this to create the tox environment.
+    - name: Setup Bionic
+      when: ansible_distribution == 'Ubuntu' and ansible_release == 'bionic'
+      block:
+        - name: Ensure python
+          include_role:
+            name: ensure-python
+          vars:
+            python_version: 3.8
+
+        - name: Setup virtualenv version
+          set_fact:
+            _venv_command: '/usr/bin/python3.8 -m venv'
+
+    # The base proposal jobs require both python3.8 and python3.9.
+    # Bring in Python 3.9 for them here.
+    - name: Setup Focal
+      when: ansible_distribution == 'Ubuntu' and ansible_release == 'focal'
+      block:
+        - name: Ensure python
+          include_role:
+            name: ensure-python
+          vars:
+            python_version: 3.9
+
+    - name: Install tox
+      include_role:
+        name: ensure-tox
+      vars:
+        ensure_global_symlinks: true
+        ensure_pip_virtualenv_command: '{{ _venv_command | default(omit) }}'

zuul.d/jobs.yaml

@@ -824,18 +824,15 @@
       Sync content to other projects as a proposed change.
     run: playbooks/proposal/propose-updates.yaml
     protected: true
-    # Note(frickler): The nodeset is pinned because we require py3[89] to be
-    # available. Needs to be adapted in the pre-run playbook when changing.
+    # Note(frickler): The nodeset is pinned to Focal because we
+    # require both python 3.8 and 3.9 to be available.  This is setup
+    # in the pre.yaml playbook above; so if this nodeset changes we
+    # need to carefully consider the python versions any change
+    # distribution provides and modify the playbook/job accordingly.
     nodeset: ubuntu-focal
     secrets:
       - secret: proposal_ssh_key
         name: ssh_key
-    vars:
-      python_version: 3.9
-      ensure_pip_from_packages: True
-      ensure_pip_from_upstream_interpreters:
-        - python3.8
-        - python3.9
 
 - job:
     name: propose-update-constraints
@@ -1190,13 +1187,6 @@
     semaphores: translations
     protected: true
     vars:
-      # ubuntu-bionic don't have python 3.9
-      python_version: 3.8
-      # ubuntu-bionic default python is 3.6 and master upper-constraints are not compatible with it
-      ensure_pip_virtualenv_command: "/usr/bin/python3.8 -m venv"
-      # Only install python3.8 in ubuntu-bionic
-      ensure_pip_from_upstream_interpreters:
-        - python3.8
       constraints_file: '{{ ansible_user_dir }}/src/opendev.org/openstack/requirements/upper-constraints.txt'
     required-projects:
       - openstack/horizon
@@ -1221,13 +1211,6 @@
     run: playbooks/translation/propose-translation-update.yaml
     post-run: playbooks/translation/post.yaml
     vars:
-      # ubuntu-bionic don't have python 3.9
-      python_version: 3.8
-      # ubuntu-bionic default python is 3.6 and master upper-constraints are not compatible with it
-      ensure_pip_virtualenv_command: "/usr/bin/python3.8 -m venv"
-      # Only install python3.8 in ubuntu-bionic
-      ensure_pip_from_upstream_interpreters:
-        - python3.8
       constraints_file: '{{ ansible_user_dir }}/src/opendev.org/openstack/requirements/upper-constraints.txt'
     semaphores: translations
     protected: true