Move 'check-cherry-picks' test to gate, n-v check

This currently runs in the 'check' pipeline, as part of the pep8 job,
which causes otherwise perfectly valid backports to report as failing
CI. There's no reason a stable core shouldn't be encouraged to review
these patches: we simply want to prevent them *merging* before their
parent(s). Resolve this conflict by moving the check to separate voting
job in the 'gate' pipeline as well as a non-voting job in the 'check'
pipeline to catch more obvious issues.

NOTE(lyarwood): Conflict as I672904e9bfb45a66a82331063c7d49c4bc0439df
isn't present on stable/victoria.

Conflicts:
  .zuul.yaml

Change-Id: Id3e4452883f6a3cf44ff58b39ded82e882e28c23
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
(cherry picked from commit 98b01c9a59)
(cherry picked from commit fef0305abe)

.zuul.yaml

@@ -56,6 +56,17 @@
       bindep_profile: test py38
     timeout: 3600
 
+- job:
+    name: nova-tox-validate-backport
+    parent: openstack-tox
+    description: |
+      Determine whether a backport is ready to be merged by checking whether it
+      has already been merged to master or more recent stable branches.
+
+      Uses tox with the ``validate-backport`` environment.
+    vars:
+      tox_envlist: validate-backport
+
 - job:
     name: nova-live-migration
     parent: tempest-multinode-full-py3
@@ -424,6 +435,8 @@
         - nova-lvm
         - nova-multi-cell
         - nova-next
+        - nova-tox-validate-backport:
+            voting: false
         - nova-tox-functional-py38
         - tempest-integrated-compute:
             # NOTE(gmann): Policies changes do not need to run all the
@@ -462,6 +475,7 @@
         - nova-tox-functional-py38
         - nova-multi-cell
         - nova-next
+        - nova-tox-validate-backport
         - nova-ceph-multistore:
             irrelevant-files: *dsvm-irrelevant-files
         - neutron-tempest-linuxbridge:

tools/check-cherry-picks.sh

@@ -4,11 +4,6 @@
 # to verify that they're all on either master or stable/ branches
 #
 
-# Allow this script to be disabled by a simple env var
-if [ ${DISABLE_CHERRY_PICK_CHECK:-0} -eq 1 ]; then
-    exit 0
-fi
-
 commit_hash=""
 
 # Check if the patch is a merge patch by counting the number of parents.

tox.ini

@@ -49,8 +49,6 @@ commands =
 description =
   Run style checks.
 envdir = {toxworkdir}/shared
-passenv =
-  DISABLE_CHERRY_PICK_CHECK
 commands =
   {[testenv:mypy]commands}
   bash tools/flake8wrap.sh {posargs}
@@ -58,7 +56,6 @@ commands =
   bash -c "! find doc/ -type f -name *.json | xargs grep -U -n $'\r'"
   # Check that all included JSON files are valid JSON
   bash -c '! find doc/ -type f -name *.json | xargs -t -n1 python -m json.tool 2>&1 > /dev/null | grep -B1 -v ^python'
-  bash tools/check-cherry-picks.sh
 
 [testenv:fast8]
 description =
@@ -67,6 +64,15 @@ envdir = {toxworkdir}/shared
 commands =
   bash tools/flake8wrap.sh -HEAD
 
+[testenv:validate-backport]
+description =
+  Determine whether a backport is ready to be merged by checking whether it has
+  already been merged to master or more recent stable branches.
+deps =
+skipsdist = true
+commands =
+  bash tools/check-cherry-picks.sh
+
 [testenv:functional]
 description =
   Run functional tests using python3.