Support TEMPEST_BRANCH with tag name & cap it to 23.0.0

TEMPEST_BRANCH which is mostly set as master so
that Tempest master is run to test the env. With
stable branch going to EM state and Tempest master
might not work due to incompatibility of code or
requirements. In that case we pin the Tempest so that
older Tempest can be used for their testing.

Till now for ocata, pike and, queens we used the gerrit style
ref to pin the Tempest which is not preferred way. We should be
able to use the tag name on TEMPEST_BRANCH.

This commit explicitly checkout the tag set in TEMPEST_BRANCH
as git_clone does not checkout the tag directly until RECLONE
is true or tempest dir does not exist.

After this stable branch or job can set the tag directly with name.
For exmaple: TEMPEST_BRANCH=23.0.0.

Tempest master is dropping the py3.5 support[1] and last
tag work well with py3.5 is 23.0.0. Stable/rocky jobs use
xenail node which has py3.5 version.

We need to pin Tempest 23.0.0 in this patch itself to
unblock the gate.

Depends-On: https://review.opendev.org/#/c/705870/
Depends-On: https://review.opendev.org/#/c/706425/

Related--Bug: 1861308
[1] https://review.opendev.org/#/c/704840/

Change-Id: Ic777e4b56c4932dde135ac909cb5c6f4a7d5cc78

lib/tempest

@@ -589,9 +589,8 @@ function configure_tempest {
         tox -revenv-tempest --notest
     fi
 
-    # The requirements might be on a different branch, while tempest needs master requirements.
-    (cd $REQUIREMENTS_DIR && git show origin/master:upper-constraints.txt) > u-c-m.txt
-    tox -evenv-tempest -- pip install -c u-c-m.txt -r requirements.txt
+    # NOTE(gmann): Use branch constraint because Tempest is pinned to the branch release instead of using master.
+    tox -evenv-tempest -- pip install -c $REQUIREMENTS_DIR/upper-constraints.txt -r requirements.txt
 
     # Auth:
     iniset $TEMPEST_CONFIG auth tempest_roles "Member"
@@ -692,6 +691,17 @@ function install_tempest {
     git_clone $TEMPEST_REPO $TEMPEST_DIR $TEMPEST_BRANCH
     pip_install 'tox!=2.8.0'
     pushd $TEMPEST_DIR
+    # NOTE(gmann): checkout the TEMPEST_BRANCH in case TEMPEST_BRANCH
+    # is tag name not master. git_clone would not checkout tag because
+    # TEMPEST_DIR already exist until RECLONE is true.
+    git checkout $TEMPEST_BRANCH
+
+    # NOTE(gmann): Use branch constraint because Tempest is pinned to the branch release
+    # instead of using master. We need to export it via env var UPPER_CONSTRAINTS_FILE
+    # so that initial creation of tempest tox use stable branch constraint
+    # instead of master constraint as defined in tempest/tox.ini
+    export UPPER_CONSTRAINTS_FILE=$REQUIREMENTS_DIR/upper-constraints.txt
+
     tox -r --notest -efull
     local ver
     ver=$(tox -evenv-tempest -- python --version)
@@ -712,9 +722,9 @@ function install_tempest {
 function install_tempest_plugins {
     pushd $TEMPEST_DIR
     if [[ $TEMPEST_PLUGINS != 0 ]] ; then
-        # The requirements might be on a different branch, while tempest & tempest plugins needs master requirements.
-        (cd $REQUIREMENTS_DIR && git show origin/master:upper-constraints.txt) > u-c-m.txt
-        tox -evenv-tempest -- pip install -c u-c-m.txt $TEMPEST_PLUGINS
+        # NOTE(gmann): Use branch constraint because Tempest is pinned to the branch release
+        # instead of using master.
+        tox -evenv-tempest -- pip install -c $REQUIREMENTS_DIR/upper-constraints.txt $TEMPEST_PLUGINS
         echo "Checking installed Tempest plugins:"
         tox -evenv-tempest -- tempest list-plugins
     fi

stackrc

@@ -310,8 +310,10 @@ REQUIREMENTS_BRANCH=${REQUIREMENTS_BRANCH:-$TARGET_BRANCH}
 
 # Tempest test suite
 TEMPEST_REPO=${TEMPEST_REPO:-${GIT_BASE}/openstack/tempest.git}
-TEMPEST_BRANCH=${TEMPEST_BRANCH:-$BRANCHLESS_TARGET_BRANCH}
-
+# Use Tempest tag 23.0.0) which is Rocky supported version for
+# Xenial jobs running on py3.5.
+# https://docs.openstack.org/releasenotes/tempest/v23.0.0.html
+TEMPEST_BRANCH=${TEMPEST_BRANCH:-23.0.0}
 
 ##############
 #

tests/test_refs.sh

@@ -15,7 +15,7 @@
 
 echo "Ensuring we don't have crazy refs"
 
-REFS=`grep BRANCH stackrc | grep -v 'TARGET_BRANCH' | grep -v 'NOVNC_BRANCH'`
+REFS=`grep BRANCH stackrc | grep -v 'TARGET_BRANCH' | grep -v 'NOVNC_BRANCH' | grep -v TEMPEST_BRANCH`
 rc=$?
 if [[ $rc -eq 0 ]]; then
     echo "Branch defaults must be one of the *TARGET_BRANCH values. Found:"