[stable-only] Cap bandit to v1.6.2 and fix constraints

This patch has multiple fixes to unblock gate.

1. bandit

We don't have bandit requirements in upper-constraints, so we need
to cap it in stable branches manually to work with Python 2.7, as
bandit 1.6.3 [1] release has dropped support for py2 [2] but the
release is faulty and pip still picks it up for py2 [3][4].

2. pip resolver

With the new pip resolver the following issues needed to be fixed:
* SQLAlchemy-Utils constraint conflicts with rocky's upper constraints
* bump lower constraint of cryptography
* bump lower constraint of oslo.messaging
* bump lower constraint of oslo.utils

3. lower-constraints

In lower-constraints job lxml requires libxml2-dev and libxslt1-dev
packages, so bindep.txt is extended with them.

install_command needs to be added to lower-constraints tox target to
work properly.

4. cinder-tempest-dsvm-lvm-lio-barbican

Fix for overlapping CIDR issue in cinder-tempest-dsvm-lvm-lio-barbican
is the same as in patch I068cf1e9618d305b5a9383c283bfa0f120bfe905.

5. pin nodeset to xenial for requirements-check job

The default nodeset for zuul jobs is now ubuntu-focal, and
requirements-check job fails on focal, because the mysql it contains is
not compatible with the commands that is used in tools/test-setup.sh.
This patch pins the nodeset to ubuntu-xenial for requirements-check job
to make it pass.

[1] https://github.com/PyCQA/bandit/releases/tag/1.6.3
[2] https://github.com/PyCQA/bandit/pull/615
[3] https://github.com/PyCQA/bandit/issues/663
[4] https://github.com/PyCQA/bandit/issues/665

Change-Id: Ie597e778e3efa61bfd98eaaa92730b5050195e7a
(cherry picked from commit a512bfc233)

.zuul.yaml

@@ -26,6 +26,8 @@
               - ^doc/.*$
               - ^releasenotes/.*$
         - openstack-tox-lower-constraints
+        - requirements-check:
+            nodeset: ubuntu-xenial
         - cinder-tox-functional-py36
         - cinder-tox-py36
         - cinder-rally-task:
@@ -163,6 +165,8 @@
               - ^releasenotes/.*$
 
         - openstack-tox-lower-constraints
+        - requirements-check:
+            nodeset: ubuntu-xenial
         - legacy-grenade-dsvm-cinder-mn-sub-volbak:
             irrelevant-files:
               - ^(test-|)requirements.txt$

bindep.txt

@@ -18,6 +18,8 @@ libssl-dev [platform:dpkg]
 openssl-devel [platform:rpm !platform:suse]
 libopenssl-devel [platform:suse !platform:rpm]
 locales [platform:debian]
+libxml2-dev [platform:dpkg]
+libxslt1-dev [platform:dpkg]
 mariadb [platform:rpm]
 mariadb-server [platform:redhat]
 mariadb-devel [platform:redhat]

lower-constraints.txt

@@ -16,7 +16,7 @@ cliff==2.11.0
 cmd2==0.8.1
 contextlib2==0.5.5
 coverage==4.0
-cryptography==2.1
+cryptography==2.1.4
 cursive==0.2.1
 ddt==1.0.1
 debtcollector==1.19.0
@@ -75,7 +75,7 @@ oslo.context==2.19.2
 oslo.db==4.27.0
 oslo.i18n==3.15.3
 oslo.log==3.36.0
-oslo.messaging==5.29.0
+oslo.messaging==7.0.0
 oslo.middleware==3.31.0
 oslo.policy==1.30.0
 oslo.privsep==1.23.0
@@ -83,7 +83,7 @@ oslo.reports==1.18.0
 oslo.rootwrap==5.8.0
 oslo.serialization==2.18.0
 oslo.service==1.24.0
-oslo.utils==3.33.0
+oslo.utils==3.34.0
 oslo.versionedobjects==1.31.2
 oslo.vmware==2.17.0
 oslotest==3.2.0
@@ -140,7 +140,7 @@ sphinx-feature-classification==0.1.0
 sphinxcontrib-websupport==1.0.1
 sqlalchemy-migrate==0.11.0
 SQLAlchemy==1.0.10
-SQLAlchemy-Utils==0.36.1
+SQLAlchemy-Utils==0.33.3
 sqlparse==0.2.4
 statsd==3.2.2
 stestr==2.0.0
@@ -161,6 +161,6 @@ uritemplate==3.0.0
 urllib3==1.22
 vine==1.1.4
 voluptuous==0.11.1
-warlock==1.3.0
+warlock==1.2.0
 WebOb==1.7.1
 wrapt==1.10.11

playbooks/legacy/cinder-tempest-dsvm-lvm-lio-barbican/run.yaml

@@ -29,6 +29,11 @@
           set -x
           cat << 'EOF' >>"/tmp/dg-local.conf"
           [[local|localrc]]
+          # to avoid https://bugs.launchpad.net/neutron/+bug/1914037
+          # as we couldn't backport the fix to stein release
+          IPV6_PUBLIC_RANGE=2001:db8:0:10::/64
+          IPV6_PUBLIC_NETWORK_GATEWAY=2001:db8:0:10::2
+          IPV6_ROUTER_GW_IP=2001:db8:0:10::1
           CINDER_ISCSI_HELPER=lioadm
           CINDER_LVM_TYPE=thin
           enable_service barbican

requirements.txt

@@ -22,7 +22,7 @@ oslo.concurrency>=3.26.0 # Apache-2.0
 oslo.context>=2.19.2 # Apache-2.0
 oslo.db>=4.27.0 # Apache-2.0
 oslo.log>=3.36.0 # Apache-2.0
-oslo.messaging>=5.29.0 # Apache-2.0
+oslo.messaging>=7.0.0 # Apache-2.0
 oslo.middleware>=3.31.0 # Apache-2.0
 oslo.policy>=1.30.0 # Apache-2.0
 oslo.privsep>=1.23.0 # Apache-2.0
@@ -30,7 +30,7 @@ oslo.reports>=1.18.0 # Apache-2.0
 oslo.rootwrap>=5.8.0 # Apache-2.0
 oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0
 oslo.service!=1.28.1,>=1.24.0 # Apache-2.0
-oslo.utils>=3.33.0 # Apache-2.0
+oslo.utils>=3.34.0 # Apache-2.0
 oslo.versionedobjects>=1.31.2 # Apache-2.0
 osprofiler>=1.4.0 # Apache-2.0
 paramiko>=2.0.0 # LGPLv2.1+
@@ -65,5 +65,5 @@ os-win>=3.0.0 # Apache-2.0
 tooz>=1.58.0 # Apache-2.0
 google-api-python-client>=1.4.2 # Apache-2.0
 castellan>=0.16.0 # Apache-2.0
-cryptography>=2.1 # BSD/Apache-2.0
+cryptography>=2.1.4 # BSD/Apache-2.0
 cursive>=0.2.1 # Apache-2.0

test-requirements.txt

@@ -13,11 +13,11 @@ os-api-ref>=1.4.0 # Apache-2.0
 oslotest>=3.2.0 # Apache-2.0
 PyMySQL>=0.7.6 # MIT License
 psycopg2>=2.6.2 # LGPL/ZPL
-SQLAlchemy-Utils>=0.36.1 # BSD License
+SQLAlchemy-Utils>=0.33.3 # BSD License
 testtools>=2.2.0 # MIT
 testresources>=2.0.0 # Apache-2.0/BSD
 testscenarios>=0.4 # Apache-2.0/BSD
 oslo.versionedobjects[fixtures]>=1.31.2 # Apache-2.0
 tempest>=17.1.0 # Apache-2.0
-bandit>=1.1.0 # Apache-2.0
+bandit>=1.1.0,<1.6.3 # Apache-2.0
 reno>=2.5.0 # Apache-2.0

tox.ini

@@ -181,6 +181,7 @@ local-check-factory = cinder.hacking.checks.factory
 import_exceptions = cinder.i18n
 
 [testenv:lower-constraints]
+install_command = pip install {opts} {packages}
 basepython = python3
 deps =
   -c{toxinidir}/lower-constraints.txt