Browse Source

Stop supporting python <3.6

+ remove sseveral unused ci files
+ ignore some warnings during pytest execution to make the output clean

Change-Id: Ic94ab503ea77c8a1582292becad872670034d49c
changes/37/696537/9
Andrey Kurilin 2 months ago
parent
commit
1e501a6e21
13 changed files with 46 additions and 431 deletions
  1. +17
    -25
      .zuul.d/python-jobs.yaml
  2. +0
    -8
      .zuul.d/zuul.yaml
  3. +8
    -0
      CHANGELOG.rst
  4. +1
    -2
      requirements.txt
  5. +1
    -5
      setup.cfg
  6. +0
    -1
      test-requirements.txt
  7. +0
    -91
      tests/ci/README.rst
  8. +0
    -13
      tests/ci/hooks/certification_post_test_hook.sh
  9. +0
    -46
      tests/ci/rally-gate.sh
  10. +0
    -159
      tests/ci/rally_gate_functions.sh
  11. +0
    -40
      tests/ci/test_install.sh
  12. +19
    -35
      tox.ini
  13. +0
    -6
      upper-constraints.txt

+ 17
- 25
.zuul.d/python-jobs.yaml View File

@@ -37,16 +37,6 @@
vars:
tox_env: functional

- job:
name: rally-tox-functional-py3
parent: rally-tox-base
description: |
Run test for rally project.

Uses tox with the ``functional-py3`` environment.
vars:
tox_env: functional-py3

- job:
name: rally-tox-self
parent: rally-tox-base
@@ -58,51 +48,53 @@
tox_env: self

- job:
name: rally-tox-py27
name: rally-tox-py36
parent: rally-tox-base
description: |
Run unit test for rally project.

Uses tox with the ``py34`` environment.
Uses tox with the ``py36`` environment.
vars:
tox_env: py27
tox_env: py36

- job:
name: rally-tox-py34
name: rally-tox-py37
parent: rally-tox-base
description: |
Run unit test for rally project.

Uses tox with the ``py34`` environment.
Uses tox with the ``py37`` environment.
vars:
tox_env: py34
tox_env: py37

# TODO(andreykurilin): Remove these jobs definitions as soon as
# rally-openstack project stops using them
- job:
name: rally-tox-py35
name: rally-tox-py27
parent: rally-tox-base
description: |
Run unit test for rally project.

Uses tox with the ``py35`` environment.
Uses tox with the ``py27`` environment.
vars:
tox_env: py35
tox_env: py27

- job:
name: rally-tox-py36
name: rally-tox-py34
parent: rally-tox-base
description: |
Run unit test for rally project.

Uses tox with the ``py36`` environment.
Uses tox with the ``py34`` environment.
vars:
tox_env: py36
tox_env: py34

- job:
name: rally-tox-py37
name: rally-tox-py35
parent: rally-tox-base
description: |
Run unit test for rally project.

Uses tox with the ``py37`` environment.
Uses tox with the ``py35`` environment.
vars:
tox_env: py37
tox_env: py35

+ 0
- 8
.zuul.d/zuul.yaml View File

@@ -35,14 +35,10 @@
jobs:
- rally-tox-docs
- rally-tox-pep8
- rally-tox-py27
- rally-tox-py34
- rally-tox-py35
- rally-tox-py36
- rally-tox-py37
- rally-tox-cover
- rally-tox-functional
- rally-tox-functional-py3
- rally-database-migration
- rally-install-ubuntu-xenial
- rally-install-ubuntu-bionic
@@ -52,14 +48,10 @@
jobs:
- rally-tox-docs
- rally-tox-pep8
- rally-tox-py27
- rally-tox-py34
- rally-tox-py35
- rally-tox-py36
- rally-tox-py37
- rally-tox-cover
- rally-tox-functional
- rally-tox-functional-py3
- rally-database-migration
- rally-install-ubuntu-xenial
- rally-install-ubuntu-bionic

+ 8
- 0
CHANGELOG.rst View File

@@ -17,6 +17,14 @@ Changelog
.. Release notes for existing releases are MUTABLE! If there is something that
was missed or can be improved, feel free to change it!

[unreleased]
------------

Removed
~~~~~~~

* Python 2.7, Python 3.4 and Python 3.5 support


[2.1.0] - 2019-11-19
--------------------

+ 1
- 2
requirements.txt View File

@@ -18,8 +18,7 @@ PrettyTable<0.8 # BSD
pyOpenSSL # Apache License, Version 2.0
PyYAML # MIT
python-subunit
requests<2.20.0;python_version=='3.4' # Apache License, Version 2.0
requests!=2.20.0;python_version!='3.4' # Apache License, Version 2.0
requests!=2.20.0 # Apache License, Version 2.0
SQLAlchemy!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8 # MIT
six # MIT
virtualenv!=16.3.0 # MIT

+ 1
- 5
setup.cfg View File

@@ -14,13 +14,9 @@ classifier =
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.4
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8

[files]
packages =

+ 0
- 1
test-requirements.txt View File

@@ -4,7 +4,6 @@

hacking>=0.12.0,!=0.13.0 # Apache Software License

pytest>=2.7,<5.0.0;python_version<='3.4' # MIT
pytest>=2.7;python_version>'3.4' # MIT
# py.test plugin for measuring coverage.
pytest-cov>=2.2.1 # MIT

+ 0
- 91
tests/ci/README.rst View File

@@ -1,91 +0,0 @@
===============
Rally Gate Jobs
===============

For each patch submitted for review on Gerrit, there is a set of tests called
**gate jobs** to be run against it. These tests check whether the Rally code
works correctly after applying the patch and provide additional guarantees that
it won't break the software when it gets merged. Rally gate jobs contain tests
checking the codestyle (via *pep8*), unit tests suites, functional tests and a
set of Rally benchmark tasks that are executed against a real *devstack*
deployment.


rally-gate.sh
-------------
This script runs a set of real Rally benchmark tasks and fetches their results
in textual / visualized form (available via a special html page by clicking the
corresponding job title in Gerrit). It checks that scenarios don't fail while
being executed against a devstack deployment and also tests SLA criteria to
ensure that benchmark tasks have completed successfully.
Jenkins uses this script by runnint the 'gate-rally-dsvm-rally' job,
'gate-rally-dsvm-manila-multibackend' job and 'gate-rally-dsvm-neutron-rally'
job.


rally_verify.py
---------------
This script runs various "rally verify" commands. This set of commands allow us
to perform Tempest tests of OpenStack live cloud and display verification
results.
The verification results obtained by running various "rally verify <cmd>"
commands including "start", "show", "list" are compared using the "rally verify
results" command, which are then saved in csv, html and json formats in the
"rally-verify" directory.
Jenkins uses this script by running the 'gate-rally-dsvm-verify' job.


test_install.sh
---------------
This script tests the correct working of the install_rally.sh, used for the
installation of Rally. Jenkins tests this script by running it against Centos7
and Ubuntu 16.04 in the corresponding jobs 'gate-rally-install-centos-7' and
'gate-rally-install-ubuntu-xenial'.


Jenkins
-------
Jenkins is a Continuous Integration system which works as the scheduler. It
receives events related to proposed changes, triggers tests based on those
events, and reports back.
For each patch that is uploaded for review on Gerrit, Jenkins runs it against
the various rally gate jobs listed below along with their functions and local
equivalents:

* gate-rally-pep8 : code style check
(equal to tox -epep8)
* gate-rally-docs : documention generation
(equal to tox -edocs)
* gate-rally-python27 : unit tests against python27
(equal to tox -epy27)
* gate-rally-python35 : unit tests against python35
(equal to tox -epy35)
* rally-coverage : generates unit test
coverage
(equal to tox -cover)
* gate-rally-install-centos-7 : testing of test_install.sh
(described above) against
Centos7
* gate-rally-install-ubuntu-xenial : testing of test_install.sh
(described above) against
Ubuntu 16.04
* gate-rally-dsvm-rally : runs rally-gate.sh
(described above) against
OpenStack deployed by
devstack with nova-network
(It is standard dsvm job)
* gate-rally-dsvm-manila-multibackend(non-voting) : runs rally-gate.sh against
manila
* gate-rally-dsvm-neutron-rally : runs rally-gate.sh against
OpenStack deployed by
devastack with neutron
* gate-rally-dsvm-cli : runs tests/functional/*
(equal to tox -ecli)
* gate-rally-dsvm-verify(non-voting) : runs rally_verify.py and
tests Rally and Tempest
integration in all possible
ways
* gate-rally-tox-self(non-voting) : not yet used

and a success in these tests(except non-voting) would mean that the patch is
approved by Jenkins.

+ 0
- 13
tests/ci/hooks/certification_post_test_hook.sh View File

@@ -1,13 +0,0 @@
#!/usr/bin/env bash

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
source $SCRIPT_DIR/../rally_gate_functions.sh

setUp

TASK=$RALLY_DIR/tasks/openstack/task.yaml
TASK_ARGS=$RALLY_DIR/rally-jobs/certifcation_task_args.yaml

TASK_ARGS="--task-args-file $TASK_ARGS"

run $TASK $TASK_ARGS

+ 0
- 46
tests/ci/rally-gate.sh View File

@@ -1,46 +0,0 @@
#!/bin/bash -ex
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

# This script is executed by post_test_hook function in devstack gate.

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
source $SCRIPT_DIR/rally_gate_functions.sh

PROJECT=`echo $ZUUL_PROJECT | cut -d \/ -f 2`

RALLY_JOB_DIR=$BASE/new/$PROJECT/rally-scenarios
if [ ! -d $RALLY_JOB_DIR ]; then
RALLY_JOB_DIR=$BASE/new/$PROJECT/rally-jobs
fi

echo $RALLY_JOB_DIR
echo $RALLY_DIR
ls $BASE/new/$PROJECT

if [ "$RALLY_SCENARIO" == "rally-keystone-api-v2" ]; then
echo "WARNING: RALLY TASK WILL NOT BE LAUNCHED."
exit 0
fi

setUp $RALLY_JOB_DIR

BASE_FOR_TASK=${RALLY_JOB_DIR}/${RALLY_SCENARIO}

TASK=${BASE_FOR_TASK}.yaml
TASK_ARGS=""
if [ -f ${BASE_FOR_TASK}_args.yaml ]; then
TASK_ARGS="--task-args-file ${BASE_FOR_TASK}_args.yaml"
fi

run $TASK $TASK_ARGS

+ 0
- 159
tests/ci/rally_gate_functions.sh View File

@@ -1,159 +0,0 @@
#!/usr/bin/env bash

RALLY_DIR=$BASE/new/rally
RALLY_PLUGINS_DIR=~/.rally/plugins
RALLY_EXTRA_DIR=~/.rally/extra

function setUp () {
set -x

sudo pip install rally_openstack

JOB_DIR=$1

mkdir -p $RALLY_PLUGINS_DIR
mkdir -p $RALLY_EXTRA_DIR

if [ -n "$JOB_DIR" ]; then
PLUGINS_DIR=${JOB_DIR}/plugins
EXTRA_DIR=${JOB_DIR}/extra

if [ -d $PLUGINS_DIR ]; then
cp -r $PLUGINS_DIR/ $RALLY_PLUGINS_DIR
fi

if [ -d $EXTRA_DIR ]; then
cp -r $EXTRA_DIR/* ~/.rally/extra/
fi
fi

touch ~/.rally/extra/fake-image.img

env
set -o pipefail

rally deployment use --deployment devstack

source ~/.rally/openrc admin admin

OPENSTACK_SERVICES=$(openstack service list)
if [[ $OPENSTACK_SERVICES == *"glance"* ]]; then
openstack image list
fi
if [[ $OPENSTACK_SERVICES == *"cinder"* ]]; then
openstack volume list --all-projects
fi
if [[ $OPENSTACK_SERVICES == *"neutron"* ]]; then
openstack network list
fi

# NOTE(ikhudoshyn): Create additional users and register a new env
# so that we could run scenarios using 'existing_users' context
if [ "$DEVSTACK_GATE_PREPOPULATE_USERS" = "1" ]; then
openstack --version

openstack project create rally-test-project-1
openstack user create --project rally-test-project-1 --password rally-test-password-1 rally-test-user-1
openstack role add --project rally-test-project-1 --user rally-test-user-1 Member

openstack project create rally-test-project-2
openstack user create --project rally-test-project-2 --password rally-test-password-2 rally-test-user-2
openstack role add --project rally-test-project-2 --user rally-test-user-2 Member

set +e
NEUTRON_EXISTS=$(openstack --os-interface admin service list | grep neutron)
set -e
if [ "$NEUTRON_EXISTS" ]; then
OS_QUOTA_STR="--networks -1 --subnets -1 --routers -1 --floating-ips -1 --subnetpools -1 --secgroups -1 --secgroup-rules -1 --ports -1"
openstack --debug quota set $OS_QUOTA_STR rally-test-project-1
openstack --debug quota show rally-test-project-1
openstack --debug quota set $OS_QUOTA_STR rally-test-project-2
openstack --debug quota show rally-test-project-2
fi

DEPLOYMENT_CONFIG_FILE=~/.rally/with-existing-users-config

echo '
{
"openstack": {
"users": [
{"username": "rally-test-user-1",
"password": "rally-test-password-1",
"project_name": "rally-test-project-1",
"user_domain_name": "Default",
"project_domain_name": "Default"
},
{"username": "rally-test-user-2",
"password": "rally-test-password-2",
"project_name": "rally-test-project-2",
"user_domain_name": "Default",
"project_domain_name": "Default"
}],
"auth_url": "'$OS_AUTH_URL'",
"region_name": "RegionOne"
}
}
' > $DEPLOYMENT_CONFIG_FILE

rally deployment create --name devstask-with-users --filename $DEPLOYMENT_CONFIG_FILE
fi

rally deployment config
rally --debug deployment check

if rally deployment check | grep 'nova' | grep 'Available' > /dev/null;
then
nova flavor-create m1.nano 42 64 0 1
fi
}

function run () {
set -x

TASK=$1
TASK_ARGS="$2 $3"

if [ "$DEVSTACK_GATE_USE_PYTHON3" = "True" ]; then
PYTHON=python3
else
PYTHON=python
fi

set +e
rally --rally-debug task start --task $TASK $TASK_ARGS
retval=$?
if [[ $? -eq 1 ]]; then
exit $retval
fi
set -e

mkdir -p rally-plot/extra
$PYTHON $RALLY_DIR/tests/ci/render.py ci/index.html > rally-plot/extra/index.html
cp $TASK rally-plot/task.txt
tar -czf rally-plot/plugins.tar.gz -C $RALLY_PLUGINS_DIR .
rally task results | python -m json.tool > rally-plot/results.json
rally task import --file rally-plot/results.json
gzip -9 rally-plot/results.json
rally task detailed > rally-plot/detailed.txt
gzip -9 rally-plot/detailed.txt
rally task detailed --iterations-data > rally-plot/detailed_with_iterations.txt
gzip -9 rally-plot/detailed_with_iterations.txt
rally task report --html-static --out rally-plot/results.html
gzip -9 rally-plot/results.html
rally task report --json --out rally-plot/new_results.json
rally task import --file rally-plot/new_results.json
gzip -9 rally-plot/new_results.json
rally task export --type junit-xml --to rally-plot/junit.xml
gzip -9 rally-plot/junit.xml
rally task export --type elastic --to rally-plot/elasticsearch.txt
gzip -9 rally-plot/elasticsearch.txt

# NOTE(stpierre): if the sla check fails, we still want osresources.py
# to run, so we turn off -e and save the return value
set +e
rally task sla-check | tee rally-plot/sla.txt
retval=$?
set -e

exit $retval
}

+ 0
- 40
tests/ci/test_install.sh View File

@@ -1,40 +0,0 @@
#!/bin/sh -ex
#
# Copyright 2013: Mirantis Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.

sudo yum remove -y python-crypto || true

# NOTE(pabelanger): We run apt-get update to ensure we don't have a stale
# package cache in the gate.
sudo apt-get update || true

sudo ./install_rally.sh --system --yes
rally deployment list
[ -d /etc/bash_completion.d ] && cat /etc/bash_completion.d/rally.bash_completion || true

sudo ./install_rally.sh --system --yes
rally deployment list

sudo ./install_rally.sh --yes -d /tmp/rallytest_root/
/tmp/rallytest_root/bin/rally deployment list
cat /tmp/rallytest_root/etc/bash_completion.d/rally.bash_completion

sudo rm -fr ~/.rally

./install_rally.sh --yes -d /tmp/rallytest_user
/tmp/rallytest_user/bin/rally deployment list

./install_rally.sh --overwrite --dbtype sqlite

+ 19
- 35
tox.ini View File

@@ -1,8 +1,8 @@
[tox]
minversion = 2.0
skipsdist = True
envlist = py35,py34,py27,pep8,samples
ignore_basepython_conflict = true
envlist = py36,py37,py38,pep8,samples

[testenv]
extras = {env:RALLY_EXTRAS:}
@@ -24,57 +24,26 @@ commands =
find . -type f -name "*.pyc" -delete
python {toxinidir}/tests/ci/pytest_launcher.py tests/unit --posargs={posargs}
distribute = false
basepython = python3
basepython = python3.6
passenv = PYTEST_REPORT http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY

[testenv:pep8]
commands = flake8
distribute = false


[testenv:py34]
basepython = python3.4


[testenv:py35]
basepython = python3.5


[testenv:py36]
basepython = python3.6


[testenv:py37]
basepython = python3.7

[testenv:py38]
basepython = python3.8

[testenv:venv]
commands = {posargs}

[testenv:debug]
commands = oslo_debug_helper -t tests {posargs}

[testenv:debug34]
basepython = python3.4
commands = oslo_debug_helper -t tests {posargs}

[testenv:debug35]
basepython = python3.5
commands = oslo_debug_helper -t tests {posargs}

[testenv:cli]
commands =
find . -type f -name "*.pyc" -delete
echo "do nothing"

[testenv:functional]
basepython = python2.7
commands =
find . -type f -name "*.pyc" -delete
python {toxinidir}/tests/ci/pytest_launcher.py tests/functional --posargs={posargs}

[testenv:functional-py3]
basepython = python3
commands =
find . -type f -name "*.pyc" -delete
python {toxinidir}/tests/ci/pytest_launcher.py tests/functional --posargs={posargs}
@@ -121,3 +90,18 @@ commands = bindep

[testenv:self]
commands = {toxinidir}/tests/ci/rally_self_job.sh {toxinidir}/rally-jobs/self-rally.yaml

[pytest]
filterwarnings =
error
ignore:.*inspect.getargspec.*deprecated, use inspect.signature.*:DeprecationWarning
ignore:the imp module is deprecated in favour of importlib:DeprecationWarning:
ignore:.*EngineFacade is deprecated; please use oslo_db.sqlalchemy.enginefacade*:
ignore::DeprecationWarning:rally.plugins.common.validators:47
ignore::DeprecationWarning:rally.cli.cliutils:71
ignore:invalid escape sequence:DeprecationWarning
ignore::DeprecationWarning:tests.unit.test_hacking
ignore::DeprecationWarning:testtools.*
# python 3.7
ignore:Using or importing the ABCs:DeprecationWarning:unittest2.*
ignore:isAlive*:PendingDeprecationWarning:

+ 0
- 6
upper-constraints.txt View File

@@ -8,15 +8,9 @@ oslo.log===3.44.1
paramiko===2.6.0
pbr===5.4.3
PrettyTable===0.7.2
Pygments==2.3.1;python_version=='3.4'
pyOpenSSL===19.0.0
python-subunit===1.3.0
PyYAML===5.1.2
requests===2.21.0;python_version=='3.4'
six===1.13.0
Sphinx==1.8.5;python_version=='2.7'
Sphinx==1.8.5;python_version=='3.4'
SQLAlchemy===1.3.10
virtualenv===16.7.7
importlib-metadata==1.1.3;python_version=='3.4'
more-itertools==7.2.0;python_version=='3.4'

Loading…
Cancel
Save