Stop supporting python <3.6

+ remove sseveral unused ci files
+ ignore some warnings during pytest execution to make the output clean

Change-Id: Ic94ab503ea77c8a1582292becad872670034d49c
This commit is contained in:
Andrey Kurilin 2019-11-28 14:59:51 +02:00
parent b5eaf963d0
commit 1e501a6e21
13 changed files with 52 additions and 437 deletions

View File

@ -37,16 +37,6 @@
vars:
tox_env: functional
- job:
name: rally-tox-functional-py3
parent: rally-tox-base
description: |
Run test for rally project.
Uses tox with the ``functional-py3`` environment.
vars:
tox_env: functional-py3
- job:
name: rally-tox-self
parent: rally-tox-base
@ -57,13 +47,35 @@
vars:
tox_env: self
- job:
name: rally-tox-py36
parent: rally-tox-base
description: |
Run unit test for rally project.
Uses tox with the ``py36`` environment.
vars:
tox_env: py36
- job:
name: rally-tox-py37
parent: rally-tox-base
description: |
Run unit test for rally project.
Uses tox with the ``py37`` environment.
vars:
tox_env: py37
# TODO(andreykurilin): Remove these jobs definitions as soon as
# rally-openstack project stops using them
- job:
name: rally-tox-py27
parent: rally-tox-base
description: |
Run unit test for rally project.
Uses tox with the ``py34`` environment.
Uses tox with the ``py27`` environment.
vars:
tox_env: py27
@ -86,23 +98,3 @@
Uses tox with the ``py35`` environment.
vars:
tox_env: py35
- job:
name: rally-tox-py36
parent: rally-tox-base
description: |
Run unit test for rally project.
Uses tox with the ``py36`` environment.
vars:
tox_env: py36
- job:
name: rally-tox-py37
parent: rally-tox-base
description: |
Run unit test for rally project.
Uses tox with the ``py37`` environment.
vars:
tox_env: py37

View File

@ -35,14 +35,10 @@
jobs:
- rally-tox-docs
- rally-tox-pep8
- rally-tox-py27
- rally-tox-py34
- rally-tox-py35
- rally-tox-py36
- rally-tox-py37
- rally-tox-cover
- rally-tox-functional
- rally-tox-functional-py3
- rally-database-migration
- rally-install-ubuntu-xenial
- rally-install-ubuntu-bionic
@ -52,14 +48,10 @@
jobs:
- rally-tox-docs
- rally-tox-pep8
- rally-tox-py27
- rally-tox-py34
- rally-tox-py35
- rally-tox-py36
- rally-tox-py37
- rally-tox-cover
- rally-tox-functional
- rally-tox-functional-py3
- rally-database-migration
- rally-install-ubuntu-xenial
- rally-install-ubuntu-bionic

View File

@ -17,6 +17,14 @@ Changelog
.. Release notes for existing releases are MUTABLE! If there is something that
was missed or can be improved, feel free to change it!
[unreleased]
------------
Removed
~~~~~~~
* Python 2.7, Python 3.4 and Python 3.5 support
[2.1.0] - 2019-11-19
--------------------

View File

@ -18,8 +18,7 @@ PrettyTable<0.8 # BSD
pyOpenSSL # Apache License, Version 2.0
PyYAML # MIT
python-subunit
requests<2.20.0;python_version=='3.4' # Apache License, Version 2.0
requests!=2.20.0;python_version!='3.4' # Apache License, Version 2.0
requests!=2.20.0 # Apache License, Version 2.0
SQLAlchemy!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8 # MIT
six # MIT
virtualenv!=16.3.0 # MIT

View File

@ -14,13 +14,9 @@ classifier =
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.4
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
[files]
packages =

View File

@ -4,7 +4,6 @@
hacking>=0.12.0,!=0.13.0 # Apache Software License
pytest>=2.7,<5.0.0;python_version<='3.4' # MIT
pytest>=2.7;python_version>'3.4' # MIT
# py.test plugin for measuring coverage.
pytest-cov>=2.2.1 # MIT

View File

@ -1,91 +0,0 @@
===============
Rally Gate Jobs
===============
For each patch submitted for review on Gerrit, there is a set of tests called
**gate jobs** to be run against it. These tests check whether the Rally code
works correctly after applying the patch and provide additional guarantees that
it won't break the software when it gets merged. Rally gate jobs contain tests
checking the codestyle (via *pep8*), unit tests suites, functional tests and a
set of Rally benchmark tasks that are executed against a real *devstack*
deployment.
rally-gate.sh
-------------
This script runs a set of real Rally benchmark tasks and fetches their results
in textual / visualized form (available via a special html page by clicking the
corresponding job title in Gerrit). It checks that scenarios don't fail while
being executed against a devstack deployment and also tests SLA criteria to
ensure that benchmark tasks have completed successfully.
Jenkins uses this script by runnint the 'gate-rally-dsvm-rally' job,
'gate-rally-dsvm-manila-multibackend' job and 'gate-rally-dsvm-neutron-rally'
job.
rally_verify.py
---------------
This script runs various "rally verify" commands. This set of commands allow us
to perform Tempest tests of OpenStack live cloud and display verification
results.
The verification results obtained by running various "rally verify <cmd>"
commands including "start", "show", "list" are compared using the "rally verify
results" command, which are then saved in csv, html and json formats in the
"rally-verify" directory.
Jenkins uses this script by running the 'gate-rally-dsvm-verify' job.
test_install.sh
---------------
This script tests the correct working of the install_rally.sh, used for the
installation of Rally. Jenkins tests this script by running it against Centos7
and Ubuntu 16.04 in the corresponding jobs 'gate-rally-install-centos-7' and
'gate-rally-install-ubuntu-xenial'.
Jenkins
-------
Jenkins is a Continuous Integration system which works as the scheduler. It
receives events related to proposed changes, triggers tests based on those
events, and reports back.
For each patch that is uploaded for review on Gerrit, Jenkins runs it against
the various rally gate jobs listed below along with their functions and local
equivalents:
* gate-rally-pep8 : code style check
(equal to tox -epep8)
* gate-rally-docs : documention generation
(equal to tox -edocs)
* gate-rally-python27 : unit tests against python27
(equal to tox -epy27)
* gate-rally-python35 : unit tests against python35
(equal to tox -epy35)
* rally-coverage : generates unit test
coverage
(equal to tox -cover)
* gate-rally-install-centos-7 : testing of test_install.sh
(described above) against
Centos7
* gate-rally-install-ubuntu-xenial : testing of test_install.sh
(described above) against
Ubuntu 16.04
* gate-rally-dsvm-rally : runs rally-gate.sh
(described above) against
OpenStack deployed by
devstack with nova-network
(It is standard dsvm job)
* gate-rally-dsvm-manila-multibackend(non-voting) : runs rally-gate.sh against
manila
* gate-rally-dsvm-neutron-rally : runs rally-gate.sh against
OpenStack deployed by
devastack with neutron
* gate-rally-dsvm-cli : runs tests/functional/*
(equal to tox -ecli)
* gate-rally-dsvm-verify(non-voting) : runs rally_verify.py and
tests Rally and Tempest
integration in all possible
ways
* gate-rally-tox-self(non-voting) : not yet used
and a success in these tests(except non-voting) would mean that the patch is
approved by Jenkins.

View File

@ -1,13 +0,0 @@
#!/usr/bin/env bash
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
source $SCRIPT_DIR/../rally_gate_functions.sh
setUp
TASK=$RALLY_DIR/tasks/openstack/task.yaml
TASK_ARGS=$RALLY_DIR/rally-jobs/certifcation_task_args.yaml
TASK_ARGS="--task-args-file $TASK_ARGS"
run $TASK $TASK_ARGS

View File

@ -1,46 +0,0 @@
#!/bin/bash -ex
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# This script is executed by post_test_hook function in devstack gate.
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
source $SCRIPT_DIR/rally_gate_functions.sh
PROJECT=`echo $ZUUL_PROJECT | cut -d \/ -f 2`
RALLY_JOB_DIR=$BASE/new/$PROJECT/rally-scenarios
if [ ! -d $RALLY_JOB_DIR ]; then
RALLY_JOB_DIR=$BASE/new/$PROJECT/rally-jobs
fi
echo $RALLY_JOB_DIR
echo $RALLY_DIR
ls $BASE/new/$PROJECT
if [ "$RALLY_SCENARIO" == "rally-keystone-api-v2" ]; then
echo "WARNING: RALLY TASK WILL NOT BE LAUNCHED."
exit 0
fi
setUp $RALLY_JOB_DIR
BASE_FOR_TASK=${RALLY_JOB_DIR}/${RALLY_SCENARIO}
TASK=${BASE_FOR_TASK}.yaml
TASK_ARGS=""
if [ -f ${BASE_FOR_TASK}_args.yaml ]; then
TASK_ARGS="--task-args-file ${BASE_FOR_TASK}_args.yaml"
fi
run $TASK $TASK_ARGS

View File

@ -1,159 +0,0 @@
#!/usr/bin/env bash
RALLY_DIR=$BASE/new/rally
RALLY_PLUGINS_DIR=~/.rally/plugins
RALLY_EXTRA_DIR=~/.rally/extra
function setUp () {
set -x
sudo pip install rally_openstack
JOB_DIR=$1
mkdir -p $RALLY_PLUGINS_DIR
mkdir -p $RALLY_EXTRA_DIR
if [ -n "$JOB_DIR" ]; then
PLUGINS_DIR=${JOB_DIR}/plugins
EXTRA_DIR=${JOB_DIR}/extra
if [ -d $PLUGINS_DIR ]; then
cp -r $PLUGINS_DIR/ $RALLY_PLUGINS_DIR
fi
if [ -d $EXTRA_DIR ]; then
cp -r $EXTRA_DIR/* ~/.rally/extra/
fi
fi
touch ~/.rally/extra/fake-image.img
env
set -o pipefail
rally deployment use --deployment devstack
source ~/.rally/openrc admin admin
OPENSTACK_SERVICES=$(openstack service list)
if [[ $OPENSTACK_SERVICES == *"glance"* ]]; then
openstack image list
fi
if [[ $OPENSTACK_SERVICES == *"cinder"* ]]; then
openstack volume list --all-projects
fi
if [[ $OPENSTACK_SERVICES == *"neutron"* ]]; then
openstack network list
fi
# NOTE(ikhudoshyn): Create additional users and register a new env
# so that we could run scenarios using 'existing_users' context
if [ "$DEVSTACK_GATE_PREPOPULATE_USERS" = "1" ]; then
openstack --version
openstack project create rally-test-project-1
openstack user create --project rally-test-project-1 --password rally-test-password-1 rally-test-user-1
openstack role add --project rally-test-project-1 --user rally-test-user-1 Member
openstack project create rally-test-project-2
openstack user create --project rally-test-project-2 --password rally-test-password-2 rally-test-user-2
openstack role add --project rally-test-project-2 --user rally-test-user-2 Member
set +e
NEUTRON_EXISTS=$(openstack --os-interface admin service list | grep neutron)
set -e
if [ "$NEUTRON_EXISTS" ]; then
OS_QUOTA_STR="--networks -1 --subnets -1 --routers -1 --floating-ips -1 --subnetpools -1 --secgroups -1 --secgroup-rules -1 --ports -1"
openstack --debug quota set $OS_QUOTA_STR rally-test-project-1
openstack --debug quota show rally-test-project-1
openstack --debug quota set $OS_QUOTA_STR rally-test-project-2
openstack --debug quota show rally-test-project-2
fi
DEPLOYMENT_CONFIG_FILE=~/.rally/with-existing-users-config
echo '
{
"openstack": {
"users": [
{"username": "rally-test-user-1",
"password": "rally-test-password-1",
"project_name": "rally-test-project-1",
"user_domain_name": "Default",
"project_domain_name": "Default"
},
{"username": "rally-test-user-2",
"password": "rally-test-password-2",
"project_name": "rally-test-project-2",
"user_domain_name": "Default",
"project_domain_name": "Default"
}],
"auth_url": "'$OS_AUTH_URL'",
"region_name": "RegionOne"
}
}
' > $DEPLOYMENT_CONFIG_FILE
rally deployment create --name devstask-with-users --filename $DEPLOYMENT_CONFIG_FILE
fi
rally deployment config
rally --debug deployment check
if rally deployment check | grep 'nova' | grep 'Available' > /dev/null;
then
nova flavor-create m1.nano 42 64 0 1
fi
}
function run () {
set -x
TASK=$1
TASK_ARGS="$2 $3"
if [ "$DEVSTACK_GATE_USE_PYTHON3" = "True" ]; then
PYTHON=python3
else
PYTHON=python
fi
set +e
rally --rally-debug task start --task $TASK $TASK_ARGS
retval=$?
if [[ $? -eq 1 ]]; then
exit $retval
fi
set -e
mkdir -p rally-plot/extra
$PYTHON $RALLY_DIR/tests/ci/render.py ci/index.html > rally-plot/extra/index.html
cp $TASK rally-plot/task.txt
tar -czf rally-plot/plugins.tar.gz -C $RALLY_PLUGINS_DIR .
rally task results | python -m json.tool > rally-plot/results.json
rally task import --file rally-plot/results.json
gzip -9 rally-plot/results.json
rally task detailed > rally-plot/detailed.txt
gzip -9 rally-plot/detailed.txt
rally task detailed --iterations-data > rally-plot/detailed_with_iterations.txt
gzip -9 rally-plot/detailed_with_iterations.txt
rally task report --html-static --out rally-plot/results.html
gzip -9 rally-plot/results.html
rally task report --json --out rally-plot/new_results.json
rally task import --file rally-plot/new_results.json
gzip -9 rally-plot/new_results.json
rally task export --type junit-xml --to rally-plot/junit.xml
gzip -9 rally-plot/junit.xml
rally task export --type elastic --to rally-plot/elasticsearch.txt
gzip -9 rally-plot/elasticsearch.txt
# NOTE(stpierre): if the sla check fails, we still want osresources.py
# to run, so we turn off -e and save the return value
set +e
rally task sla-check | tee rally-plot/sla.txt
retval=$?
set -e
exit $retval
}

View File

@ -1,40 +0,0 @@
#!/bin/sh -ex
#
# Copyright 2013: Mirantis Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
sudo yum remove -y python-crypto || true
# NOTE(pabelanger): We run apt-get update to ensure we don't have a stale
# package cache in the gate.
sudo apt-get update || true
sudo ./install_rally.sh --system --yes
rally deployment list
[ -d /etc/bash_completion.d ] && cat /etc/bash_completion.d/rally.bash_completion || true
sudo ./install_rally.sh --system --yes
rally deployment list
sudo ./install_rally.sh --yes -d /tmp/rallytest_root/
/tmp/rallytest_root/bin/rally deployment list
cat /tmp/rallytest_root/etc/bash_completion.d/rally.bash_completion
sudo rm -fr ~/.rally
./install_rally.sh --yes -d /tmp/rallytest_user
/tmp/rallytest_user/bin/rally deployment list
./install_rally.sh --overwrite --dbtype sqlite

54
tox.ini
View File

@ -1,8 +1,8 @@
[tox]
minversion = 2.0
skipsdist = True
envlist = py35,py34,py27,pep8,samples
ignore_basepython_conflict = true
envlist = py36,py37,py38,pep8,samples
[testenv]
extras = {env:RALLY_EXTRAS:}
@ -24,57 +24,26 @@ commands =
find . -type f -name "*.pyc" -delete
python {toxinidir}/tests/ci/pytest_launcher.py tests/unit --posargs={posargs}
distribute = false
basepython = python3
basepython = python3.6
passenv = PYTEST_REPORT http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY
[testenv:pep8]
commands = flake8
distribute = false
[testenv:py34]
basepython = python3.4
[testenv:py35]
basepython = python3.5
[testenv:py36]
basepython = python3.6
[testenv:py37]
basepython = python3.7
[testenv:py38]
basepython = python3.8
[testenv:venv]
commands = {posargs}
[testenv:debug]
commands = oslo_debug_helper -t tests {posargs}
[testenv:debug34]
basepython = python3.4
commands = oslo_debug_helper -t tests {posargs}
[testenv:debug35]
basepython = python3.5
commands = oslo_debug_helper -t tests {posargs}
[testenv:cli]
commands =
find . -type f -name "*.pyc" -delete
echo "do nothing"
[testenv:functional]
basepython = python2.7
commands =
find . -type f -name "*.pyc" -delete
python {toxinidir}/tests/ci/pytest_launcher.py tests/functional --posargs={posargs}
[testenv:functional-py3]
basepython = python3
commands =
find . -type f -name "*.pyc" -delete
python {toxinidir}/tests/ci/pytest_launcher.py tests/functional --posargs={posargs}
@ -121,3 +90,18 @@ commands = bindep
[testenv:self]
commands = {toxinidir}/tests/ci/rally_self_job.sh {toxinidir}/rally-jobs/self-rally.yaml
[pytest]
filterwarnings =
error
ignore:.*inspect.getargspec.*deprecated, use inspect.signature.*:DeprecationWarning
ignore:the imp module is deprecated in favour of importlib:DeprecationWarning:
ignore:.*EngineFacade is deprecated; please use oslo_db.sqlalchemy.enginefacade*:
ignore::DeprecationWarning:rally.plugins.common.validators:47
ignore::DeprecationWarning:rally.cli.cliutils:71
ignore:invalid escape sequence:DeprecationWarning
ignore::DeprecationWarning:tests.unit.test_hacking
ignore::DeprecationWarning:testtools.*
# python 3.7
ignore:Using or importing the ABCs:DeprecationWarning:unittest2.*
ignore:isAlive*:PendingDeprecationWarning:

View File

@ -8,15 +8,9 @@ oslo.log===3.44.1
paramiko===2.6.0
pbr===5.4.3
PrettyTable===0.7.2
Pygments==2.3.1;python_version=='3.4'
pyOpenSSL===19.0.0
python-subunit===1.3.0
PyYAML===5.1.2
requests===2.21.0;python_version=='3.4'
six===1.13.0
Sphinx==1.8.5;python_version=='2.7'
Sphinx==1.8.5;python_version=='3.4'
SQLAlchemy===1.3.10
virtualenv===16.7.7
importlib-metadata==1.1.3;python_version=='3.4'
more-itertools==7.2.0;python_version=='3.4'