Commit Graph

332 Commits

Author SHA1 Message Date
ZhongShengping
161e6b80f0 Replace git.openstack.org URLs with opendev.org URLs
Thorough replacement of git.openstack.org URLs with their opendev.org
counterparts.

Change-Id: Ifc446e00d7f69cb23411b3a50c8d880c719f1e73
2019-04-23 10:00:45 +08:00
Ghanshyam Mann
ca0566c99b Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein[2].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2]
https://governance.openstack.org/tc/reference/runtimes/stein.html
https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I6cb8b66ce4b3e456a0e9ae534fd8168b49ac414f
2019-04-15 02:25:39 +00:00
Doug Wiegley
55f2336981
Add method to tox to use local neutron-lib with unit tests
Change-Id: Ic96bb04cc3f9ed1438596b2150030ab22887b091
2019-03-02 14:56:15 -07:00
Bernard Cafarelli
9359366d93 Switch default functional tests to python3, add job for python2
As a part of the python 3 community goal, this converts the functional
tests to run with python3 by default, and in Zuul.

As discussed at the Stein PTG in Denver, unit and functional tests will
still run on both versions, so this adds a python2 job for functional
tests.

This patch also suppress logging levels from some external libraries
to avoid issues with subunit.parser and python 3. For details see bug
reported for Cinder [1].

[1] https://bugs.launchpad.net/cinder/+bug/1728640

Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>

Change-Id: I8958d0b5b9147ffd1ef2d1cef5dcbf79c8be5cd4
2019-01-30 13:31:29 +00:00
Zuul
1f94793aff Merge "tox: Hide deprecation warnings from distgit" 2018-12-20 17:03:42 +00:00
Brian Haley
711a955116 tox: Hide deprecation warnings from distgit
We did this for site warnings, let's do it for distgit as well.

Trivialfix

Change-Id: I656b7f4571c42156aeba309723c3e129ba51f906
2018-12-17 16:59:20 -05:00
Slawek Kaplonski
d01e309f2d Disable displaying warnings in functional tests
Because of known issue with stestr running on Python 3,
that too much output on stdout/stderr cause some subunit.parser
errors, we need to avoid displaying python warnings during
functional tests.

Change-Id: I1a80f62542c68fe891e445920dc89a63efef9175
2018-12-17 20:48:13 +00:00
Akihiro Motoki
f8984c6699 Convert policy.json into policy-in-code
This commit introduces a framework for policy-in-code support
in the neutron stadium and converts the existing policy.json
in the neutron repository into the policy-in-code style.

NOTES:
1) This commit tries not to change the existing policy behavior
provided by the neutron repository even if there are some stale policies
or policies to be defined in a neutron-related project.
They should be clean up later in Stein release.

2) 'default' policy should be dropped from the default policies
as all default policies should be defined in the code (as many projects
which already completed policy-in-code do). However, dropping 'default'
policy potentially affects policy behavior in neutron-related projects,
so it needs to be visit carefully. Considering this, this commit decides
to keep the 'default' policy.

Partially Implements: blueprint neutron-policy-in-code
Change-Id: I6a61079da4d4f5080ee32d640144e6bdb14735fa
2018-12-13 20:37:53 +00:00
Brian Haley
26a4b3eac0 tox: Reuse envdirs
This massively reduces the amount of time needed to install
dependencies and saves a lot of disk space to boot.

Modeled on Nova change, https://review.openstack.org/#/c/534382/

Change-Id: Icae3b6b2b0d014aec17b04b3b516b13b2fa47c5d
2018-12-10 14:47:04 -05:00
Zuul
dd14501c12 Merge "tox: Hide deprecation warnings" 2018-11-27 20:58:28 +00:00
Zuul
8418ecdbc6 Merge "Add python 3.6 unit test job" 2018-11-21 03:38:45 +00:00
Lajos Katona
db06d24414 tox: Hide deprecation warnings
Running tox targets on python3 write out a lot of DeprecationWarning,
the idea to hide them comes from this patch:
https://review.openstack.org/611884 from Stephen Finucane.

Change-Id: Id002ced641bba9833a633b331a3aa82ff0252f47
2018-11-20 17:46:37 +01:00
Bernard Cafarelli
e67d99b654
Make neutron-fullstack job run py3
In the Neutron CI meeting on 2018-10-02 [1] we decided on the proper
approach for fullstack testing of Neutron in Zuul.  The approach is:

1. There should be only one fullstack CI job, named "neutron-fullstack"
2. The neutron-fullstack job should invoke python3 in tox.ini

This change implements the agreed-upon approach.
Switching the nodeset to newer OS will be handled in a separate step

[1] http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-02-16.00.html

Co-Authored-By: Nate Johnston <nate.johnston@redhat.com>
Change-Id: I37c1cb42b3a94532e7b7fea21a929a39fe63f00b
2018-11-13 14:31:19 +01:00
Nguyen Hai Truong
13b7f4f7da Add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: I08e6b9fb4223569c3db04b54137d9092062c22ea
2018-11-08 23:41:08 +07:00
Brian Haley
fc95db987d Fix flake8 N534 untranslated exception message
Fix N534 untranslated exception message warnings and
enable enforcement.

Trivialfix

Change-Id: I9e2b51c768cbb6fcf5588070d1b9e9835775b374
2018-10-19 15:46:04 -04:00
Zuul
8c05cd31ef Merge "Fix flake8 H404 errors" 2018-10-09 07:00:33 +00:00
Brian Haley
2b57f08576 Fix flake8 H404 errors
Fix H404 error and start enforcing it.

Trivialfix

Change-Id: Iaa6fb4f1f07dee32a944259ab65204360d9db7ea
2018-10-05 21:48:41 +00:00
Doug Hellmann
1307cb7587 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I9530cd2433e0e34f05667d33bd6469b2592f8738
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-10-04 18:04:02 +00:00
Slawek Kaplonski
82224347de Update list of skipped bandit plugins
Plugin B111 "Test for the use of rootwrap running as root"
was removed from bandit in [1].
Blacklist call of hashlib.sha1 was blacklisted in bandit
with [2] and it is now added to list of skipped tests in
Neutron.
This commit adds also plugin B604 to list of skipped plugins.

[1] d93eed5492
[2] 35e35446b0

Change-Id: I33d6089b497fccfca4b212e2a62fb881cae9cb94
2018-08-17 09:36:11 +02:00
IWAMOTO Toshihiro
0f180d8e2e Adopt hacking 1.1.0
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8.  This ensures future python3 compatibility
and a bit better code styling.

Change-Id: Ia7c7c5a44727f615a151e1e68dd94c7ed42f974f
2018-07-05 11:31:40 +09:00
Doug Hellmann
5c89506181 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: Id3804896efc11de24d5e13500cdce7571e6d3314
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-13 15:02:25 -04:00
Slawek Kaplonski
ec537c49e9 Switch to stestr
According to Openstack summit session [1] stestr is
maintained project to which all Openstack projects
should migrate.
Let's switch it then.

[1] https://etherpad.openstack.org/p/YVR-python-pti

Change-Id: If3383707c9baf69e03bcccc50970da748a834141
2018-06-02 13:03:33 +02:00
Zuul
e89678b28f Merge "Add a debug env for unit tests" 2018-05-05 02:26:21 +00:00
Brian Haley
7cfdf4aa81 Fix all pep8 E129 errors
Fixed all pep8 E129 errors and changed tox.ini to no longer
ignore them.

Change-Id: I0b06d99ce1d473b79a4cfdd173baa4f02e653847
2018-05-03 13:44:04 +09:00
Brian Haley
c3b83a9ca6 Fix all pep8 E265 errors
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them.  Also removed an N536 comment missed from a
previous change.

Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
2018-04-30 16:35:52 -04:00
Zuul
0aeccc5500 Merge "Remove N536 ignore in tox.ini" 2018-04-29 16:09:31 +00:00
Manjeet Singh Bhatia
dd8aea24bd Add a debug env for unit tests
Debugging unit tests locally with pdb ends in failure with tox.
this patch adds debug environment to debug tests with pdb
break points by following below steps.

1. set break point in code by adding import pdb;pdb.set_trace
2. run tox -e debug test_regex

Change-Id: I088a558a1fb68b3bc2a218ccda0911ed1389f91f
2018-04-28 00:08:43 +00:00
Brian Haley
a8ab93b846 Increase lower-constraints job timeout to 60 minutes
The openstack-tox-lower-constraints job is timing out
a lot at the 40 minute mark, increase to 60 minutes
to try and stabilize it.

Change-Id: I5becda7dd3d2a2ff4883fc4265eb35882173b1c0
2018-04-26 11:45:18 -04:00
Brian Haley
f38b7b9dca Remove N536 ignore in tox.ini
Seems we already comply, so don't ignore.

Change-Id: I5c6b1800105aa2f78577d0b5383905064e06fb7f
2018-04-25 17:32:09 -04:00
Zuul
5e1ddc7247 Merge "Eliminate possible insecure usage of temp file/directory" 2018-04-25 20:24:55 +00:00
Max Pavlukhin
f6b0f7635c Eliminate possible insecure usage of temp file/directory
This patch is a fix for the issue B108: hardcoded_tmp_directory
found by bandit scanner.

The main reasoning for that is the need for safely creating
a temporary file or directory.

Change-Id: I4277e9196b9b707b83bb298faeccb59f07d6f10b
Related-Bug: #1759250
2018-04-24 17:57:30 +04:00
Brian Haley
de243a3513 Fix E402 pep8 errors
Fix E402 (module level import not at top of file) pep8 errors
and no longer ignore new failures.

Trivialfix

Change-Id: If9a202f1f322b5b03966c692e33e94b7c872079b
2018-04-20 18:13:07 +00:00
Brian Haley
90cd939047 Fix W503 pep8 warnings
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.

Trivialfix

Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
2018-04-17 14:22:58 +00:00
Zuul
4ec9fae6af Merge "Replace assert to raise AssertionError" 2018-04-11 20:10:35 +00:00
Akihiro Motoki
7a714aeb13 Fix pep8 errors
Recent pep8 upgrade and corresponding pycodestyle update break
pep8 job due to the new rules.

This commit fixes the following new errors:
- E266 too many leading '#' for block comment
- E501 line too long
- H903  Windows style line endings not allowed in code

The following errors are added to the ignore list
as there are many errors:
- E402 module level import not at top of file
- E731 do not assign a lambda expression, use a def
- W503 line break before binary operator

Change-Id: I1fd3357479bb2ba3d89de92739ffac99900761b6
2018-04-11 01:10:49 +09:00
Max
6e6166bda4 Replace assert to raise AssertionError
This patch is a fix for the issue B101 assert_used found by bandit
scanner:
https://docs.openstack.org/bandit/latest/plugins/b101_assert_used.html
As assert functionality could be turned off by the -O option for python
interpreter, assert statements were replaced to raise AssertionError.

The main reasoning for that is rest code after assert could be not
ready for values that assert filters.

Change-Id: I92a871b1b496c96c0c76cb37e4dda51bfc007e38
Related-Bug: #1759250
2018-04-10 16:01:48 +04:00
Zuul
bfefbaf1c7 Merge "add lower-constraints job" 2018-03-25 23:43:45 +00:00
Nguyen Hai
0556a2ab73 Follow the new PTI for document build
For compliance with the Project Testing Interface as described in:
https://governance.openstack.org/tc/reference/project-testing-interface.html
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Remove the '[build_sphinx]' as described in:
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128594.html

Change-Id: Id3d7065028b4c0831e6b5c2363b19626734817a3
2018-03-24 13:23:33 +09:00
Doug Hellmann
3c85a1e140 add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I78f0beea8f78aa635b59bc80c97fec41412049e0
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-03-22 16:27:44 -04:00
Zuul
1f2a5b34de Merge "Update upper-constraints consumption" 2018-03-09 19:49:30 +00:00
Monty Taylor
4a0200a2d4 Update upper-constraints consumption
In order to deal with neutron going into the upper-constraints file so
that neutron plugins can consume it normally, we need to align the
constraints invocation in neutron's tox.ini. This means putting the -C
argument into the dependencies list rather than the pip install line so
that it doesn't get appended to tox installing neutron itself.

Needed-By: https://review.openstack.org/550475
Change-Id: I014177c8afcadd3e6acb20bd38bb9d9fcc778ed6
2018-03-08 14:58:33 +00:00
Luong Anh Tuan
0d3ffa6ebc Enable more extensions in pep8
This patch enable extensions:
-[H204] Use assert(Not)Equal to check for equality.
-[H205] Use assert(Greater|Less)(Equal) for comparison.

Change-Id: I9ad51e006561251030b5a6759c930b87ff94a861
2018-03-05 02:25:23 +00:00
Sławek Kapłoński
f5d9890c8d [Fullstack] Limit number of test workers to 4
By default there is as many test workers running fullstack
tests as cores on host.
On gate tests it is 8 cores so 8 test workers.
Each such workers spawns own neutron-server and agents.
That cause quite big load on such test node. From my tests
it looks that it can be up to 25.
By lowering number of test workers to 4 I observed that
load is at max about 10.
Overall time of tests is longer for about 10-15 minutes
(about 45-50 minutes with 8 workers and about 60-65 minutes
with 4 workers).

Change-Id: I58cb2ade598e8dc2377e6187483cb39881a9dc3c
2018-02-23 12:40:53 +00:00
Boden R
fc6e9a71d8 add doc link validation to release checklist and tox
This patch updates our doc conf.py to support the linkcheck builder in
addition to adding a new 'linkcheck' target in tox to run the builder.
Also the release checklist is updated suggesting the linkcheck tox
target be run prior to a release.

Change-Id: Ia7c282b7331f0b624bb3324f27dfec223cf414f7
Closes-Bug: #1716005
2017-09-21 15:24:03 -06:00
Mike Bayer
67589fbf99 Allow OS_TEST_TIMEOUT to be configurable from env
the OS_TEST_TIMEOUT of 180 is too short for
some test servers.  Allow it to be modified from the
outside via environment.  Currently I have to sed the tox.ini
file to fix this.

Change-Id: Ib0e9aa0fff4cd26be115cc4b12036e9ee5e737f3
2017-09-12 15:11:03 -04:00
Jenkins
5d2017efc3 Merge "Hook bandit security linter to pep8 target" 2017-08-24 06:10:40 +00:00
Inessa Vasilevskaya
7322bd6efb Make code follow log translation guideline
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.

Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
2017-08-14 02:01:48 +00:00
Armando Migliaccio
85a3e93537 Hook bandit security linter to pep8 target
This helps catch a number of potential issues with the neutron
code in advance. A false positive can be skipped with #nosec
at the offending line, just like #noqa can be added on
offending module imports.

Change-Id: I3e4cbc94539dd2cce61bfa5cd0265e75d7336311
2017-08-09 19:48:05 -07:00
Armando Migliaccio
bd4c2c9760 Add bandit target to tox
Change-Id: I3ca2f77e026167bf6828072ccd02872e7c504c1c
2017-07-28 17:49:28 -07:00
YAMAMOTO Takashi
af4bfb35a2 Stop using non-ascii characters
Namely, Replace U+2019 RIGHT SINGLE QUOTATION MARK
with U+0027 APOSTROPHE where appropriate.

Change-Id: Iec9e8f85bcddeb90e9a7fb8f845a7923cac38f52
2017-07-18 17:11:23 +09:00