351 Commits

Author SHA1 Message Date
wu.shiming
a1d89e6f0e Remove install unnecessary packages
The docs and releasenotes requirements migrated to doc/requirements.txt
we need not install things from requirements.txt.

Change-Id: I207be0b861c13bc6cf954b2247bb00546990bbf0
2020-09-14 12:03:43 +08:00
Brian Haley
055036ba2b Improve terminology in the Neutron tree
There is no real reason we should be using some of the
terms we do, they're outdated, and we're behind other
open-source projects in this respect. Let's switch to
using more inclusive terms in all possible places.

Change-Id: I99913107e803384b34cbd5ca588451b1cf64d594
2020-08-19 16:47:53 -04:00
Rodolfo Alonso Hernandez
ddd5832323 Remove "six" library
Last step to remove "six" library usage in Neutron.

Change-Id: Idd42e0c51c8c3bd598c9cf91602596be238bccae
2020-07-28 16:55:52 +00:00
Hervé Beraud
3df1679658 Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Also enabled the hacking check that should have caught this, it
was missing from tox.ini along with most of the other in-tree
hacking checks we have added over the years.

Change-Id: Id91175d0db8b8edc72f0dd98925ddbf7415bb881
2020-06-17 14:42:03 +00:00
Akihiro Motoki
52e3fee5ef Switch to hacking 3.0.1
In hacking 2.0 or later, local-check-factory was removed as it is not
compatible with flake8 3.x and it is advised to use flake8's local
plugins [1]. neutron-lib provided a factory to register common hacking
rules, but it no longer works with hacking 2, so we need to define rules
defined in neutron-lib as flake8 local check plugin [2] explicitly.
This needs to be done in each neutron related project, so it is the
downside of the migration to hacking 2.x (I explored a way to continue
to use the factory but failed to find a good way to achieve this) but
I believe it is good to migrate the newer libraries.

* flake8ext decorator in neutron/hacking/checks.py is also replaced with
  hacking.core.flake8ext to avoid the copy-and-paste code.
* neutron-lib dependency is updated as neutron-lib 2.3 added hacking 3 support.
* Python modules related to coding style checks (listed in blacklist.txt in
  openstack/requirements repo) are dropped from lower-constraints.txt
  as they are not actually used in tests (other than pep8).
* HackingDocTestCase is now converted into normal test cases.
  HackingDocTestCase depends on the internal of hacking and pycodestyle
  so it looks better to use normal style of writing tests.

[1] https://docs.openstack.org/releasenotes/hacking/unreleased.html#relnotes-2-0-0
[2] https://flake8.pycqa.org/en/3.7.0/user/configuration.html#using-local-plugins

Change-Id: I92cf50a84bb587a0649a7cffee15cce4ce37d086
2020-05-12 16:58:41 -04:00
Sean McGinnis
783441ef1f
Bump default tox env from py37 to py38
Python 3.8 is now our highest level supported python runtime.
This updates the default tox target environments to swap out
py37 for py38 to make sure local development testing is
covering this version.

This does not impact zuul jobs in any way, nor prevent local
tests against py37. It just changes the default if none is
explicitly provided.

Change-Id: I4c60a7474da79da30122327ef3a6d8e87ccc4371
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-24 10:25:46 -05:00
Slawek Kaplonski
8456240e7d Run fullstack security group test always serially
Security groups are based on iptables in many cases and can interfere
between other tests. So it means that if some other test will manipulate
with iptables on host during SG related test is run, this SG related
test may fail without any obvious reason.
So lets try to run those test serially to be sure that no other agents
will manipulate iptables/openflow rules in same time.

Change-Id: I4d19f2a457e004306fdf40980a943073f1b8704a
Closes-Bug: #1779328
2020-03-03 15:18:18 +01:00
Daniel Bengtsson
f8a3b2c589 Stop configuring install_command in tox.
Currently, we are overriding 'install_command' to use 'pip'. This is
considered poor behavior and 'python -m pip' should be used instead:

https://snarky.ca/why-you-should-use-python-m-pip/

It turns out that this is the the default value provided by tox:

https://tox.readthedocs.io/en/latest/config.html#conf-install_command

So we can remove the line and simply use the default value.
This change showed we needed a newer version of debtcollector in lower
constraints, aliging with the version required by os-vif. This change
showed we needed also a newer version of cffi and keystoneauth1 in lower
constraints. Update also the requirements file for debtcollector and
keystoneauth1.

Change-Id: I5e190c3db8bed0a264b911cdf425aa4c9b51f768
2020-01-28 11:07:40 +01:00
Zuul
cfe31e8b47 Merge "Add a tox requirements target" 2019-12-18 18:19:19 +00:00
Elod Illes
07be793435 Use constraints for docs tox target
In Stein the docs target started to fail when new release of
neutron-lib appeared. This is because tox installs neutron and its
requirements without any constraints. To fix this both the upper
constraints and neutron requirements needs to be added to dependencies
of docs target.

Closes-Bug: #1856156
Change-Id: Iea61238f37fdf24c0264f96d104ee0b3b6aec8e2
2019-12-12 11:14:10 +01:00
Brian Haley
ed26c606c1 Add a tox requirements target
In some other repos you can run 'tox -e requirements' and
run the requirements checks, which is useful when adding
or changing requirements.  Add it to tox.ini and fix a
python2 warning it noticed as well.

Change-Id: I8caa7f755d1276ffe581583a1ea61bb5f0473924
2019-12-11 13:21:42 -05:00
Jens Harbott
d296188c2e Set ignore_basepython_conflict to true in tox.ini
Since dropping py2 support and adding a basepython setting to the global
tox testenv, tox complains about a basepython mismatch when the
installed python3 is python3.6 and the py37 environment is executed (or
vice versa). Setting ignore_basepython_conflict = True will avoid this
message and enforce the correct python version to be used.

Needs a bump in the minimum tox version supported.

See also [0].

[0] https://review.opendev.org/582392

Change-Id: I3d9beeaed104dcf12cfd97590b44c4003754b4e4
2019-11-28 12:47:19 +00:00
Brian Haley
6842465260 Stop testing python 2
Since it's no longer supported past Train, lets stop
running the tests.

Updated docs and made some pep8 code tweaks as well.

Change-Id: I1c171ab906a3b4c66558163ad26947ebf710a276
2019-10-25 18:50:08 +00:00
Akihiro Motoki
66274cdae3 PDF documentation build
The following changes are made for PDF document.

- Sample config/policy files are skipped for PDF doc build.
  We hit several problems around verbatim of inline samples
  in neutron, nova and others, so we decided to skip them now.
  Downloadable links also should be avoided in PDF doc
  as PDF doc is expected to work alone.
- tocdepth for latex is explicitly set to 3 for better navigation
  in the generated PDF. The default was 1 and we cannot provide
  enough information in PDF TOC.
- The module index is disabled as the neutron document does not
  generate a full module index and it shows only partial entries.
  'makeindex' and 'printindex' are set to empty in latex_elements.

The existing contents (mainly the top page) are reorganized a bit
for a better TOC of the generated PDF document:

- The top page of the documentation is simplified only with toctree
  to avoid almost duplicated entries in the TOC of the PDF documentation.
  The current TOC for PDF doc seems to use section titles as the first
  level and title of linked pages as the second level. They are almost
  same and the generated TOC would be redundant.
- The link to the API reference in the top page was moved
  to a subdirectory so that PDF TOC can include it.
- 'Search' section is shown in index.rst only when the builder is 'html'
  as this section uses a form and makes sense only for HTML doc.

Story: 2006099
Task: 35127
Depends-On: https://review.opendev.org/664555
Change-Id: I6f1614a643efa7fad3c2a6ce5692d6873ca0ebd0
2019-08-26 02:50:58 +09:00
Brian Haley
b79842f289 Start enforcing E125 flake8 directive
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues.  Didn't think it was going to be
close to 100 files when I started.

Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
2019-07-19 23:39:41 -04:00
Zuul
a89244459a Merge "Add Python 3 Train unit tests" 2019-07-09 14:45:57 +00:00
Corey Bryant
0a5b6d32de Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: I749a23bbed7074116fba2b3c06e617601e6d4620
Story: #2005924
Task: #34225
2019-07-05 13:50:43 -04:00
chenke
e6e5c1316d Switch to the new canonical constraints URL on master
Reference:
1. http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html
2. https://github.com/openstack/nova/blob/master/tox.ini#L17

Change-Id: Ic2c23439190d30143071c38d8e3b40b3a13fa06c
2019-07-05 15:10:32 +08:00
ZhongShengping
5f5918df8b Use openstack-python3-train-jobs template
This goal is to implement the process set out in the 2018-10-24 Python
Update Process TC resolution[1], for the Train cycle to ensure unit
testing is in place for all of the Tested Runtimes for Train[2].
In practice, this generally means adding unit tests for Python 3.7 and
dropping unit tests for Python 3.5.
For Neutron, which already runs py36 and py37 unit tests it generally
means switch to use the Zuul template openstack-python3-train-jobs
for Train.
Using this template will ensure that all projects that support Python3
will be tested against the agreed runtime versions, and make it easier
to update them in future.

[1]https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html
[2]https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I41d232a10a40d97347566e189457d8044e3639db
Depends-On: https://review.opendev.org/#/c/641878/
2019-05-20 08:58:48 +00:00
ZhongShengping
161e6b80f0 Replace git.openstack.org URLs with opendev.org URLs
Thorough replacement of git.openstack.org URLs with their opendev.org
counterparts.

Change-Id: Ifc446e00d7f69cb23411b3a50c8d880c719f1e73
2019-04-23 10:00:45 +08:00
Ghanshyam Mann
ca0566c99b Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein[2].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2]
https://governance.openstack.org/tc/reference/runtimes/stein.html
https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I6cb8b66ce4b3e456a0e9ae534fd8168b49ac414f
2019-04-15 02:25:39 +00:00
Doug Wiegley
55f2336981
Add method to tox to use local neutron-lib with unit tests
Change-Id: Ic96bb04cc3f9ed1438596b2150030ab22887b091
2019-03-02 14:56:15 -07:00
Bernard Cafarelli
9359366d93 Switch default functional tests to python3, add job for python2
As a part of the python 3 community goal, this converts the functional
tests to run with python3 by default, and in Zuul.

As discussed at the Stein PTG in Denver, unit and functional tests will
still run on both versions, so this adds a python2 job for functional
tests.

This patch also suppress logging levels from some external libraries
to avoid issues with subunit.parser and python 3. For details see bug
reported for Cinder [1].

[1] https://bugs.launchpad.net/cinder/+bug/1728640

Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>

Change-Id: I8958d0b5b9147ffd1ef2d1cef5dcbf79c8be5cd4
2019-01-30 13:31:29 +00:00
Zuul
1f94793aff Merge "tox: Hide deprecation warnings from distgit" 2018-12-20 17:03:42 +00:00
Brian Haley
711a955116 tox: Hide deprecation warnings from distgit
We did this for site warnings, let's do it for distgit as well.

Trivialfix

Change-Id: I656b7f4571c42156aeba309723c3e129ba51f906
2018-12-17 16:59:20 -05:00
Slawek Kaplonski
d01e309f2d Disable displaying warnings in functional tests
Because of known issue with stestr running on Python 3,
that too much output on stdout/stderr cause some subunit.parser
errors, we need to avoid displaying python warnings during
functional tests.

Change-Id: I1a80f62542c68fe891e445920dc89a63efef9175
2018-12-17 20:48:13 +00:00
Akihiro Motoki
f8984c6699 Convert policy.json into policy-in-code
This commit introduces a framework for policy-in-code support
in the neutron stadium and converts the existing policy.json
in the neutron repository into the policy-in-code style.

NOTES:
1) This commit tries not to change the existing policy behavior
provided by the neutron repository even if there are some stale policies
or policies to be defined in a neutron-related project.
They should be clean up later in Stein release.

2) 'default' policy should be dropped from the default policies
as all default policies should be defined in the code (as many projects
which already completed policy-in-code do). However, dropping 'default'
policy potentially affects policy behavior in neutron-related projects,
so it needs to be visit carefully. Considering this, this commit decides
to keep the 'default' policy.

Partially Implements: blueprint neutron-policy-in-code
Change-Id: I6a61079da4d4f5080ee32d640144e6bdb14735fa
2018-12-13 20:37:53 +00:00
Brian Haley
26a4b3eac0 tox: Reuse envdirs
This massively reduces the amount of time needed to install
dependencies and saves a lot of disk space to boot.

Modeled on Nova change, https://review.openstack.org/#/c/534382/

Change-Id: Icae3b6b2b0d014aec17b04b3b516b13b2fa47c5d
2018-12-10 14:47:04 -05:00
Zuul
dd14501c12 Merge "tox: Hide deprecation warnings" 2018-11-27 20:58:28 +00:00
Zuul
8418ecdbc6 Merge "Add python 3.6 unit test job" 2018-11-21 03:38:45 +00:00
Lajos Katona
db06d24414 tox: Hide deprecation warnings
Running tox targets on python3 write out a lot of DeprecationWarning,
the idea to hide them comes from this patch:
https://review.openstack.org/611884 from Stephen Finucane.

Change-Id: Id002ced641bba9833a633b331a3aa82ff0252f47
2018-11-20 17:46:37 +01:00
Bernard Cafarelli
e67d99b654
Make neutron-fullstack job run py3
In the Neutron CI meeting on 2018-10-02 [1] we decided on the proper
approach for fullstack testing of Neutron in Zuul.  The approach is:

1. There should be only one fullstack CI job, named "neutron-fullstack"
2. The neutron-fullstack job should invoke python3 in tox.ini

This change implements the agreed-upon approach.
Switching the nodeset to newer OS will be handled in a separate step

[1] http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-02-16.00.html

Co-Authored-By: Nate Johnston <nate.johnston@redhat.com>
Change-Id: I37c1cb42b3a94532e7b7fea21a929a39fe63f00b
2018-11-13 14:31:19 +01:00
Nguyen Hai Truong
13b7f4f7da Add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: I08e6b9fb4223569c3db04b54137d9092062c22ea
2018-11-08 23:41:08 +07:00
Brian Haley
fc95db987d Fix flake8 N534 untranslated exception message
Fix N534 untranslated exception message warnings and
enable enforcement.

Trivialfix

Change-Id: I9e2b51c768cbb6fcf5588070d1b9e9835775b374
2018-10-19 15:46:04 -04:00
Zuul
8c05cd31ef Merge "Fix flake8 H404 errors" 2018-10-09 07:00:33 +00:00
Brian Haley
2b57f08576 Fix flake8 H404 errors
Fix H404 error and start enforcing it.

Trivialfix

Change-Id: Iaa6fb4f1f07dee32a944259ab65204360d9db7ea
2018-10-05 21:48:41 +00:00
Doug Hellmann
1307cb7587 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I9530cd2433e0e34f05667d33bd6469b2592f8738
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-10-04 18:04:02 +00:00
Slawek Kaplonski
82224347de Update list of skipped bandit plugins
Plugin B111 "Test for the use of rootwrap running as root"
was removed from bandit in [1].
Blacklist call of hashlib.sha1 was blacklisted in bandit
with [2] and it is now added to list of skipped tests in
Neutron.
This commit adds also plugin B604 to list of skipped plugins.

[1] d93eed5492
[2] 35e35446b0

Change-Id: I33d6089b497fccfca4b212e2a62fb881cae9cb94
2018-08-17 09:36:11 +02:00
IWAMOTO Toshihiro
0f180d8e2e Adopt hacking 1.1.0
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8.  This ensures future python3 compatibility
and a bit better code styling.

Change-Id: Ia7c7c5a44727f615a151e1e68dd94c7ed42f974f
2018-07-05 11:31:40 +09:00
Doug Hellmann
5c89506181 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: Id3804896efc11de24d5e13500cdce7571e6d3314
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-13 15:02:25 -04:00
Slawek Kaplonski
ec537c49e9 Switch to stestr
According to Openstack summit session [1] stestr is
maintained project to which all Openstack projects
should migrate.
Let's switch it then.

[1] https://etherpad.openstack.org/p/YVR-python-pti

Change-Id: If3383707c9baf69e03bcccc50970da748a834141
2018-06-02 13:03:33 +02:00
Zuul
e89678b28f Merge "Add a debug env for unit tests" 2018-05-05 02:26:21 +00:00
Brian Haley
7cfdf4aa81 Fix all pep8 E129 errors
Fixed all pep8 E129 errors and changed tox.ini to no longer
ignore them.

Change-Id: I0b06d99ce1d473b79a4cfdd173baa4f02e653847
2018-05-03 13:44:04 +09:00
Brian Haley
c3b83a9ca6 Fix all pep8 E265 errors
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them.  Also removed an N536 comment missed from a
previous change.

Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
2018-04-30 16:35:52 -04:00
Zuul
0aeccc5500 Merge "Remove N536 ignore in tox.ini" 2018-04-29 16:09:31 +00:00
Manjeet Singh Bhatia
dd8aea24bd Add a debug env for unit tests
Debugging unit tests locally with pdb ends in failure with tox.
this patch adds debug environment to debug tests with pdb
break points by following below steps.

1. set break point in code by adding import pdb;pdb.set_trace
2. run tox -e debug test_regex

Change-Id: I088a558a1fb68b3bc2a218ccda0911ed1389f91f
2018-04-28 00:08:43 +00:00
Brian Haley
a8ab93b846 Increase lower-constraints job timeout to 60 minutes
The openstack-tox-lower-constraints job is timing out
a lot at the 40 minute mark, increase to 60 minutes
to try and stabilize it.

Change-Id: I5becda7dd3d2a2ff4883fc4265eb35882173b1c0
2018-04-26 11:45:18 -04:00
Brian Haley
f38b7b9dca Remove N536 ignore in tox.ini
Seems we already comply, so don't ignore.

Change-Id: I5c6b1800105aa2f78577d0b5383905064e06fb7f
2018-04-25 17:32:09 -04:00
Zuul
5e1ddc7247 Merge "Eliminate possible insecure usage of temp file/directory" 2018-04-25 20:24:55 +00:00
Max Pavlukhin
f6b0f7635c Eliminate possible insecure usage of temp file/directory
This patch is a fix for the issue B108: hardcoded_tmp_directory
found by bandit scanner.

The main reasoning for that is the need for safely creating
a temporary file or directory.

Change-Id: I4277e9196b9b707b83bb298faeccb59f07d6f10b
Related-Bug: #1759250
2018-04-24 17:57:30 +04:00