It's seen that these tests interfered(removed router namespace) with
test_metadata_proxy_rate_limiting_ipv6 but can interfere with others
too, let's run these serially to avoid random failures.
Following tests will run serially now:-
- test_periodic_sync_routers_task
- test_periodic_sync_routers_task_routers_deleted_while_agent_down
- test_periodic_sync_routers_task_routers_deleted_while_agent_sync
Closes-Bug: #2069744
Change-Id: I34598cb9ad39c96f5e46d98af1185992c5eb3446
As mentioned in the related bug we are seeing
hign memory usage in the job since sqlalchemy
bump to 2.0 and this leads to gate instability.
Adding some swap to handle this until the issue
is fixed.
Also reverts the concurrency reduction change[1]
that was done for this issue as that shouldn't be
needed now.
[1] https://review.opendev.org/c/openstack/neutron/+/920766
Related-Bug: #2065821
Change-Id: I33c4869c1d5230c0c8cc2f0dd4f6d071600bf220
Add '--concurrency 4' when running coverage job as it
helps work around an OOM killer issue we are seeing in
the gate. It typically only adds between 10 and 30
minutes to job. Can revert once issue is resolved.
Partial-bug: #2065821
Change-Id: I33aea3a3c884877573d4d45c6e07955bb177c766
There are currently two issues affecting the neutron gate, one
for pep8 job and the other for fullstack. We need to fix them
both together since they cannot merge separately.
Bump pylint to the latest version to fix pep8 job - it was
pinned to an old version which is causing the job to hang.
Do not use str(url) to stringify a URL for subsequent use
to fix the fullstack job. The str(url) function in SQLAlchemy
hides the password. For a URL string that is to be re-used, use
render_as_string(hide_password=False).
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Closes-Bug: #2065779
Change-Id: I8d04db91ddc2995fbff05b4c4c48baebcc418522
Remove B604 (any_other_function_with_shell_equals_true),
so use subprocess.Popen with the default shell=False,
thus removing the shell parameter of common.utils.subprocess
Popen method.
Depends-On: https://review.opendev.org/c/915991
Change-Id: I1857e982211c4de29b3e24ec2485ea9c17c98b10
Remove B324 (prohibit list calls: md5, sha1 for python>=3.9)
from bandit skip list, for this replace sha1 with blake2b.
Change-Id: Iafe571ad0de0408414ed321f4b9e9588916a873d
Remove B303 (md5, sha1 for python<3.9) and
remove B311 (Standard pseudo-random generators are
not suitable for security/cryptographic purpose) from
the skip list of bandit execution.
Change-Id: I6e9e61e7f94dc9ca339942529af8997adef45e38
Update hacking to a more recent version, along with
flake8-import-order.
Remove N347 (import mock library) check as that is the
default with later hacking versions.
Update the builtins override of '_' to be the neutron.i18n
version due to the code triggering a false positive. This
is done in a couple of other projects as well.
Fix a number of new warnings it found.
Added some ignore directives for new whitespace issues
found in the test tree, can fix later.
TrivialFix
Change-Id: I5923255af86cf1fa11ab8e3b03bb9efac7dd7b58
tox now always recreates an env although the env is shared using envdir
options.
~~~
$ tox -e genpolicy
genpolicy: recreate env because env type changed from
{'name': 'genconfig', 'type': 'VirtualEnvRunner'} to
{'name': 'genpolicy', 'type': 'VirtualEnvRunner'}
~~~
According to the maintainer of tox, this functionality is not intended
to be supported.
https://github.com/tox-dev/tox/issues/425#issuecomment-1011944293
Change-Id: Ibbb4ab56ca84f5a8f9fe309a46469d49ff4ef9f7
Most code uses convert_version_to_tuple() from
oslo_utils.versionutils to determine minimum version
numbers, but there were two places that used the
packaging.version class instead. Change to always
use the same code throughout the tree.
Also added a flake8 enforcement check for it so we
don't regress.
TrivialFix
Change-Id: Ida4dcd504562646f0a450160e57680a44c387b1d
Move the ``BaseOVSTestCase`` class tests to the stestr executor with
concurrency=1. That will prevent that the minimum bandwidth tests
interfere among them.
Closes-Bug: #2025740
Change-Id: Ia3741abe5fa0c8ff65cf36d1ec31c089a1759f05
It has been over 2 years since we set the version of
pylint and associated tools in tox.ini, update to
something newer, specifically:
bashate>=2.1.1
bandit>=1.7.5
flake8-import-order==0.18.2
pylint==2.17.4
This required disabling additional checks for new
warnings that were found, and adding a directive to
add the "no-self-use" plugin that is now required
with the newer version.
Will work on fixing new warnings and re-enabling
checks in subsequent patches.
Trivialfix
Change-Id: I5d74d47ac1d8a085b447ff531d7abbecfb4cce6e
Metadata agent has been experiencing intermittent failures
mostly because of test conccurency and how the metadata agent
code assumes its the only process running on the system and
operating on the ovnmeta-* namespaces. See comment#1 the
linked bug for more details. Although I dont like forcing
--concurrency 1 for this test class, I think that is going
to be the best solution and any new tests that will be added
in the future.
Closes-Bug: #2011377
Change-Id: Ie7f3b496de6b23be5739fbeba10f53602e8b300d
tox4 even when running specific env with -e option tries
to find python version for all envs defined in envlist[1]
when running with --skip-missing-interpreters=false and
fails if interpreter missing for any of the env.
With py38 in envlist it tries to find python3.8
which is not available in ubuntu-jammy by default
and fails, hence use generic version instead i.e py3
to handle it.
Once [1] is fixed we can update the list as per
default supported py3 version for a release.
[1] https://github.com/tox-dev/tox/issues/2811
Change-Id: I3afc539c59f223338920d7fd59f0df76e842ece1
Since [1], the functional tests are being executed in Ubuntu 22.04.
The test "test_get_all_devices" is failing frequently, maybe because
other tests are interfering and modifying the root namespace
interface list.
Because this test cannot be executed inside a temporary namespace,
this patch moves the execution of this test to the section of
"concurrency=1". That will ensure no other test interferes the
result.
Closes-Bug: #1997341
Change-Id: If0631bf0cdfd5f0989c60d40e31f89cee317e497
--black-regex has been deprecated and using it can sometimes
cause failures in the fullstack job.
Change-Id: Iac1d4c4004e1348ced388ee4dd44711f0e27bcc2
Closes-bug: #1995901
In the neutron gate we want to run functional tests in 2 steps:
* all tests except MySQL and PostgreSQL tests,
* MySQL and PostgreSQL tests runs serially
And that was done like that in the dsvm-functional tox environment.
But there is no really point to run it as such always when someone
e.g. runs some tests from the functional tests suite locally, using
dsvm-functional environment.
So this patch adds dsvm-functional-gate environment which will be used
in the Zuul jobs and will have this 2 steps configuration.
Regular dsvm-functional env will run all tests together.
Change-Id: Ic66d9961393d9dff64ebd14802ad38c18808b1da
Similar to [1], this patch moves the PostgreSQL tests to a second
test execution step, where all tests are executed with one single
worker, avoiding any parallel execution on the SQL tests.
[1]https://review.opendev.org/c/openstack/neutron/+/834746
Closes-Bug: #1980127
Related-Bug: #1962594
Change-Id: I3a9f665653c3e2a5f75175867a45391e6ad01176
Master has dropped py3.6 support so upper-constraints
requirements are not installable with python3.6
and due to this fips jobs which are running on py3.6
are failing. Let's run the functional/fullstack tests
with python3.8 on CentOS 8-Stream itself.
Also disable dbcounter installation as it's not
installing on python3.6.
The hacks can be cleaned up when these jobs are
switched to CentOS 9-Stream, currently have some
issues as mentioned in the Related Bug.
Depends-On: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/843989
Related-Bug: #1976323
Change-Id: I757dbf929cb1dc27027614bbb2147a61606f85a9
Since we use usedevelop=True, as a result neutron
installation happens at 'develop-inst' phase and it installs
requirements from requirements.txt without considering
tox constraints, This can lead to issue where broken
requirements are installed in docs job and as a result
it will fail.
In order to get constraints be considered for this case, add
requirements.txt to deps for docs env.
Change-Id: Ic8f6472fe708d12f8fc129729465367083e1095a
This change in the "dsvm-functional" job definition will avoid the
parallel execution of any MySQL test.
Related-Bug: #1962594
Change-Id: I4fe6a6920b9b2711563e85b51f18a9d5a9086ceb
Functional SQL tests (MySQL and PostgreSQL) use the DB installed
backend. To avoid overloading the DB engine, the test cases are
grouped [1] together and executed by the same worker process.
That will ensure those tests are not executed in parallel,
interfering each other.
All test classes ended in "MySQL" or "PostgreSQL" are now grouped.
That means the tests implemented inside the same class are
executed in the same worker, not in parallel. **HOWEVER**, that
doesn't guarantee all test cases using the same database backend
will run in the same worker serially. The FT job can still execute
test cases from different classes, using the same DB backend,
in two workers at the same time.
This patch also fixes the ``TestReservation*`` classes to inherit
from the parent classes in the correct order. That sets the
"DRIVER" value to the expected one "mysql" or "postgresql".
[1]https://review.opendev.org/c/openstack/neutron/+/785568
Closes-Bug: #1962594
Related-Bug: #1687027
Change-Id: I6bca7daf4d3e46071686bb62d5988ae314af8e7e
UPPER_CONSTRAINTS_FILE is old name and deprecated
This allows to use upper-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.
Change-Id: Iae53ccf077796eb0d2518b41d0e262d564e7af10
In order to mitigate the OOM problems in the CI, this patch reduces
the fullstack concurrency to 2. That should reduce the memory
consumption and will avoid those nasty OOM exceptions.
Closes-Bug: #1938455
Change-Id: I4de4b089e615353b7fde82bd54e0abefa244c8ab
Since pyroute 0.6.2, this dependency is not needed anymore. The
pyroute version is bumped both in requirements and doc/requirements.
This patch bumps the pyroute2 version to 0.6.4 (newest version
provided in upper-requirements).
Change-Id: I627e902e32fe59c6829c3bc92af19abff4b70f9a
Related-Bug: #1928913
The assertItemsEqual has been replaced with assertCountEqual
since I7c20fec08e5dc9f67b34100c925ea6724bbd25f0 .
However the hacking check has not been enabled.
This patch enables the hacking check
and replace remaining assertItemsEqual with assertCountEqual.
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Change-Id: I86202f36f1b7c0bce8b29558ad1e377c336b7258
Mitogen is dependency of the pyroute2 and was added link to it
the docs in pyroute2 0.6.1. Due to the missing mitogen in our
docs venv, docs job was failing.
So let's install mitogen as docs env requirement, at least for now
to unblock our gate.
Later we can think about better solution as e.g. adding it properly to
the requirements or report and fix the issue in the pyroute2.
Change-Id: Iba37581916842bb5fc9f42f7f483d3431747bf8d
Closes-Bug: #1928913
Functional SQL tests (MySQL and PostgreSQL) use the DB installed
backend. To avoid overloading the DB engine, the test cases are
grouped [1] together and executed by the same worker process. That
will ensure those tests are not executed in parallel, interfering
each other.
Classes that are grouped:
- TestModelsMigrationsMysql
- TestModelsMigrationsPsql
[1]https://stestr.readthedocs.io/en/latest/MANUAL.html#group-regex
Related-Bug: #1687027
Change-Id: If04e360c01cdb16e956d16357e2181db55445137
If there is too much output produced to the stdout/stderr during the
tests, stestr can hangs and job will then time out.
To avoid that in fullstack job, this patch is doing the same what
was already done for functional job as well - set PYTHONWARNINGS
variable to "ignore" to ignore all warnings and not send them to stdout.
All info really needed to debug tests is still in the tests' logs in
specific directories.
Change-Id: I080381cb8a88a27130ac21c0e27fc915e38bdedd
Fullstack job is using different tox env in the gate
(dsvm-fullstack-gate). To reduce memory usage in that job, some
time ago, in patch [1] we reduced number of test workers from 4
to 3. But it was done for "dsvm-fullstack" tox env, not for
"dsvm-fullstack-gate" thus it had no effect on our CI runs.
Now it is changed also in the CI job's tox env.
[1] https://review.opendev.org/c/openstack/neutron/+/764907
Change-Id: Iec949922be0779a00bf405c2f1d5646c588d323c
Related-Bug: #1906366
After commit 89e2fad8ef856f0e3651e4ea5784d28bc9aeb229,
pep8/bandit/bashate tox envs have different dependencies than
the default tox env. If tox envs with different dependencies share
some common envdir, tox env will be recreated every time a different
tox env is specified. (For example, "tox -e cover" after "tox -e pep8"
recreates the tox env.)
To address it, this commit introduces a new common envdir for linters
(envdir = {toxworkdir}/lint). It also moves deependencies of bashate
and bandit envs to pep8 env to ensure these three tox envs have the
same dependencies.
Cleanups are also made in test-requirements and lower-constraints.
* astroid and isort can be dropped from test-requirements.txt
as pylint has proper version cappings for astroid and isort.
(These cappings are available in pylint 2.5 and later.)
* reno is a document dependency, so it is unnecessary in test-requirements.
* reno, astroid and isort are dropped from lower-constraints
as they are not installed in lower-constraints tests.
Change-Id: Iaa3168764d050875e3d08784aee2dfba809e53be
Another item noted with the new pip resolver [1], linters dependencies
in test-requirements.txt may cause resolver issues (trying to pull in
enum34), conflicting requirements and cause them to be installed for all
test jobs. Move them to tox.ini as was done for some projects already
(this may be backported/squashed with pip resolver fix in stable
branches depending on how fixing these will go).
[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019362.html
Change-Id: I0111c41bea6a6caf5ffba1f5c34489854d9c9747
To limit usage of resources (mostly memory) in the fullstack job and
to avoid oom killer to kill e.g. mysqld service, this patch:
* Makes number of API workers changeable by tests, as a parameter to
EnvironmentDescription and defaults its value to 1. As neutron server
is spawned separately for each test and is used only to process just
few API requests during that single test so this should be still
enough there, and where more API workers are needed (like some dhcp HA
tests: TestDhcpAgentHARaceCondition) it can be changed,
* reduces number of test run workers from 4 to 3 - job will run slower
but hopefully more stable,
* in the functional and fullstack tests job definition disable etcd3
service - this will not save us a lot of memory but still it's not
needed at all so why to run it there.
Change-Id: If19803ab6db144e2d17d6805d379c1c76d8fa343
Closes-Bug: #1906366
Bump astroid test requirement to 2.4.0
Older versions trigger an error on wrapt dependency:
https://github.com/PyCQA/astroid/issues/755
Bump pylint accordingly to new astroid
Fix some new PEP8 warnings appearing with new versions, and filter out
the larget I202 "Additional newline in a group of imports" one for now
Drop psutil from functional requirements, it indicated an old version
and we have it in common requirements now
Bump a series of lower-constraints and requirements to work with new pip
resolver, testing with steps outlined at:
http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019285.html
This includes eventlet 0.22.1, previous versions triggered a hard to
track error on enum34
Cap cryptography in lower-constraints to prevent discovery failure in
relevant job (other jobs have it capped via upper-constraints)
Change-Id: Ie74ea517a403e6e2a7a4e0a245dd20e5281339e8
Closes-Bug: #1907242
The docs and releasenotes requirements migrated to doc/requirements.txt
we need not install things from requirements.txt.
Change-Id: I207be0b861c13bc6cf954b2247bb00546990bbf0
