This change extends the guest xml parsing such that
the source device path can be extreacted from interface
elements of type vdpa.
This is required to identify the interface to remove when
detaching a vdpa port from a domain.
This change fixes a latent bug in the libvirt fixutre
related to the domain xml generation for vdpa interfaces.
Change-Id: I5f41170e7038f4b872066de4b1ad509113034960
The PowerVM driver was deprecated in November 2021 as part of change
Icdef0a03c3c6f56b08ec9685c6958d6917bc88cb. As noted there, all
indications suggest that this driver is no longer maintained and may be
abandonware. It's been some time and there's still no activity here so
it's time to abandon this for real.
This isn't as tied into the codebase as the old XenAPI driver was, so
removal is mostly a case of deleting large swathes of code. Lovely.
Change-Id: Ibf4f36136f2c65adad64f75d665c00cf2de4b400
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Since we dropped support for python 2 [1], we no longer need to use the
mock library, which existed to backport py3 functionality into py2.
Change Ib44b5bff657c8e76c4f701e14d51a4efda3f6d32 cut over to importing
the stock mock, which must be done by saying::
from unittest import mock
...because if you say::
import mock
...you will be using the third party mock library instead, which may or
may not be installed.
This commit adds hacking check N371 to enforce the former.
[1] https://review.opendev.org/#/c/687954/
Change-Id: I71439580e80d33cff62aba807df2b35164a47cbe
These are currently non-voting since we don't care about this stuff for
Zed. It does get us ready for a 3.10-having future, however.
Change-Id: I7740dafd6523eca27fa4e725d7eaf8558e434779
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
When tox 'docs' target is called, first it installs the dependencies
(listed in 'deps') in 'installdeps' phase, then it installs nova (with
its requirements) in 'develop-inst' phase. In the latter case 'deps' is
not used so that the constraints defined in 'deps' are not used.
This could lead to failures on stable branches when new packages are
released that break the build. To avoid this, the simplest solution is
to pre-install requirements, i.e. add requirements.txt to 'docs' tox
target.
Change-Id: I4471d4488d336d5af0c23028724c4ce79d6a2031
Spotted this in a review recently. We don't want people using six
anymore.
Change-Id: Ie107a95bc06390ab519d3b3af9b07103a9a14316
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Install these via tox.ini when required. Note that we don't add them to
'test-requirements.txt' since packagers consume those and would be
forced to package these dependencies or modify this file.
A small bug in the tox file is corrected: you can't share an environment
directory if the dependencies in that directory are different. As such,
pep8 must go in its own directory again, not the 'shared' directory.
Change-Id: Iffb9ff2e300213ec01cccaf3b967e73331c9c9ff
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This is a follow up change to I168fffac8002f274a905cfd53ac4f6c9abe18803
which added a hackaround to enable our tests to pass with
fasteners>=0.15 which was upgraded recently as part of a
openstack/requirements update.
The ReaderWriterLock from fasteners (and thus lockutils) cannot work
correctly with eventlet patched code, so this adds a wrapper containing
the aforementioned hackaround along with a hacking check to do our best
to ensure that future use of ReaderWriterLock will be through the
wrapper.
Change-Id: Ia7bcb40a21a804c7bc6b74f501d95ce2a88b09b5
Yoga testing runtime is updated now
- https://governance.openstack.org/tc/reference/runtimes/yoga.html
which needs to test py38 and py39. Unit tests update are
handled by the job template change in openstack-zuul-job and
this commit makes changes to fucntional job to run py39 as voting
and updating the metdata in setup file.
Change-Id: I314fd61f20f2c3551f6231d191b7dcaaefabd2b5
Well, sort of. We enable them but immediately filter out the ones we're
actually seeing, the rationale being that we can address these in a
piecemeal fashion without the risk of introducing new issues.
There's a lot more to be done here. However, the work done in oslo.db
[1] should provide a guide for how to resolve the outstanding issues.
[1] https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+project:openstack/oslo.db
Change-Id: Iafe726d3819031c357460fd131bb2bb58babb4e2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
autopep8 is a code formating tool that makes python code pep8
compliant without changing everything. Unlike black it will
not radically change all code and the primary change to the
existing codebase is adding a new line after class level doc strings.
This change adds a new tox autopep8 env to manually run it on your
code before you submit a patch, it also adds autopep8 to pre-commit
so if you use pre-commit it will do it for you automatically.
This change runs autopep8 in diff mode with --exit-code in the pep8
tox env so it will fail if autopep8 would modify your code if run
in in-place mode. This allows use to gate on autopep8 not modifying
patches that are submited. This will ensure authorship of patches is
maintianed.
The intent of this change is to save the large amount of time we spend
on ensuring style guidlines are followed automatically to make it
simpler for both new and old contibutors to work on nova and save
time and effort for all involved.
Change-Id: Idd618d634cc70ae8d58fab32f322e75bfabefb9d
We have placement-nova-tox-functional-py38 job defined and run
on placement gate[1] to run the nova functional test excluding
api and notification _sample_tests, and db-related tests but that
job skip those tests via tox_extra_args which is not right way
to do as we currently facing error when tox_extra_args is included
in tox siblings task
- c02c28a982
- https://zuul.openstack.org/build/a8c186b2c7124856ae32477f10e2b9a4
Let's define a new tox env which can exclude the required test
in stestr command itself.
[1] bd5b19c00e/.zuul.yaml (L83)
Change-Id: I20d6339a5203aed058f432f68e2ec1af57030401
As the bug and fix If71620e808744736cb4fe3abda76d81a6335311b showed
it is dangerous to forget instantiating the Mock class before it is
used in the test as changes on the class directly leaks out from the
test. In almost all the cases using Mock class directly is a bug and the
author original intention is to use an instance instead, just forgot
about the parents. So this patch adds two new hacking rules:
N367: catches the case when Mock class is aliased in the test:
self.mock_mystuff = mock.Mock
N368: catches when mock.patch instructed to use the Mock class as
replacement value during patching:
mock.patch('Bar.foo', new=mock.Mock)
For N367 the previous patch removed the last hit. For N368 this patch
removes the two hits exists.
Change-Id: Id42ca571b1569886ef47aa350369e9d2068e77bc
Related-Bug: #1936849
Some things that were missed in previous patches and are thrown together
here:
- Add alembic as an explicit dependency (we were getting it transitively
from oslo.db). We also bump the sqlalchemy dependency to a 1.4.x
release, which is the minimum supported by our chosen version of
alembic (more on this below)
- Remove tooling related to the old migrations
- Fix the tox whitelisting of the flaky MySQL tests
On the SQLAlchemy front, we opt for 1.4.13. Technically alembic should
support anything from 1.4.0, however, with SQLAlchemy >= 1.4.0, < 1.4.13
we see errors like the following in some tests:
sqlalchemy.exc.InvalidRequestError: Entity namespace for
"count(instance_mappings.id)" has no property "queued_for_delete"
There's nothing specific about this in the release notes for 1.4.13 [1]
but it definitely fixes things.
[1] https://docs.sqlalchemy.org/en/14/changelog/changelog_14.html#change-1.4.13
Change-Id: I4c8eb13f11aa7471c26a5ba326319aef245c9836
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This reverts commit 89e321b7b5c88ab7cfab53103295774ca535f0df.
Reason for revert: The previous change require develper to install multiple spcific version python
locally to run the test envs and it arbitrally pins lower constratins to the
oldest python we support but lower contratis shoudl work with the newwst python we support too.
on fedora or other operating systems that ship with default python version that exceed our supported
versions test can be run with tox -e py38 if required.
similarly our updated constratins shoudl work with the older python.
Change-Id: I358ff8128746c9d0c584a03381e55817c4aace3f
Declare specific versions of Python for environments that don't
specifically encode a version. We use the highest version officially
supported, Python 3.8, for most environments except lower-constraints,
which uses the lowest version, Python 3.6. This is necessary for users
on Fedora, where the default Python version is Python 3.9.
This will unfortunately introduce a small amount of busy work whenever
we change supported Python versions, as tox will need to be updated, but
that's a small price to pay for the usability wins that Fedora users
will see.
Change-Id: I240d5b8aa3eb1925b97af57e2644be61bfc106c2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/801773
This currently runs in the 'check' pipeline, as part of the pep8 job,
which causes otherwise perfectly valid backports to report as failing
CI. There's no reason a stable core shouldn't be encouraged to review
these patches: we simply want to prevent them *merging* before their
parent(s). Resolve this conflict by moving the check to separate voting
job in the 'gate' pipeline as well as a non-voting job in the 'check'
pipeline to catch more obvious issues.
Change-Id: Id3e4452883f6a3cf44ff58b39ded82e882e28c23
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Just removing mypy and then we will revert this patch once
I035d6dc752eaa83105cc12797765b304d843e1f7 is merged.
Change-Id: Iaa02e9d35419bf7100cef1014217210c1d5b6855
I4f551dc4b57905cab8aa005c5680223ad1b57639 introduced the environment
variable to disable the check-cherry-pick.sh script but forgot to allow
it to be passed into the pep8 tox env.
Change-Id: Ie8a672fd21184c810bfe9c0e3a49582189bf2111
This significantly speeds up our doc build process. This requires a
newer version of 'sphinx-feature-classification' and some tweaks to our
own in-tree extensions. While we're here, we drop the '-d DOCTREE_DIR'
parameter since it's of no use when we blast away our previously built
docs each time we build.
Change-Id: I679da65d44c40880f720df8a2f06286a19eb8d22
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Introduce API microversion 2.88, which makes the following changes to
a number of 'os-hypervisors'. Specifically, the following fields are
dropped from both the '/os-hypervisors/detail' (detailed list) and
'/os-hypervisors/{hypervisor_id}' (show) APIs:
- current_workload
- cpu_info
- vcpus
- vcpus_used
- free_disk_gb
- local_gb
- local_gb_used
- disk_available_least
- free_ram_mb
- memory_mb
- memory_mb_used
- running_vms
In addition, the '/os-hypervisors/statistics' API, which provided a
summary of the above stats but for all hypervisors in the deployment, is
dropped entirely.
Finally, the '/os-hypervisors/{hypervisor}/uptime' API, which provided a
similar response to the '/os-hypervisors/{hypervisor}' API but with an
additional 'uptime' field, has been removed in favour of including this
field in the primary '/os-hypervisors/{hypervisor}' API.
A small tweak to 'tox.ini' that allows us to share some venvs is
included.
Part of blueprint modernize-os-hypervisors-api
Change-Id: I515e484ade6c6455f82a3067940a418a0d7d965a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
In a heavily IO deprived CI VM the db migration tests could take a
significant amount of time and eventually time out. This patch moves the
tests into the same test executor worker process to spread the load
generated by these test in time until a final solution is found. For
example we hope that [1] will help eventually to decrease the load.
[1] https://review.opendev.org/q/topic:bp/compact-db-migrations-wallaby
Change-Id: I6ce930fa86c82da1008089791942b1fff7d04c18
Related-Bug: #1823251
Nova does not run gabbi test since placement is moved to a separate
git repository. So the gabbi related tox.ini comment is removed.
Change-Id: Ic324e3e32fa03478895b32fa583e805ee6c721e2
This patch adds both a tox target and a zuul job to run functional tests
with python3.9
Depends-On: https://review.opendev.org/760932
Change-Id: I672904e9bfb45a66a82331063c7d49c4bc0439df
This linting makes building docs an even more painful process than it
would otherwise be. We already do this as part of the pep8 target,
which the gate runs, so there's no need to do this here.
Change-Id: I36ae872dd21299ad5d165422cb83564eafd89bea
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Add the following hacking rule.
* N366: The assert_has_calls is a method rather than a variable.
Not correct: mock_method.assert_has_calls = [mock.call(0)]
Correct: mock_method.assert_has_calls([mock.call(0)])
This patch is a follow-up patch for
Id094dd90efde09b9a835d4492f4a92b8f8ad296e.
Change-Id: I892f8c23ee44f2b3518776a9705e3543f3115cae
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
These will not be used in a world without the XenAPI driver.
Change-Id: I5bc3c7855b817c4ce2b8919c76be80cceabeec9e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The Python 2.7 Support has been dropped since Ussuri.
So remove hacking rules for compatibility between python 2 and 3.
- [N325] str() and unicode() cannot be used on an exception.
Remove or use six.text_type()
- [N327] Do not use xrange(). xrange() is not compatible with Python 3.
Use range() or six.moves.range() instead.
- [N344] Python 3: do not use dict.iteritems.
- [N345] Python 3: do not use dict.iterkeys.
- [N346] Python 3: do not use dict.itervalues.
See also line 414 in https://etherpad.opendev.org/p/nova-victoria-ptg
Change-Id: If4335b2e8ef5bbabba37598110c1aa8269635c2f
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
This adds a tools/ script that checks any cherry-picked hashes
on the current commit (or a provided commit) to make sure that
all the hashes exist on at least master or stable/.* branches.
This should help avoid accidentally merging stable backports
where one of the hashes along the line has changed due to conflicts.
Change-Id: I4afaa0808b75cc31a8dd14663912c162281a1a42
The N319 check previously asserted that debug-level logs were not
translated. Now that we've removed all log translations, we can
generalize this to all logs. We reuse the same number since these
numbers are really just metadata and not public contracts.
This also allows us to update the N323 and N326 checks, which ensure we
import the translation function, '_', wherever it's used and don't
concatenate translated and non-translated strings. Since we're no longer
translating logs and the '_LE', '_LW' and '_LI' symbols are no longer
provided, we don't need to consider logs in either of these cases.
Change-Id: I64d139ad660bc382e8b9d7c8cd03352b26aadafd
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
mypy is an experimental optional static type checker for Python that
aims to combine the benefits of dynamic (or "duck") typing and static
typing. While still in development, most features are supported and it's
already being used by real world projects like Sphinx. Let's start small
by integrating it into some of the interfaces that nova exposes. We can
further build upon this if it works out.
This change sets up the boilerplate necessary to use mypy in nova. Type
annotations are not included for any module - these will be added
separately. We're calling mypy by way of a script, as this allows us to
store a list of files that we have converted while we're in the process
of adding type annotations where necessary.
Change-Id: I75ab46a6768c4ca2050fdde2b7f8eeb90724c8c6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
As per Victoria testing runtime[1], we need to tests py3.6,
py3.7, and py3.8.
- py3.7 is being tested with integration jobs.
- py3.6 and py3.8 are tested with unit test jobs.
Nova functional tests are testing py3.6 which can be moved to the latest
python available in this cycle which is 3.8. We do not need to run functional
or integration tests on each supported python version. Testing them on
the latest python version is enough. Coverage of all supported python version
is achieved via unit tests job.
[1] https://governance.openstack.org/tc/reference/runtimes/victoria.html
Change-Id: I1d6a2986fcb0435cfabdd104d202b65329909d2b
nova-tox-functional is py2 job and need placement as required project.
and started failing for incompatible py version:
ERROR: Package 'openstack-placement' requires a different Python: 2.7.17 not in '>=3.6'
- https://zuul.opendev.org/t/openstack/build/b460a8c59ad64e57b871aa3c00638b01/log/job-output.txt#782
Placement master is now py3 only which mean master version will stop
working on py2 env:
- I6f458fb60b5a33b5aa2ce3ab292862ab98eb4670
Current tox env mention the placement master link as deps, where need to use
the stable version for stable jobs. Making deps with version not the
fresh git clone like how other test-requirements.txt deps work.
Change-Id: I96bb9fcb55f2a4fc3aefa01db70b45740db166c8
Add the validation framework necessary to verify extra specs along with
the definitions for every extra spec we currently recognize in-tree.
None of this is currently used since we don't have the API microversions
wired up, but that will come in a future patch.
Note that we must add the H238 hacking check to the ignore list here,
since this includes our first use of Python 3-type classes without the
explicit 'object' subclass. This can be removed when that check is
removed from hacking.
Part of blueprint flavor-extra-spec-validators
Change-Id: Ib64a1348cce1dca995746214616c4f33d9d664bd
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This bumps the version of flake8 and pycodestyle to something much
newer, which resolves a long-standing warning about nested sets and
allows us to use new fangled features like f-strings if we so choose.
Change-Id: I0bb9077f1cea2243b7945e87cfa140f9cf89d558
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
It's Ussuri. We can *finally* stop testing Python 2 [1]. Time to party.
We don't attempt any cleanup but simply stop testing with Python 2,
indicate that we only support Python 3 via 'setup.cfg' and remove any
Python 2 only dependencies. Our 'tox.ini' is modified such that
'functional' now runs with 'python3', whatever that may point to, though
the gate will only use a versioned variant (currently
'functional-py36').
This should free up a significant amount of resources from the gate and
let us start using Python 3 idioms in our code. Win-win.
[1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html#python2-deprecation-timeline
Change-Id: Ie1a0cbd82a617dbcc15729647218ac3e9cd0e5a9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Change Iba797243d2a137b551223165a1af1a8676bcea02 was a bit
overzealous in using {[testenv]deps} and changed the docs
tox target to also install requirements.txt and
test-requirements.txt which means for docs builds we're
installing things like psycopg2 which we shouldn't be doing
if we don't have the correct native packages installed to
build those types of dependencies.
Change-Id: Ib718911596b93ec6ec7e899210300d2f0d9572ed
Closes-Bug: #1849870
Ieb4ab13cf8ca5683fcd7b18ed669e8a26659bff1 removed the upper-constraints
from the install_command which caused that only the test-requirements
are installed with the upper-constraints enforced. This caused that when
tox installed nova in the virtual env it installed the content of the
requirement.txt without enforcing the upper-constraints. Today networkx
2.4 package has been released to pypi. The taskflow lib depends on
networkx but does not pin the requirement but the openstack
upper-constraints pins the networkx requirements properly. Nova depends
on taskflow therefore when nova is installed by tox without the
upper-constraints the new networkx 2.4 is installed. This broke the nova
unit tests.
This patch makes sure that all the requirements are installed with the
upper-constraints enforced.
Change-Id: Iba797243d2a137b551223165a1af1a8676bcea02
Closes-Bug: #1848499
