This has introduced a new version of pycodestyle, which has improved its
parser to catch new corner cases and introduced some new tests.
Closes-Bug: #1878317
Change-Id: I853cf4dbd7ad6b7903a7f444f5c1da3f0fb21f77
mypy is an experimental optional static type checker for Python that
aims to combine the benefits of dynamic (or "duck") typing and static
typing. While still in development, most features are supported and it's
already being used by real world projects like Sphinx. Let's start small
by integrating it into some of the interfaces that nova exposes. We can
further build upon this if it works out.
This change sets up the boilerplate necessary to use mypy in nova. Type
annotations are not included for any module - these will be added
separately. We're calling mypy by way of a script, as this allows us to
store a list of files that we have converted while we're in the process
of adding type annotations where necessary.
Change-Id: I75ab46a6768c4ca2050fdde2b7f8eeb90724c8c6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
hacking 3.0.1 fix the pinning of flake8 to avoid bringing in a new
version with new checks.
bumping the min version for hacking so that any older hacking versions
which auto adopt the new checks are not used.
Depends-On: https://review.opendev.org/#/c/728335/
Change-Id: Ie00c10332bd7110169dbb150d601c157b6694d05
It is possible, though unlikely, that hacking might introduce new rules
in a minor version that would break the gate for nova. See this off
before it can ever be an issue by doing $subject.
Change-Id: I063488a2ad4924ffe2a48cbfc0c3c750221d37b7
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This bumps the version of flake8 and pycodestyle to something much
newer, which resolves a long-standing warning about nested sets and
allows us to use new fangled features like f-strings if we so choose.
Change-Id: I0bb9077f1cea2243b7945e87cfa140f9cf89d558
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Finish the job by removing all the now-unused modules. This also allows
us to - wait for it - kill mox at long last. It's a great day in the
parish.
Partial-Implements: blueprint remove-nova-network-ussuri
Partial-Implements: blueprint mox-removal-ussuri
Change-Id: Ia33ec2604b2fc2d3b6830b596cac669cc3ad6c96
This reverts commit 33a2a5f2e4. We're now
at M1 and the DevStack default has been changed. We no longer need to
pretend to support Python 2.7. This is slightly modified to drop an
additional Python 2.7-only dependency from lower-constraints.txt.
[1] I52b03caee0ba700da3a15035201ea6cd91baa06b
Change-Id: I1068850072e639ee268d68ca82249f569a40310e
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Revert part of the changes from
Ie1a0cbd82a617dbcc15729647218ac3e9cd0e5a9
This way it is possible again for nova to be installed
using python 2, thus allowing other devstack-based
jobs to work until everything switches to python 3
according the plans.
While py2 support may break anytime, its days are limited anyway.
Change-Id: I99ff84e6c1c033c28d499b0b8ac17ac5b0565f61
It's Ussuri. We can *finally* stop testing Python 2 [1]. Time to party.
We don't attempt any cleanup but simply stop testing with Python 2,
indicate that we only support Python 3 via 'setup.cfg' and remove any
Python 2 only dependencies. Our 'tox.ini' is modified such that
'functional' now runs with 'python3', whatever that may point to, though
the gate will only use a versioned variant (currently
'functional-py36').
This should free up a significant amount of resources from the gate and
let us start using Python 3 idioms in our code. Win-win.
[1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html#python2-deprecation-timeline
Change-Id: Ie1a0cbd82a617dbcc15729647218ac3e9cd0e5a9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The following file should have been removed
in I4181f39dea7eb10b84e6f5057938767b3e422aff.
* etc/nova/placement-policy-generator.conf
A description of generating the placement policy.yaml file
should also have been removed in the commit.
Therefore remove them.
And the 'wsgi-intercept' package is used for OSAPIFixture and
not for placement functional tests currently.
So a comment is removed in test-requirements.txt.
Change-Id: Ie8d0d7085ccd82a47ea27b98be75c1bc9ba6195b
By default, mock.patch's autospec argument is None, meaning that
there's no signature checking for the patched methods and functions.
oslotest.mock_fixture.patch_mock_module fixes a few issues within
mock.patch functions, as well as setting autospec=True by default,
unless otherwise specified or new_callable, create, spec arguments
are passed in.
Co-Authored-By: Claudiu Belu <cbelu@cloudbasesolutions.com>
Change-Id: I4484e63c97bd1cdde3d88855eabe7545784f365e
Closes-Bug: #1735588
Several existing tests patch open() to fake the contents of a file
outside the test virtualenv, whilst avoiding interfering with reading
and writing of other files inside the test virtualenv. Currently they
do it by duplicating logic.
Furthermore, in the near future, more tests (specifically, some SEV
tests) will want to do the same selective patching, and similarly will
need to avoid impacting reads of other files within the test
virtualenv, e.g. placement-policy.yaml.
So create new patch_open() context manager / decorator in nova.test
for selectively patching open based on the path parameter, and reuse
this for existing tests.
Also add unit tests for all these cases.
mock >= 3.0.0 is required because configparser.RawConfigParser._read()
uses enumerate() to iterate over the lines of the (mocked)
/etc/nova/release config file, and this uses __iter__() under the hood
which was not supported via mock_open until a bug was fixed and
backported to the external mock library for 3.0.0:
https://bugs.python.org/issue21258https://bugs.python.org/issue3293373f6eed0d6
Change-Id: I19f49c923d2c41eab0c7b4cab28c50498dc07046
Several existing tests patch os.path.exists() to return a given value,
whilst avoiding interfering with other checks for path existence
inside the test virtualenv. Currently they do it by duplicating
logic.
Furthermore, in the near future, more tests (specifically, some SEV
functional tests) will want to do the same selective patching, and
similarly will need to avoid impacting existence checks on other files
within the test virtualenv, e.g. placement-policy.yaml.
TestGuestConfigSysinfoSerialOS already had a nice patch_exists()
context manager for selectively patching os.path.exists based on the
path parameter, so move this to nova.test.TestCase so that it can be
reused to reduce duplication, and add appropriate unit test cases.
In order to make patch_exists() generic, the path has to be introduced
as an extra parameter alongside the desired return value.
Additionally make it usable as a decorator, not just a context
manager. This is achievable natively in Python 3 with contextlib, and
in Python 2 via contextlib2.
Change-Id: Ibe7cb29620f06d31059f2a5f94ca180b8671046e
This brings in a couple of new checks which must be addressed, many of
which involve a rather large amount of changes, so these are ignored for
now. A series of follow-up changes will resolved these.
'pycodestyle' is added as a dependency rather than it being pulled in
transitively. This is necessary since we're using it in tests.
Change-Id: I35c654bd39f343417e0a1124263ff31dcd0b05c9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
In the review of a similar change in placement [1], it was realized that
the nova lower-constraints tox job probably had the same problems.
Testing revealed this to be the case. This change fixes the job and
updates the related requirements problems accordingly.
The are two main factors at play here:
* The default install_command in tox.ini uses the upper_contraints.txt
file. When there is more than one constraints.txt they are merged and
the higher constraints win. Using upper and lower at the same time
violates the point of lower (which is to indicate the bare minimum
we are capable of using).
* When usedevelop is true in tox, the command that is run to install the
current projects code is something like 'python setup.py develop',
which installs a project's requirements _after_ the install_command has
run, clobbering the constrained installs. When using pbr,
'python setup.py install' (used when usedevelop is False) does not do
this.
Fixing those then makes it possible to use the test to fix the
lower-constraints.txt and *requirements.txt files, changes include:
* Defining 'usedevelop = False' in the 'lower-constraints' target and
removing the otherwise superfluous 'skipsdist' global setting to
ensure requirements aren't clobbered.
* Removing packages which show up in lower-constraints.txt but not in
the created virtualenv. Note that the job only runs unit tests, so
this may be incomplete. In the placement version of this both unit and
functional are run. We may want to consider that here.
* Updating cryptography. This version is needed with more recent
pyopenssl.
* Updated keystonemiddleware. This is needed for some tests which
confirm passing configuration to the middleware.
* Update psycopg2 to a version that can talk with postgresql 10.
* Add PyJWT, used by zVMCloudConnector
* Update zVMCloudConnector to a version that works with Python 3.5 and
beyond.
* Update olso.messaging to versions that work with the tests, under
Python 3.
* Adding missing transitive packages.
* Adjusting alpha-ordering to map to how pip freeze does it.
* setuptools is removed from requirements.txt because the created
virtualenv doesn't contain it
NOTE: The lower-constraints.txt file makes no commitment to expressing
minimum requirements for anything other than the current basepython.
So the fact that a different set of lower-constraints would be present
if we were using python2 is not relevant. See discussion at [1].
However, since requirements.txt _is_ used for python2, the
requirements-check gate job requires that enum34 be present in
lower-constraints.txt because it is in requirements.txt.
NOTE: A test is removed because it cannot work in the
lower-constraints context: 'test_policy_generator_from_command_line'
forks a call to 'oslopolicy-policy-generator --namespace nova' which
fails because stevedore fails to pick up nova-based entry points when
in a different process. This is because of the change to usedevelop.
After discussion with the original author of the test removal was
considered an acceptable choice.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-dev/%23openstack-dev.2019-03-05.log.html#t2019-03-05T13:28:23
Closes-Bug: #1822575
Change-Id: Ic6466b0440a4fe012731a63715cf5d793b6ae4dd
This is related to change I5f78d21c39ed2fd58d2a0f3649116e39883d5a2c
which depends on python-ironicclient change
I610836e5038774621690aca88b2aee25670f0262 which was released in
2.7.0. The former did not have a direct dependency or version
bump since it is being backported to stable branches where we
don't bump minimum required versions of dependencies.
Change-Id: I5edf350b29ef3bd1f103cdd4b0b997cb5ca32117
Related-Bug: #1818295
In preparation for stein, use 'endpoint' instead of 'ironic_url'
when calling get_client in order to remove the following warning:
WARNING ironicclient.client The argument "ironic_url" passed to
get_client is deprecated and will be removed in Stein release,
please use "endpoint" instead.
For reference:
In the python-ironicclient code, in the ironicclient/client.py#L24
TODO(vdrok): remove in Stein
[...]
('ironic_url',): 'endpoint',
Introduced in commit:
58c39b7a80
Change-Id: I1b3ce1955622c40b780c0b15ec7e09be3e8ace72
With a fix in wsgi-intercept 1.7.0 we can properly
use the PlacementFixture as a context manager to
simulate when placement is configured for a given
operation. This allows us to remove the ugly stub
that one of the tests in here had to rely on.
While in here, the CastAsCall fixture is removed
since we shouldn't rely on that in these tests
where we're trying to simulate the user experience.
Change-Id: I2074b45126b839ea6307a8740364393e9dddd50b
The Project Testing Interface [1] asks that we list requirements in
'doc/requirements.txt' and build docs by calling 'sphinx-build' directly
instead of via the 'build_sphinx' setuptool/distutils wrapper. Start
doing this.
[1] https://governance.openstack.org/tc/reference/project-testing-interface.html
Change-Id: If9342c8ea757b1735f2488db751008984fb33baf
Nova switched to using stestr for all it's in tree testing except for
the coverage job. This commit updates that job so it also uses stestr
and cleans up the last bits of testr setup left in tree. As part of
this change this moves all the one off per job configuration into the
tox job definitions instead of hiding it in either the .stestr.conf or
in a wrapper script.
Change-Id: I36e0e791d485b44641a1ed8770eca8e95aca19be
Use whereto to test that the redirect rules do what we expect. The
test coverage is a bit minimal for now, but should be useful as an
example of how it can be extended. The test files in the
openstack-manuals repo provide some other examples.
Depends-On: I36003d3525dcc9fef8e5f008ecabb2b2ac78088e
Change-Id: Ia14bfa6c89fd04844842d727d11ff8874ec7fc2a
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
python-subunit is not used directly anywhere
and it is dependency of both testrepository
and os-testr
(probably was used by some tox wrapper script before)
Change-Id: Ic2139ceb5a200c3b32bcc87e81fc30f208645771
There have been some major changes to how scheduling works in Nova
during the Pike and Queens cycles. This documents these design changes
so that this new, more complex workflow is clearly spelled out.
Co-Authored-By: Ed Leafe <ed@leafe.com>
Change-Id: I15121d8fe9b715c0aec39dee4bfdf25ced42b481
Gabbi 1.35 provides template variable susbtitution on the left hand side
of a JSONPath expression. That's used here to remove the need for static
fake uuids in the test data for allocation-candidates and the
SharedStorageFixture.
A couple of other clean ups to the use of gabbi, unrelated to the 1.35
update:
* It's possible to test the entire value in one response:
resources:
VCPU: 4
DISK_GB: 5
instead of:
resources.VCPU: 4
resources.DISK_GB: 5
This is handy when the root of the expression is calculated and
repeating that calculation is noisy.
* [?foo.bar=baz] is the more canonical form of [?(@.foo.bar=baz)]. The
former is built into jsonpath, the latter dips into the underlying
implementation.
Change-Id: Ie6b5e0186533581815d04411ed7b321948464bb6