Commit Graph

187 Commits

Author SHA1 Message Date
zhangbailin
61b99a1295 Bump hacking min version to 3.1.0
This has introduced a new version of pycodestyle, which has improved its
parser to catch new corner cases and introduced some new tests.

Closes-Bug: #1878317
Change-Id: I853cf4dbd7ad6b7903a7f444f5c1da3f0fb21f77
2020-05-19 08:09:43 +08:00
Stephen Finucane
34ecf5ab91 tox: Integrate mypy
mypy is an experimental optional static type checker for Python that
aims to combine the benefits of dynamic (or "duck") typing and static
typing. While still in development, most features are supported and it's
already being used by real world projects like Sphinx. Let's start small
by integrating it into some of the interfaces that nova exposes. We can
further build upon this if it works out.

This change sets up the boilerplate necessary to use mypy in nova. Type
annotations are not included for any module - these will be added
separately. We're calling mypy by way of a script, as this allows us to
store a list of files that we have converted while we're in the process
of adding type annotations where necessary.

Change-Id: I75ab46a6768c4ca2050fdde2b7f8eeb90724c8c6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2020-05-15 15:59:53 +01:00
Ghanshyam Mann
9f2e24a5a4 Bump hacking min version to 3.0.1
hacking 3.0.1 fix the pinning of flake8 to avoid bringing in a new
version with new checks.

bumping the min version for hacking so that any older hacking versions
which auto adopt the new checks are not used.

Depends-On: https://review.opendev.org/#/c/728335/

Change-Id: Ie00c10332bd7110169dbb150d601c157b6694d05
2020-05-15 02:36:37 +00:00
Stephen Finucane
233f2715dd requirements: Limit hacking to one minor version
It is possible, though unlikely, that hacking might introduce new rules
in a minor version that would break the gate for nova. See this off
before it can ever be an issue by doing $subject.

Change-Id: I063488a2ad4924ffe2a48cbfc0c3c750221d37b7
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2020-01-20 14:35:19 +00:00
Stephen Finucane
e3da87a45d Switch to hacking 2.x
This bumps the version of flake8 and pycodestyle to something much
newer, which resolves a long-standing warning about nested sets and
allows us to use new fangled features like f-strings if we so choose.

Change-Id: I0bb9077f1cea2243b7945e87cfa140f9cf89d558
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2020-01-17 11:30:40 +00:00
Stephen Finucane
f5f73b4c4e nova-net: Kill it
Finish the job by removing all the now-unused modules. This also allows
us to - wait for it - kill mox at long last. It's a great day in the
parish.

Partial-Implements: blueprint remove-nova-network-ussuri
Partial-Implements: blueprint mox-removal-ussuri

Change-Id: Ia33ec2604b2fc2d3b6830b596cac669cc3ad6c96
2020-01-14 21:25:56 +00:00
Stephen Finucane
3dbdbb4e2b Revert "(Temporarily) readd bare support for py27"
This reverts commit 33a2a5f2e4. We're now
at M1 and the DevStack default has been changed. We no longer need to
pretend to support Python 2.7. This is slightly modified to drop an
additional Python 2.7-only dependency from lower-constraints.txt.

[1] I52b03caee0ba700da3a15035201ea6cd91baa06b

Change-Id: I1068850072e639ee268d68ca82249f569a40310e
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-12-13 12:45:52 +00:00
Luigi Toscano
33a2a5f2e4 (Temporarily) readd bare support for py27
Revert part of the changes from
Ie1a0cbd82a617dbcc15729647218ac3e9cd0e5a9

This way it is possible again for nova to be installed
using python 2, thus allowing other devstack-based
jobs to work until everything switches to python 3
according the plans.

While py2 support may break anytime, its days are limited anyway.

Change-Id: I99ff84e6c1c033c28d499b0b8ac17ac5b0565f61
2019-11-19 15:38:33 +01:00
Stephen Finucane
14872caae1 Stop testing Python 2
It's Ussuri. We can *finally* stop testing Python 2 [1]. Time to party.
We don't attempt any cleanup but simply stop testing with Python 2,
indicate that we only support Python 3 via 'setup.cfg' and remove any
Python 2 only dependencies. Our 'tox.ini' is modified such that
'functional' now runs with 'python3', whatever that may point to, though
the gate will only use a versioned variant (currently
'functional-py36').

This should free up a significant amount of resources from the gate and
let us start using Python 3 idioms in our code. Win-win.

[1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html#python2-deprecation-timeline

Change-Id: Ie1a0cbd82a617dbcc15729647218ac3e9cd0e5a9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-11-18 10:31:06 +00:00
Takashi NATSUME
338a0603d2 Remove an unused file and a related description
The following file should have been removed
in I4181f39dea7eb10b84e6f5057938767b3e422aff.

* etc/nova/placement-policy-generator.conf

A description of generating the placement policy.yaml file
should also have been removed in the commit.
Therefore remove them.

And the 'wsgi-intercept' package is used for OSAPIFixture and
not for placement functional tests currently.
So a comment is removed in test-requirements.txt.

Change-Id: Ie8d0d7085ccd82a47ea27b98be75c1bc9ba6195b
2019-09-13 10:33:32 +09:00
Takashi NATSUME
1eec451b1b Tests: autospecs all the mock.patch usages
By default, mock.patch's autospec argument is None, meaning that
there's no signature checking for the patched methods and functions.

oslotest.mock_fixture.patch_mock_module fixes a few issues within
mock.patch functions, as well as setting autospec=True by default,
unless otherwise specified or new_callable, create, spec arguments
are passed in.

Co-Authored-By: Claudiu Belu <cbelu@cloudbasesolutions.com>
Change-Id: I4484e63c97bd1cdde3d88855eabe7545784f365e
Closes-Bug: #1735588
2019-08-19 09:43:10 -05:00
Zuul
36e7502879 Merge "Exclude broken ironicclient versions 2.7.1" 2019-06-03 19:44:11 +00:00
翟小君
11e7dda320 Exclude broken ironicclient versions 2.7.1
Depends-On:https://review.opendev.org/#/c/659612/

Change-Id: Ia887be9e61ab68ab5cf1586082f2791118fc9c93
2019-06-02 20:25:21 +08:00
Adam Spiers
ca8f4be2a8 Move selective patching of open() to nova.test for reuse
Several existing tests patch open() to fake the contents of a file
outside the test virtualenv, whilst avoiding interfering with reading
and writing of other files inside the test virtualenv.  Currently they
do it by duplicating logic.

Furthermore, in the near future, more tests (specifically, some SEV
tests) will want to do the same selective patching, and similarly will
need to avoid impacting reads of other files within the test
virtualenv, e.g. placement-policy.yaml.

So create new patch_open() context manager / decorator in nova.test
for selectively patching open based on the path parameter, and reuse
this for existing tests.

Also add unit tests for all these cases.

mock >= 3.0.0 is required because configparser.RawConfigParser._read()
uses enumerate() to iterate over the lines of the (mocked)
/etc/nova/release config file, and this uses __iter__() under the hood
which was not supported via mock_open until a bug was fixed and
backported to the external mock library for 3.0.0:

    https://bugs.python.org/issue21258
    https://bugs.python.org/issue32933
    73f6eed0d6

Change-Id: I19f49c923d2c41eab0c7b4cab28c50498dc07046
2019-05-25 02:49:40 +00:00
Adam Spiers
f0edbff6f2 Move patch_exists() to nova.test.TestCase for reuse
Several existing tests patch os.path.exists() to return a given value,
whilst avoiding interfering with other checks for path existence
inside the test virtualenv.  Currently they do it by duplicating
logic.

Furthermore, in the near future, more tests (specifically, some SEV
functional tests) will want to do the same selective patching, and
similarly will need to avoid impacting existence checks on other files
within the test virtualenv, e.g. placement-policy.yaml.

TestGuestConfigSysinfoSerialOS already had a nice patch_exists()
context manager for selectively patching os.path.exists based on the
path parameter, so move this to nova.test.TestCase so that it can be
reused to reduce duplication, and add appropriate unit test cases.

In order to make patch_exists() generic, the path has to be introduced
as an extra parameter alongside the desired return value.

Additionally make it usable as a decorator, not just a context
manager.  This is achievable natively in Python 3 with contextlib, and
in Python 2 via contextlib2.

Change-Id: Ibe7cb29620f06d31059f2a5f94ca180b8671046e
2019-05-24 16:22:48 +01:00
Stephen Finucane
3e65f778bd Bump to hacking 1.1.0
This brings in a couple of new checks which must be addressed, many of
which involve a rather large amount of changes, so these are ignored for
now. A series of follow-up changes will resolved these.

'pycodestyle' is added as a dependency rather than it being pulled in
transitively. This is necessary since we're using it in tests.

Change-Id: I35c654bd39f343417e0a1124263ff31dcd0b05c9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-04-12 16:23:49 +01:00
Chris Dent
e5269f2fbb Correct lower-constraints.txt and the related tox job
In the review of a similar change in placement [1], it was realized that
the nova lower-constraints tox job probably had the same problems.
Testing revealed this to be the case. This change fixes the job and
updates the related requirements problems accordingly.

The are two main factors at play here:

* The default install_command in tox.ini uses the upper_contraints.txt
  file. When there is more than one constraints.txt they are merged and
  the higher constraints win. Using upper and lower at the same time
  violates the point of lower (which is to indicate the bare minimum
  we are capable of using).

* When usedevelop is true in tox, the command that is run to install the
  current projects code is something like 'python setup.py develop',
  which installs a project's requirements _after_ the install_command has
  run, clobbering the constrained installs. When using pbr,
  'python setup.py install' (used when usedevelop is False) does not do
  this.

Fixing those then makes it possible to use the test to fix the
lower-constraints.txt and *requirements.txt files, changes include:

* Defining 'usedevelop = False' in the 'lower-constraints' target and
  removing the otherwise superfluous 'skipsdist' global setting to
  ensure requirements aren't clobbered.

* Removing packages which show up in lower-constraints.txt but not in
  the created virtualenv. Note that the job only runs unit tests, so
  this may be incomplete. In the placement version of this both unit and
  functional are run. We may want to consider that here.

* Updating cryptography. This version is needed with more recent
  pyopenssl.

* Updated keystonemiddleware. This is needed for some tests which
  confirm passing configuration to the middleware.

* Update psycopg2 to a version that can talk with postgresql 10.

* Add PyJWT, used by zVMCloudConnector

* Update zVMCloudConnector to a version that works with Python 3.5 and
  beyond.

* Update olso.messaging to versions that work with the tests, under
  Python 3.

* Adding missing transitive packages.

* Adjusting alpha-ordering to map to how pip freeze does it.

* setuptools is removed from requirements.txt because the created
  virtualenv doesn't contain it

NOTE: The lower-constraints.txt file makes no commitment to expressing
minimum requirements for anything other than the current basepython.
So the fact that a different set of lower-constraints would be present
if we were using python2 is not relevant. See discussion at [1].
However, since requirements.txt _is_ used for python2, the
requirements-check gate job requires that enum34 be present in
lower-constraints.txt because it is in requirements.txt.

NOTE: A test is removed because it cannot work in the
lower-constraints context: 'test_policy_generator_from_command_line'
forks a call to 'oslopolicy-policy-generator --namespace nova' which
fails because stevedore fails to pick up nova-based entry points when
in a different process. This is because of the change to usedevelop.
After discussion with the original author of the test removal was
considered an acceptable choice.

[1] http://eavesdrop.openstack.org/irclogs/%23openstack-dev/%23openstack-dev.2019-03-05.log.html#t2019-03-05T13:28:23

Closes-Bug: #1822575

Change-Id: Ic6466b0440a4fe012731a63715cf5d793b6ae4dd
2019-04-01 19:16:33 +00:00
Matt Riedemann
229101e6af Require python-ironicclient>=2.7.0
This is related to change I5f78d21c39ed2fd58d2a0f3649116e39883d5a2c
which depends on python-ironicclient change
I610836e5038774621690aca88b2aee25670f0262 which was released in
2.7.0. The former did not have a direct dependency or version
bump since it is being backported to stable branches where we
don't bump minimum required versions of dependencies.

Change-Id: I5edf350b29ef3bd1f103cdd4b0b997cb5ca32117
Related-Bug: #1818295
2019-03-12 15:10:11 -04:00
Daniel Abad
35f49f4035 Fix ironic client ironic_url deprecation warning
In preparation for stein, use 'endpoint' instead of 'ironic_url'
when calling get_client in order to remove the following warning:

WARNING ironicclient.client The argument "ironic_url" passed to
get_client is deprecated and will be removed in Stein release,
please use "endpoint" instead.

For reference:
In the python-ironicclient code, in the ironicclient/client.py#L24
TODO(vdrok): remove in Stein
[...]
  ('ironic_url',): 'endpoint',

Introduced in commit:
58c39b7a80

Change-Id: I1b3ce1955622c40b780c0b15ec7e09be3e8ace72
2018-10-24 17:59:10 +02:00
Matt Riedemann
ee6a257f58 Cleanup ugly stub in TestLocalDeleteAllocations
With a fix in wsgi-intercept 1.7.0 we can properly
use the PlacementFixture as a context manager to
simulate when placement is configured for a given
operation. This allows us to remove the ugly stub
that one of the tests in here had to rely on.

While in here, the CastAsCall fixture is removed
since we shouldn't rely on that in these tests
where we're trying to simulate the user experience.

Change-Id: I2074b45126b839ea6307a8740364393e9dddd50b
2018-05-16 09:23:55 -04:00
OpenStack Proposal Bot
2c19b5640e Updated from global requirements
Change-Id: I004b8ea8fd3a8de76c04f70af5ed5e516aa2bfbc
2018-03-23 07:40:46 +00:00
Stephen Finucane
c21ab6b134 Follow the new PTI for document build
The Project Testing Interface [1] asks that we list requirements in
'doc/requirements.txt' and build docs by calling 'sphinx-build' directly
instead of via the 'build_sphinx' setuptool/distutils wrapper. Start
doing this.

[1] https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: If9342c8ea757b1735f2488db751008984fb33baf
2018-03-16 14:39:36 +00:00
OpenStack Proposal Bot
0896a0ae28 Updated from global requirements
Change-Id: I518860c51e6b19def5efc3f624dceb9350cb9494
2018-03-16 07:34:06 +00:00
OpenStack Proposal Bot
4607f95f24 Updated from global requirements
Change-Id: Iff8f99d4ff97f03ccdeb29405f3cbb853068161d
2018-03-13 07:09:16 +00:00
OpenStack Proposal Bot
2d738f7082 Updated from global requirements
Change-Id: Ibe70f1f60180a7c8da9887e4466dd1e1d25b157f
2018-03-10 13:31:01 +00:00
OpenStack Proposal Bot
611a11297e Updated from global requirements
Change-Id: I371ee0cb4156ec4e0a20f55852b0756502d0dd90
2018-01-29 00:21:52 +00:00
Zuul
02852c8799 Merge "Updated from global requirements" 2018-01-27 15:29:24 +00:00
Zuul
233285de49 Merge "add whereto for testing redirect rules" 2018-01-25 06:48:57 +00:00
OpenStack Proposal Bot
dd1a776315 Updated from global requirements
Change-Id: I1fae25690551ff75a9838e760851a990e216be72
2018-01-24 01:14:00 +00:00
OpenStack Proposal Bot
bc899d8b9e Updated from global requirements
Change-Id: I1c1dc245f98cb5c3b57733ddc338394dfb3bc0b6
2018-01-18 03:11:57 +00:00
OpenStack Proposal Bot
0f2700b1e4 Updated from global requirements
Change-Id: Icada15d601dc5399eeb927bb7cba9bea828883da
2018-01-16 12:40:17 +00:00
Matthew Treinish
9c56c78513
Finish stestr migration
Nova switched to using stestr for all it's in tree testing except for
the coverage job. This commit updates that job so it also uses stestr
and cleans up the last bits of testr setup left in tree. As part of
this change this moves all the one off per job configuration into the
tox job definitions instead of hiding it in either the .stestr.conf or
in a wrapper script.

Change-Id: I36e0e791d485b44641a1ed8770eca8e95aca19be
2017-11-24 16:51:12 -05:00
OpenStack Proposal Bot
0de0681895 Updated from global requirements
Change-Id: I2ddeb4baef5e535923a8e16ef763782595df614d
2017-11-16 11:16:21 +00:00
Doug Hellmann
cec0faa711 add whereto for testing redirect rules
Use whereto to test that the redirect rules do what we expect. The
test coverage is a bit minimal for now, but should be useful as an
example of how it can be extended. The test files in the
openstack-manuals repo provide some other examples.

Depends-On: I36003d3525dcc9fef8e5f008ecabb2b2ac78088e
Change-Id: Ia14bfa6c89fd04844842d727d11ff8874ec7fc2a
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-10-30 12:37:23 -04:00
Jenkins
933dd29d74 Merge "cleanup test-requirements" 2017-09-26 17:44:17 +00:00
Eric Fried
f839a97e0f Live Migration sequence diagram
Based on mriedem's hand-drawn version [1] (but not as pretty).

[1] https://photos.google.com/share/AF1QipNpWVQKU8GK4_9wxVbiRJUqJnMzqPcBh6DvjVyBPIjjmi6ZU8r9TleQNo6pV1t9SA?key=NUl3OUFGYkRFTE8tMHhSX0lfc0Y1eEdoeHo4SUhn

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Change-Id: I63046079cd3135b4b19c0c6745075f090d04e396
2017-09-25 16:19:44 +00:00
caoyuan
f22df69b32 cleanup test-requirements
python-subunit is not used directly anywhere
and it is dependency of both testrepository
and os-testr
(probably was used by some tox wrapper script before)

Change-Id: Ic2139ceb5a200c3b32bcc87e81fc30f208645771
2017-09-25 19:29:20 +08:00
OpenStack Proposal Bot
8c44be5c9c Updated from global requirements
Change-Id: I10ea32764a04cfe0dd8445708f04bcd184c0d308
2017-09-16 23:18:37 +00:00
OpenStack Proposal Bot
cfdec41eee Updated from global requirements
Change-Id: I043babceb3b1be03550c8e90c9abb9450d5f5cde
2017-09-13 12:57:19 +00:00
OpenStack Proposal Bot
a23663e799 Updated from global requirements
Change-Id: I197925602a5b76cad961cc3cb2eebb67c0ea82bf
2017-08-30 23:26:21 +00:00
Stephen Finucane
002ca8c7f4 docs: Document the scheduler workflow
There have been some major changes to how scheduling works in Nova
during the Pike and Queens cycles. This documents these design changes
so that this new, more complex workflow is clearly spelled out.

Co-Authored-By: Ed Leafe <ed@leafe.com>

Change-Id: I15121d8fe9b715c0aec39dee4bfdf25ced42b481
2017-08-24 15:18:11 +00:00
OpenStack Proposal Bot
d48ba9873a Updated from global requirements
Change-Id: Idda59f3cf870479b5ca6ea3ab4516306d8e0075d
2017-08-18 15:53:20 +00:00
OpenStack Proposal Bot
92c4689256 Updated from global requirements
Change-Id: I96e70a950ed04862d512f91211c4cb6fc3fd1159
2017-08-07 00:49:24 +00:00
OpenStack Proposal Bot
b1244e795d Updated from global requirements
Change-Id: I35689a80f0a36779a32e8c49b0950daff306ea45
2017-07-29 02:37:27 +00:00
OpenStack Proposal Bot
e08cda0a70 Updated from global requirements
Change-Id: I812b4b9c7b05ab032f436e81d6e10b5a0f6598c1
2017-07-28 16:31:04 +00:00
chenxing
fa0e8f69b4 doc: Switch to openstackdocstheme
Change-Id: If7afc2cb58759b16fc6f7caa44d0cf6b7bcf4d06
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
2017-07-21 11:35:48 +01:00
Chris Dent
7792ba33a0 [placement] Update allocation-candidates.yaml for gabbi 1.35
Gabbi 1.35 provides template variable susbtitution on the left hand side
of a JSONPath expression. That's used here to remove the need for static
fake uuids in the test data for allocation-candidates and the
SharedStorageFixture.

A couple of other clean ups to the use of gabbi, unrelated to the 1.35
update:

* It's possible to test the entire value in one response:

    resources:
         VCPU: 4
         DISK_GB: 5

  instead of:

    resources.VCPU: 4
    resources.DISK_GB: 5

  This is handy when the root of the expression is calculated and
  repeating that calculation is noisy.

* [?foo.bar=baz] is the more canonical form of [?(@.foo.bar=baz)]. The
  former is built into jsonpath, the latter dips into the underlying
  implementation.

Change-Id: Ie6b5e0186533581815d04411ed7b321948464bb6
2017-07-17 13:04:48 +00:00
OpenStack Proposal Bot
742ff15d31 Updated from global requirements
Change-Id: Ib9135d0d73d3ec508455c3a703fe7c1c2208d250
2017-07-13 19:36:14 +00:00
OpenStack Proposal Bot
759b586a9e Updated from global requirements
Change-Id: I5373e0dbd746211cda9aa8a5162daab823fd8221
2017-06-27 12:16:50 +00:00
OpenStack Proposal Bot
0072f70d9b Updated from global requirements
Change-Id: I0159683fbd67ae11e745ed4f739f77fd5fa63141
2017-06-03 13:10:15 +00:00