We had carried over the full list of pep8 whitespace ignores from nova.
Trying to address them all in the entire nova repository would have been
too big a task; but it's tenable here in placement.
Do it now rather than letting these whitespace issues compound.
This change removes the E* whitespace ignores and fixes the pep8 issues
they were masking.
Change-Id: Icbabdb9b56fd5a3e9fd14ab537abf3d0d6456fee
In tox versions after 3.0.0rc1 [1], setting the environment variable
PYTHONDONTWRITEBYTECODE will cause tox not to write .pyc files, which
means you don't have to delete them, which makes things faster.
In older tox versions, the env var is ignored.
If we bump the minimum tox version to something later than 3.0.0rc1, we
can remove the commands that find and remove .pyc files.
[1] 336f4f6bd8
Change-Id: Id9a97af032a2dbefdc50057270368ad087e6bb5f
While exploring removing unused packages from lower-constraints.txt it
became clear that the lower-constraints job was not working as expected:
Because our tox config has usedevelop=True 'setup.py develop' is called
to install the placement package after the install command is called.
This means that the lower-constraints are clobbered.
I had mistakenly assumed that turning off 'usedevelop', which causes
'setup.py install' would not make any difference, because it usually
installs dependencies too. It turns out however, that when using pbr
and within a git working dir, it does not. That took some time to
figure out. Oh well.
This change makes it so that we create the tox environment using
usedevelop=False and with our own install_command, to avoid upper
constraints conflicting with lower constraints.
This flagged up a few changes, the main one being that we did not have
a new enough version of keystonemiddleware in order to require use of
www_authenticate_uri. requirements.txt is updated for this as well.
And PasteDeploy needed to be updated to work with Python 3's notion
of namespace packages.
psycopg2 need a newer version to work with Postgresql 10.
oslotest needs to be raised to 3.4.0 because the tests in
cmd.test_manage use features to control what is capture by the Output
fixture from oslotest. Note that the lower-constraints job found
this problem and also demonstrates why we must run the lower-constraints
job without upper-constraints being involved. upper-constraints will
"win" and we don't want that. The point of the job is find packages
where lower-constraints are wrong, so it must "win".
The end result here is a lower-constraints.txt file that starts from
the lower-constraints.txt defined by nova, and then is adapted to
update the versions of packages that were not up to date, remove
those packages which are no longer present, and add some that are
now required.
Change-Id: Id66a28f7ace6fc2adf0e1201d9de5f901234d870
While looking into making py36 available when running tox without '-e'
and making py37 more available for both unit testing and functional
testing, it became clear that our tox.ini was a bit messy. This change
cleans it up to:
* Removes all py* jobs and lets tox define those automatically. Their
commands come from the base [testenv].
Non-unit tests need to define their commands specifically.
* move pep-related tox below both unit and functional
* remove some out of date comments
* set the default when using tox without -e to unit and functional
in 2.7 and 3.6, and pep8
* removes the 'stestr slowest' from the unit and functional test env's.
It's not used consistently and the output is far from consistent so
it doesn't seem to have any particular value (for this project)
This is one of several ways to do this cleanup. This way assumes some
knowledge of the fact that tox defines a lot default jobs with default
python versions (via factors[1]).
Note that the releasenotes job does not currently work and did
not work before these changes, a conf.py is missing.
[1] https://tox.readthedocs.io/en/latest/config.html#factors-and-factor-conditional-settings
Change-Id: I9a877439e13eb753c7f98dc02e41db891ce33690
In tox.ini file, the doctrees directory is specified
as an argument of sphinx-build command in api-ref,
releasenotes targets.
But it is missing in docs. So add it.
TrivialFix
Change-Id: I767b675b503cd627e80c7c1ff434d4a8ad53203a
The dependencies knock down the four most complex methods in the
project, bringing our most complex method to 14 (and hence our
max-complexity limit to 15).
Ife303d018a26ddcf8ea41d5950dc774c5e898a56
placement/util.py: 'parse_qs_request_groups' (17)
I9f3aa66b4806cf2c34f5271a5f4426839cb872ce
placement/objects/resource_provider.py:
'ResourceProviderList._get_all_by_filters_from_db' (16)
placement/objects/resource_provider.py:
'_get_provider_ids_matching' (16)
I80f8b1a74b8afdabbb85afc6caef1abef8ac3751
placement/objects/resource_provider.py:
'AllocationCandidates._get_by_requests' (16)
Change-Id: I11d3e3c0e1baf729f2acceafbe2f115212a08385
Before this patch, the project's max-complexity of 17 was perpetrated by
RequestGroup.dict_from_request, the request querystring parsing method
used for GET /allocation_candidates. This refactor-only patch splits
that method into smaller chunks, the most complex of which now comes in
at 8.
There is no functional change, hence no deltas to tests.
Change-Id: Ife303d018a26ddcf8ea41d5950dc774c5e898a56
This dries up tox.ini a bit while preserving the explanatory
comment. Having the grouping expression on test jobs that don't
require it has only minor impact: the regex is applied to the
list of tests and no groups are found.
Change-Id: I89330cf5b0ace25db6a7575cfa4f93d9575b08ac
Placement has always relied heavily on functional testing for both API
and database testing, so not including those tests in 'coverage' leads
to misleading results.
This change adjusts the cover job in tox to use a test path of
placement/tests, resulting in all the tests running.
In the process .coveragerc is updated to be placement oriented
and make sure we do not ignore branches.
This is in preparation for turning on a check job for coverage.
See this email for more on that:
http://lists.openstack.org/pipermail/openstack-dev/2018-September/134385.html
A followup patch will turn on a check job for it.
Change-Id: I6c122facc9c9f6e0b2a1d69c6aaf5faeee7cf253
Change [1] added support for tox -e genconfig via a configuration file
called placement-config-generator.conf. Since we're in the placement
project and only have one config generator, the placement- part is
redundant. In keeping with similar pieces (e.g. policy-generator.conf)
this patch renames the file and fixes references.
[1] https://review.openstack.org/#/c/601838/
Change-Id: I8ea9c00d26539feab141ada493bc825987046b3d
The current max-complexity in placement code is 17, so set the
upper bound in the pep8 tests to max of 18 so we get an error
when we're going to make it worse. The current high complexity
(but still under 18) functions are:
./placement/util.py:415:1: C901 'parse_qs_request_groups' is too complex (17)
./placement/objects/resource_provider.py:1341:1: C901 'ResourceProviderList._get_all_by_filters_from_db' is too complex (16)
./placement/objects/resource_provider.py:2848:1: C901 '_get_provider_ids_matching' is too complex (16)
./placement/objects/resource_provider.py:4076:1: C901 'AllocationCandidates._get_by_requests' is too complex (16)
Change-Id: I58dc5acb9bb13dcdb5623d13515820591ec5f974
When part of Nova, many files were named in such a way as to indicate
that they were separate from the nova files that did the same thing. Now
that that distinction is no longer needed, we can rename them to sound
more natural.
Additionally, references to these file names in other parts of the code
have been updated.
Change-Id: Id4439b4af827b64bbac213085ac74d1283c96db0
This patch does pretty much the minimum possible to get `tox -e docs`
building and gating. Much, much more needs to be done to make the
content good.
Also corrects a link in the api-ref which, once these are publishing,
would have pointed to the wrong place.
Change-Id: I5cbc3a3cceeaeaa7be5593658b6a03fa25fb69d0
The pip-missing-reqs job does not exist but because of the way
we have tox configured, running 'tox' by itself will run what
amount to a null job, consuming time but testing nothing.
See also: I8355372da93b93cc1c7f7a1501af0fbd01eee615
Change-Id: I0969908ca7f5acbcde4a3d561366295bf2403c83
Begin the process of getting placement tests running, by getting the
test harness opertional.
Before we can expect the tests to be able to run the configuration for
the placement package and its test harness needs to be adjusted for
"nova" -> "placement". These changes are made by inspection and based
on previous iterations of doing these kinds of cleanups, it won't be
until later in this stack of change that actually passing tests will
show up. However, the logs for attempted tests runs should be getting
a little bit further than the parent.
setup.cfg changes quite a bit because there are many things in it that
are not relevant. placement (thus far) really only has one entry point.
As noted, by a TODO, a placement-manage and placement-status will be
desired in the near future.
Change-Id: Iaa385c6e8653a9646a21bb65203deaa05ec43217
Xen UT failures seem to have cleared up, which may or may not have
anything to do with [1]. Remove the blacklist file and get these back
in the py3 jobs.
[1] https://review.openstack.org/#/c/591061/
Change-Id: Iacc1ae01f535caed64793d57b757ea07a8c46620
Without this, you can successfully run a 'functional-py36' environment,
but no actual tests are run. Python 3.6 is common in many environments
these days, and we want devs to test locally. This helps.
Change-Id: I2c04b7a8b6ee06638c6ce92dc51e200fa2b13539
In commit df669caf, we modified tox.ini to reuse a shared directory for
environments with the same dependencies. The 'venv' directory was
chosen as this seemed like a suitably generic target. However, the
'venv' target actually uses a non-default values for 'deps' as doc
requirements are included to allow things like reno using this target.
This means switching between the 'venv' target and any of the other
other targets using this directory will result in the virtualenv being
rebuilt each time.
Resolve this by using an explicit directory for these shared
environments. A single setting of 'deps' for one of the targets is
removed to make this clear.
Change-Id: I239606f5f354a5536b20b1d31ee81d80b4572106
There's nothing we can do to fix this so simply silence it until we no
longer support that version.
Change-Id: Ic7a199a1374e6f36c092e5ca6faa435aa5bf4cbd
- This change adds requirements.txt to the list
of requirements files used when createing
the tox venv environment.
Change-Id: Ib222b3cb68d2129c061400f51f50a19d186b9080
This massively reduces the amount of time needed to install dependencies
and saves a lot of disk space to boot #winning
Note that it's not possible to use 'envdir = {[testenv:py27]envdir}' as
that syntax doesn't work with implicit attributes.
Change-Id: I229e1961a66db08be2d4f7235d2c3724e875f13b
Take advantage of tox's expressiveness to avoid duplication wherever
possible and document the many targets we have.
Change-Id: I33675b7eedcffcebf1115dd48757874cb73f3e2f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We want to default to running all tox environments under python 3, so
set the basepython value in [testenv].
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We must explicitly set basepython in envs that are usually "automatic" -
otherwise due to https://github.com/tox-dev/tox/issues/425 these will
use `basepython` instead of their automatic equivalents.
Change-Id: I1026110f0f1736127cf0af8ec2ef791a0f999e3a
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This adds a granular policy checking framework for
placement based on nova.policy but with a lot of
the legacy cruft removed, like the is_admin and
context_is_admin rules.
A new PlacementPolicyFixture is added along with
a new configuration option, [placement]/policy_file,
which is needed because the default policy file
that gets used in config is from [oslo_policy]/policy_file
which is being used as the nova policy file. As
far as I can tell, oslo.policy doesn't allow for
multiple policy files with different names unless
I'm misunderstanding how the policy_dirs option works.
With these changes, we can have something like:
/etc/nova/policy.json - for nova policy rules
/etc/nova/placement-policy.yaml - for placement rules
The docs are also updated to include the placement
policy sample along with a tox builder for the sample.
This starts by adding granular rules for CRUD operations
on the /resource_providers and /resource_providers/{uuid}
routes which use the same descriptions from the placement
API reference. Subsequent patches will add new granular
rules for the other routes.
Part of blueprint granular-placement-policy
Change-Id: I17573f5210314341c332fdcb1ce462a989c21940
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Change-Id: Ic28558ee6481d49a9b4e5dc2c4182504e330448f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Co-Authored-by: Eric Fried <efried@us.ibm.com>
Co-Authored-by: Jim Rollenhagen <jim@jimrollenhagen.com>
We recently fixed a couple of pep8 failures under py3 [0] which got
through because we're not gating pep8 on py3. This changes the python
version for the pep8 tox env to python3 because the py3 checks are
stricter. This updates the fast8 tox env as well.
[0] https://review.openstack.org/#/c/557633
Change-Id: I540377606a8224ba569b8c145773e7d057575bd8
The Project Testing Interface [1] asks that we list requirements in
'doc/requirements.txt' and build docs by calling 'sphinx-build' directly
instead of via the 'build_sphinx' setuptool/distutils wrapper. Start
doing this.
[1] https://governance.openstack.org/tc/reference/project-testing-interface.html
Change-Id: If9342c8ea757b1735f2488db751008984fb33baf
This commit removes the single quotes from all the stestr run commands
in the tox.ini. These were originally added as a workaround from the
ostestr days to ensure people could run a subset of tests with a regex
like:
tox -epy27 regex
With the switch to stestr this is no longer needed because stestr treats
extra arguments as a filter regex. Quoting the posargs has a negative
effect because '--' parameters are treated as a string on input instead
of a parameter.
Fixing this also uncovered another issue in the stestr commands for the
functional jobs. The CLI arg for stestr versions <=1.1.0 is
--group_regex not --group-regex. This commit fixes this to make those
jobs pass without the quotes.
Change-Id: I512a517fa6890b29de24060376b4d7e061dc3add
With the current super-explicit version of tox.ini, it is not possible
to run tox -epy<any version> and have tests discovered and run. This
is because the default testenv does not run tests. Therefore, the
easiest way to add another version is to simply add another testenv,
as done in this change.
Python3.6 is the default Python3 on recent distributions so it needs
to be present if people are expected to be able to run python3 unit
tests before pushing their changes.
Change-Id: I728e482af3142bb101fc6bf52ec7926e29a42594
Nova switched to using stestr for all it's in tree testing except for
the coverage job. This commit updates that job so it also uses stestr
and cleans up the last bits of testr setup left in tree. As part of
this change this moves all the one off per job configuration into the
tox job definitions instead of hiding it in either the .stestr.conf or
in a wrapper script.
Change-Id: I36e0e791d485b44641a1ed8770eca8e95aca19be