230 Commits

Author SHA1 Message Date
Jens Rosenboom
df0aa8acaa Add a py35 environment to tox
Infra will start running python35 jobs soon [1], targeting to replace
the current python34 jobs. This is due to the change from Ubuntu Trusty
to Xenial, where only python3.5 will be available.

So we add a py35 environment with the same definitions for skipping
not-yet-working checks as we have in place for py34 already.

[1] https://review.openstack.org/336272

Change-Id: I017478fafa31121f40c95d73e49ac4dced80c7a2
2016-07-01 17:04:41 +02:00
Jenkins
d68acca919 Merge "Add ability to select specific tests for py34" 2016-06-29 16:31:23 +00:00
Andrew Laski
5e38fa3cb5 Add policy sample generation
This adds the entry point and config file necessary for using the
oslo.policy sample generation script. It also adds a tox target to
simplify the usage of it.

The command is also run in the docs tox target since the sample file
can be considered documentation.

Change-Id: I18a1fa567fc04b0e0a37c02f20024b2ce483f060
Partially-Implements: bp policy-in-code
2016-06-28 20:52:44 +00:00
Jens Rosenboom
0196f4676e Add ability to select specific tests for py34
For py27 you can simply run specific tests by calling

    tox -e py27 nova.tests.unit.test_something

Add a similiar option for the py34 environment. This will skip the
blacklist of tests that would otherwise be ignored for python3.

Change-Id: I9bf5901afed4abf592e1aaff2f4820b9e3abc671
2016-06-28 05:24:48 +02:00
Akihiro Motoki
c30562f458 tox.ini: Remove unnecessary comments in api-ref target
We pass -W option to sphinx-build to generate api-ref now.
The comment looks no longer correct.

Change-Id: Ie5dfaf46e9d3369c32c43da07c02501b224fb092
2016-06-10 04:23:52 +00:00
Jenkins
f48c58915d Merge "Let setup.py compile_catalog process all language files" 2016-05-24 20:56:24 +00:00
melanie witt
c19dd499f0 Pass OS_DEBUG to the tox test environment
The StandardLogging fixture defaults to log level INFO and has
the ability to log at level DEBUG if the OS_DEBUG environment
variable is set to a true value. Let tox copy OS_DEBUG to the
test environment to allow debug log capture for a command line
like:

  OS_DEBUG=True tox -epy27

Change-Id: Ibe51653f652c99aee56f702be571130cbb7f913d
2016-05-21 01:48:06 +00:00
Sven Anderson
ece7b8f1e7 Let setup.py compile_catalog process all language files
Two years ago the translation files have been split into several
files, separating the log messages of different log levels from each
other, like X.pot, X-log-warning.pot, X-log-info.pot, and so on.
However, the setup.py command `compile_catalogs`, that comes from the
babel package and compiles the corresponding .po files into .mo
files, only supported one file per python package.  This means that
during packaging `compile_catalogs` never compiled the X-log-*.po
files, so the corresponding translations were always missing.

Since babel 2.3 the domain can be set to a space separated list of
domains.  This change adds the the additional log level files to the
domain list.

The obsolete check that .po and .pot files are valid is removed from
tox.ini.

Change-Id: I0d9cb16ac50caae570c57a1db6bed656bf13c3dc
Closes-Bug: #1536226
2016-05-20 17:56:06 +02:00
Sean Dague
1555736e3c Final warnings removals for api-ref
This removes the final warnings from the api-ref code base, and flips
the sphinx switch to enforce warnings as errors.

It also adds code to the rest_parameters extension to make it more
clear where a problem is when it comes to included parameters. This
puts us in a place where we can start doing the file per file look at
content.

Part of bp:api-ref-in-rst

Change-Id: Ic2c99d96d6addcafa00b9f16785c2fe59b1798d3
2016-04-20 11:25:30 -04:00
ChangBo Guo(gcb)
b1871e5759 Nuke cliutils from oslo-incubator
Oslo team won't create new libary to include functions of module cliutils.
Nova only use two methods in cmd/manage.py, so copy these two methods
to nova/utils.py and clean up oslo-incubator related stuff.

Note:There is tiny change of method validate_args to make it Python 3
compatible.

Change-Id: Ibef5076f9556fa3224c2ca32e8c7fe2c4c1f739b
2016-04-03 23:38:31 +08:00
Sean Dague
329898ffbc Add sample API content
This adds sample API content for versions and servers resources,
including the parameters.yaml that is needed in both of these cases.

It also makes a new tox.ini target 'api-ref' for building these docs,
which will be used as part of the publishing pipeline.

Change-Id: I310ed352dc5dd81d01f2fd5f1a2fab662c29f0dc
2016-03-29 16:35:33 -04:00
Andreas Jaeger
f25648a963 Error on API Guide warnings
Error on warnings in building the API guide and fix any problem
encountered:
* tox.ini: Enable tread warnings as errors
* index.rst: Add unreferenced file extra_specs_and_properties as
  hidden file so that it does not show up in index. It is still
  built and will be referenced from other files.
* general_info.rst: Remove reference to non-existing file
* server_concept: Add missing empty line after code directives

Also fix a typo and remove extra EOL whitespace.

Change-Id: Ie766f8d2f7d82b7b5b02541b342f9829ca4550a0
2016-03-17 07:57:23 +01:00
Andreas Jaeger
7c7a6e0d1b Do not use constraints for venv
The venv environment is used by infra scripts to build tarballs and
documents - and used by both check and post jobs. In post jobs, we
currently do not support constraints, so add the non-constraints install
command here so that nova-docs (publishing of documents to
docs.openstack.org) and the tarball jobs work again.

Change-Id: I2e5c0a4d739296b8b9d4ef7f58b69e1eff3c9575
2016-02-28 19:06:50 +01:00
Sean Dague
1e287ae14b reduce pep8 requirements to just hacking
Because we are constrained by the constraints mechanism we no longer
need to install all of test-requirements to get hacking at the right
version, because we'll only install the upper-constraints version.

Change-Id: I877ffec045cc43a32f97be8afe453d4755507217
2016-02-24 20:23:05 -05:00
Stephen Finucane
19758d9303 tox: Remove 'oslo.versionedobjects' dependency
The dependency mentioned in the TODO has been resolved, thus the TODO
can now be addressed.

Change-Id: I306f83bb9474371a4aa8277a2a5e1e258471b400
2016-02-23 15:59:25 +00:00
Jenkins
2cff8de4be Merge "always use pip constraints" 2016-02-22 19:45:25 +00:00
Sean Dague
05583d1496 always use python2.7 for pep8
pep8 doesn't work with python3 on our codebase. If someone is on a
platform that defaults to python => python3, pep8 won't work for
them. This is actually really easy to fix with a single line in tox.

Change-Id: I7a888e4f7cc828638a9d61d2249a854ba1f3cb7b
Closes-Bug: #1456955
2016-02-19 21:10:42 -05:00
Sean Dague
1f0dec3a03 always use pip constraints
This flips the tox.ini to always use constraints installation for all
targets. It drops the extra -constraints targets in the process. This
makes it so there is no developer change required to operate in our
new world order.

Change-Id: I857032002c2ca873510ce7c39138e0e35cc5d3b3
2016-02-16 07:57:28 -05:00
Jenkins
853424ca11 Merge "Remove releasenotes/build between releasenotes runs" 2016-01-23 04:10:39 +00:00
Andreas Jaeger
b18c5c3422 Validate translations
Validate that translations are ok using msgfmt, so that no invalid
ones can be imported.

Closes-Bug: #1536214
Change-Id: Ic1f106684270a8f2197e0fb2a67d6fc00535ee65
2016-01-21 07:52:56 +00:00
Matt Riedemann
ce8030f251 Remove releasenotes/build between releasenotes runs
Just like in the docs target, we should remove the releasenotes/build
directory between runs of the releasenotes target to avoid stale
sphinx results corrupting the current run results.

Change-Id: Ia9e0409344671e04148308e08c26e8796227f0f7
Closes-Bug: #1534640
2016-01-18 12:42:27 -08:00
Matt Riedemann
702c01e26b Revert "Workaround reno reverts by accepting warnings"
This reverts commit ef8335185a517cb94bfa5d745298f810c304a507

Now that Ibf84ee0a604da94f709a0eeb436efe988d21274c is merged
on stable/liberty we should be OK again.

Change-Id: I1a56426a86747bdcd1a9179ecbff46cf54a98202
Related-Bug: #1534613
2016-01-16 14:31:06 +00:00
Sylvain Bauza
ef8335185a Workaround reno reverts by accepting warnings
b2acc9fa864b6fe10bc0c5f3786b976b472b1b27 on stable/liberty introduced a
sphinx warning which was not treated as an error in stable/liberty nova
releasenotes tox runs so it slipped in, but it is treated as an error
in master branch nova releasenotes tox runs, and since reno scans the
branches, master is broken because of the error in stable/liberty.

Plus, reno scans the git history, so right now it's not obvious how to
travel back in time and fix the stable/liberty change.

This change reduces the level of control we have by not treating reno warnings
as errors. This is not fun but it's the only way we can still land
release notes until an appropriate fix is merged (in reno or nova).

Again, this is a temporary patch and we consider reverting it once bug 1534613
is fixed.

Change-Id: Iedb69c66632383e056840e3294a90cda651a8778
Related-Bug: #1534613
2016-01-15 15:53:10 +00:00
Sean Dague
088bf2bd80 update min tox version to 2.0
The commands used by constraints need at least tox 2.0. Update to
reflect reality, which should help with local running of constraints
targets.

Change-Id: I27348462a21daab479d76b24b48e3a4e017504cc
2016-01-13 13:51:10 -05:00
Matthew Treinish
056bfd538f
Stop explicitly running test discovery for py34
This commit removes the direct call to discovery in the tox py34 job.
This was added as a temporary step to enable importing of everything
and is now include by standard in the use of ostestr. There isn't any
reason to keep it around anymore.

Change-Id: I117fbc2108b285aa148cb447c1c01145c43031dd
2016-01-05 11:35:34 -05:00
Jenkins
89dbc207c2 Merge "Add a note about fixing "db type could not be determined" with py34" 2015-12-16 16:15:31 +00:00
Ondřej Nový
944f4ac9d3 Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0

More info:
https://testrun.org/tox/latest/config.html#confval-downloadcache=path
https://pip.pypa.io/en/stable/reference/pip_install/#caching

Change-Id: I83b5333f4f4b1600b9bbd508b5e0bc04c0068836
2015-12-11 21:22:29 +01:00
Matt Riedemann
bc62d7b258 Add a note about fixing "db type could not be determined" with py34
Due to testrepository bug 1212909, if you run py27 before py34, py34
fails with "db type could not be determined". The workaround is to
delete the .testrepository directory and try again. As long as you run
py34 before py27 you're OK.

Change-Id: I13c4f9a39b97573e78fe397efc7d87f78bac4a58
2015-12-08 14:39:08 -08:00
Sean Dague
6b9c534658 add api-samples tox target
api-samples generation is no longer straight forward, now that tox is
blocking environment changes, and that PYTHONHASHSEED is randomized in
each run.

This creates a tox target to make it easy to generate the api-samples.

Change-Id: Ie88597fecee5ce81fb29ce2cfb73b15d60f3f389
2015-12-03 22:43:26 +00:00
Sylvain Bauza
9235151e00 force releasenotes warnings to be treated as errors
Per http://lists.openstack.org/pipermail/openstack-dev/2015-November/080521.html,
we need to make sure that there are no warnings for reno.

Change-Id: I50ce202f82e3213d07c3989780833e18b4c2bf08
2015-11-30 16:04:19 +01:00
Jenkins
9b6938b2c6 Merge "Use ObjectVersionChecker fixture from oslo.versionedobjects" 2015-11-16 19:51:32 +00:00
Jenkins
5e37254a91 Merge "Add reno for release notes management" 2015-11-13 16:24:51 +00:00
Andreas Jaeger
47ad006952 Fix paths for api-guide build
Add html for sphinx invocation so that files end in this directory
as usual. This is the usual place for sphinx documentation.

Change-Id: Iae7a3b91d407c06b7f4d2dce909a15c38ea72b9b
2015-11-13 15:02:29 +01:00
Sylvain Bauza
3a65e4848f Add reno for release notes management
Since reno is the new tool for Relnotes [1], we need to add it
to Nova and provide some Sphinx docs for Liberty and Mitaka.

Change-Id: Ibd74e62b6f36076dbec17dd146632fd42aad6eb2
2015-11-12 17:34:24 +01:00
Anne Gentle
17961c41a3 Rearranges to create new Compute API Guide
Uses openstackdocstheme to match other content

Has a dependent change in project-config also so that
file will build to developer.openstack.org/compute
at https://review.openstack.org/#/c/231000/

Change-Id: Ic060a1e79e4b2f8695cb788ff4df018e0cfd3286
2015-11-12 10:47:23 +00:00
Matt Riedemann
62bd5f26a5 Use ObjectVersionChecker fixture from oslo.versionedobjects
The fixture in the o.vo library allows us to remove duplicate test
code in Nova, so let's use that.

Change-Id: I896e7ee53856955e0338e7c83cd112d43b27367d
2015-11-10 11:50:56 -08:00
Sachi King
a42e53a8a7 Add -constraints sections for CI jobs
Tox does not currently support using factors with sections, and work
towards enabling this has stalled, as such we will need to duplicate
sections adding -constraints to facilitate running sections with the
constrained install_command.

Implements Blueprint: Requirements-Management

Change-Id: Icbbb78cfcd074b0050e60c54557637af723f9b92
2015-10-27 10:41:01 +09:00
Robert Collins
b13c33caa0 Remove redundant deps in tox.ini.
Tox installs the current tree (either via an sdist or as a develop
install) always. Our requirements.txt are reflected into the metadata
of that tree, so we shouldn't use requirements.txt directly - it just
means that we'd be hiding from ourselves issues like e.g. using URLs
which can't be reflected properly.

Change-Id: Id096f1d1971cb8a19f9e86ba57957b39e00bd6f5
2015-10-16 14:30:59 +13:00
Davanum Srinivas
9537c85763 Use os-testr for py34 tox target
os-testr has a blacklist we can use to specify which
tests to avoid. It uses "testr run --parallel" so tests
are run in parallel as well. We also get rid of the
huge list of tests in tox.ini into a separate file.

<soap_box>
The reason for using a black list is that when new tests
are being added, they are automatically either fixed to
work under python34 or added to a well known list of
tests so whoever is working on python34 can try to fix
those tests over a period of time. Hoping really that
this black list shrinks over time quickly. This is also
a pool of low hanging tasks that folks when they have
some time to pull a few tests from here and fix them.
</soapbox>

Bumps number of tests executed from 5606 to 7206

Depends-On: Ib5e682d0380cf7bc5e288a1e4d125379b452fa92
Change-Id: Ib641bb0f7553eb7704b419b9d00f86174f6d831d
2015-09-30 19:53:53 -04:00
Davanum Srinivas
b4e86c8ea1 Identify more py34 tests that already pass
Couple of tiny fixes and we essentially double our py34
coverage.

Change-Id: I77355043d613b8b3b6789950302c54b16b690049
2015-09-29 08:03:24 -04:00
jichenjc
eef23489ee Pep8 didn't check api/openstack/common.py
flake8 rule in tox.ini has following
exclude rule is exclude = .venv,.git,.tox,dist,doc,*openstack/common*
which excludes api/openstack/common.py

Change-Id: I39b82cf90048970e7bfc6be77de2e1543130f81b
Closes-Bug: 1498634
2015-09-19 23:23:19 +08:00
Sachi King
ad38ce1fc9 Add constraint target to tox.ini
This adds a pip install command to tox.ini that is only used when the
tox env is passed with the 'constraints' factor appended onto it.
As such this will not effect developer workflows or current unit tests.

The initial use of this will be in a non-voting job, to verify that the
constrained checks with tox are stable.  DevStack is already running
constrained jobs, as such problems are no expected.

To run a tox with pip using constraints on a developer system a
developer should run the desired tox environment with -constraints.
For example: $(tox -epy27-constraints)
Pip will pull the current version of the upper-constraints.txt file down
from the git.openstack.org, however this method can be overriden to use
a local file setting the environment variable "UPPER_CONSTRAINTS_FILE"
to the local path or a different URL, it is passed directly to pip.

This is currently not enabled in the default tox run, however it is
possible to enable it as a default by adding it to 'envlist' in tox.ini

Depends-On: I17ac389f78af241917b6da7f049085f2b13d30f2
Change-Id: I8ea1710a4e3a287405978b467e0b2dfbb025b795
Implements Blueprint: Requirements-Management
2015-09-04 15:06:11 -07:00
Matt Riedemann
426a5ffa8e Remove doc/source/api and doc/build before building docs
doc/source/api and doc/build are build artifacts from building the docs
using the tox -e docs target. These are ignored in .gitignore since we
don't care about them from build to build - but leaving them around can
actually make the docs build fail if modules no longer exist but are
still sitting in these directories.

So this change just cleans those up each time you run tox -e docs.

Closes-Bug: #1488637

Change-Id: I646ae0b77a472c425d23815316e804b09bbe867d
2015-08-25 13:08:31 -07:00
Jenkins
4abb475327 Merge "Gate on nova.conf.sample generation" 2015-08-24 11:38:20 +00:00
Davanum Srinivas
6761726ee9 Gate on nova.conf.sample generation
If someone makes a change that breaks "tox -e genconfig"
it is not caught today by any of the gate/check jobs.
We should fail at least the docs job by running
oslo-config-generator to make sure it works.

Change-Id: I63fede4a7b5f358f2212383e2870cc64c4f9f1e4
2015-08-21 13:23:03 -04:00
Andrey Kurilin
9e78de9fa9 Re-write way of compare APIVersionRequest's
`cmp` method was removed in Python 3, so it would be nice to use __lt__,
__le__, __eq__, __ne__, __gt__ and __ge__ instead, which are Py2 and Py3
compatible.

Change-Id: I1c89da0831b77b73f55d8681fd7d946535cc89b5
2015-08-18 15:36:44 +03:00
Jenkins
2d7bfab667 Merge "Undo tox -e docs pip install sphinx workaround" 2015-07-31 16:06:40 +00:00
Matt Riedemann
1d5b6964d2 Undo tox -e docs pip install sphinx workaround
Now that we require at least pbr 1.3 and that has
commit 61a3a1ecdf99c680e509742fdc6c441b499af130 in it,
let's remove the workaround.

Change-Id: I4074b05a823d32f5db0051062527b0db4d379b62
Related-Bug: #1473401
2015-07-30 18:40:15 -07:00
Davanum Srinivas
ed0196ebb6 Get py34 subunit.run test discovery to work
Currently the tox py34 target uses testtools.run to run
a subset of our test harness. We need to be able to use
pretty_tox.sh just like py27 as we make progress with py34
support.

The first step is to make sure we can discover all the
tests using:
python -m subunit.run discover -t . ./nova/tests/ --list

So, we need to fix a bunch of things for the discovery
to work including updating to a new version of websockify.

In the xen code, we should keep the original import and
add except for py34 as xen uses an older python that does
not work with six.moves.

Depends-On: Ib4ef2e79b28b7180e564b3d6dc2188456c66c08a
Change-Id: I88b6746da6136a7386a173f6cacd42f0b470deee
2015-07-28 10:40:56 +00:00
Davanum Srinivas
5bb1933cc4 Enable python34 tests for nova/tests/unit/scheduler/test*.py
nova/pci/stats.py:
  need to specify a key parameter which is a function that returns
  a sorting key. since the original intention with pool_cmp was to
  sort by number of items in each pool, we just use key parameter
  with a lambda
nova/scheduler/filters/trusted_filter.py
nova/scheduler/host_manager.py
  needed list() so we could delete stuff inside the loop in py3
nova/tests/unit/scheduler/test_filter_scheduler.py
  needed the import of test as some tests were failing in py3,
  added noqa as test is not explicitly referenced within the
  module
nova/tests/unit/scheduler/test_scheduler_options.py
  need BytesIO for py3, StringIO will fail in many tests
nova/tests/unit/scheduler/test_scheduler_utils.py
  message is not available in py3, so we need to adjust
  use test.nested already added in a previous commit
nova/test.py
  if we have a key which is a list, treat it just like a set
nova/utils.py
  use reduce from six and explicitly typcast version_int as
  an integer

Change-Id: Ic862873f38767f12b990a030b5812ae50a44c4d0
2015-07-28 10:40:36 +00:00