201 Commits

Author SHA1 Message Date
Eric Harney
d3abafdee4 Move config-generator to tools
Moving this out of cinder/ separates it from the
actual run-time Cinder code.

Change-Id: I056f40120fdef60a0dc1a5926729748fd2835938
2017-08-16 16:18:48 -04:00
Eric Harney
6460050c5c Add releasenotes to check_exec list
Prevent addition of executable files in
the releasenotes/notes directory.

Change-Id: I80ac0b7fce7b16a453fe9139f0dacf88325af8e7
2017-08-16 15:50:44 -04:00
Jay S. Bryant
429281bee2 Ignore all .egg-info directories in doc8 check
Doc builds can fail if you have .egg-info directories
in your repo.  This change makes sure that those
directories are ignored in the doc8 check.

Change-Id: I8e713651919f43feb80fca59b74c103b3255dd75
2017-07-13 10:43:56 -05:00
Ngo Quoc Cuong
8ca93ac3f7 Enable H904 check
H904  String interpolation should be delayed to be handled by the
logging code, rather than being done at the point of the logging call.
Use ',' instead of '%'.

See: https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#adding-variables-to-log-messages

Change-Id: Ib5781b837ba60028dce6dddd5b198647e002c8f1
2017-07-05 21:27:05 -04:00
jeremy.zhang
85a1b8d40b Enable some off-by-default checks
Some of the available checks are disabled by default in flake8, like:
[H106] Don’t put vim configuration in source files
[H203] Use assertIs(Not)None to check for None

This patch is to enable the H106 and H203 checks in Cinder project.
The C312 hacking rule will be removed when turn on H203.

Change-Id: I2e883c301b64d5977bbb907b63c9c144bc6f959d
2017-06-22 02:17:54 +00:00
Dirk Mueller
2db0497fde Add a local bindep.txt override
This avoids falling back to the global bindep-fallback.txt which
installs a pretty big environment. Without this change, a lot
of additional but unnecessary packages are installed on the nodepool
workers.

Change-Id: I8c581225eada929bc947b13f851bb228000f32fe
2017-05-25 23:27:10 +02:00
Jenkins
67fc1f76b4 Merge "Add bandit-baseline to tox.ini" 2017-03-29 22:29:40 +00:00
Nicholas Jones
2bec754d50 Add bandit-baseline to tox.ini
To add a bandit gate, a new tox environment is needed

Change-Id: If247add30d8dc4761dacb82c8d02ea4bd71546f0
2017-03-29 10:55:25 -05:00
Eric Harney
52e0f003fb Revert "Remove v1 API reference"
The v1 API is still in the tree, even if it is off by
default and deprecated.  We need to keep documentation on
how it works at least until we remove the code.  This was
published to [1] which was the primary way to find this
information, and it is now missing.

[1] https://developer.openstack.org/api-ref/block-storage/v1/

This marks v1 as deprecated in the toctree.

This reverts commit 8139f7f108786112ec6eef47228ecbc7f54b6f47.

Change-Id: Ic9d0fcce5ca0f8455c212292664b8f9694edfce3
2017-03-15 17:04:22 -04:00
Jenkins
d9278b5a60 Merge "Remove support for py34" 2017-02-17 10:29:04 +00:00
Danny Al-Gaaf
e3e749fa30 doc: verify all rst files
Make use of doc8 to verify all rst files which are not
autogenerated for errors and fail if there are any issues
found. The doc8 checks are now part of the tox 'docs'
environment and ran automatically. Checks can also be called
direcly via 'tox -e docs'.

Fix all issues found by doc8.

Closes-Bug: #1566765

Change-Id: I2b25247030a1aadcfe029c9e071ef17f2f72046b
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2017-02-14 12:36:57 +01:00
yanjun.fu
69c24cd5f4 Remove support for py34
The gating on python 3.4 is restricted to <= Mitaka. This is due to
the change from Ubuntu Trusty to Xenial, where only python3.5 is
available. There is no need to continue to keep these settings.

Change-Id: Ief6f4b8b37113f08dd0ae767f3dd2b5b2177e3ce
2017-02-08 17:32:00 +08:00
xianming mao
dbf61cdb4a Enable DeprecationWarning in test environments
Many deprecations are triggered early (on imports, for example)
before the warnings are enabled by the WarningsFixture in the
base test class.
To make sure all DeprecationWarning messages are emitted we enable
them via the PYTHONWARNINGS environment variable.

Note: this copy from 1be35ff039d67c50866f2556ef847b048cbf3578

Change-Id: I7c6106149832d048d4d3bd2701c2c0db81bd87cb
2017-01-05 12:22:22 +00:00
Jeremy Liu
45bfe2ecd5 Fix warning when running tox -e docs
We get a warning "WARNING:test command found but not installed in testenv"
when building docs. This patch fix it.

Change-Id: Ib9798e59c55099d5a4bf8989401b63487b0c6338
2016-12-13 17:30:47 +08:00
Sean McGinnis
ca0786778c Switch default py3x to py35
Per our conversation in the Cinder weekly meeting [1] we will
now switch to just running local tests against python 3.5 as
that is the more recent and common 3.x version available on new
distros. Other versions can still be run explicitly, but this
changes the default tox run to only use 3.5.

[1] http://eavesdrop.openstack.org/meetings/cinder/2016/cinder.2016-11-30-16.00.log.html#l-54

Change-Id: If2226cb229f016e36c4dfab6224d5ae353b926bb
2016-12-09 14:22:28 -06:00
Michał Dulko
792108f771 Remove ovo[fixtures] dependency from tox.ini
Looks like we should now be able to add oslo.versionedobjects[fixtures]
to test-requirements.txt and remove it from tox.ini, so this commit does
that.

Change-Id: I420ea16d8ad6d883e7c2c8de80f912ea35500367
2016-09-28 18:31:49 +02:00
Jenkins
bd63288029 Merge "Let setup.py compile_catalog process all language files" 2016-09-08 05:01:36 +00:00
Jenkins
aeabde14f3 Merge "Use constraints for api-ref environment" 2016-08-30 23:14:39 +00:00
Jenkins
226dd7c8c1 Merge "Add functional-py35 to tox" 2016-08-28 19:23:47 +00:00
Andreas Jaeger
f589c6e2d1 Cleanup api-ref
Remove useage of old os-api-ref version and replace with current version.

Follow other repositories and merge v1 and v2 in a single document.

Note that we need to have a single document so that preview of the
document works properly in our CI.

Change-Id: I82c8b9ca317298b3dc1f7133ce8c0e9b3503730d
2016-08-27 17:13:04 +02:00
Sheel Rana
7f2131e3a1 Update api-ref path
api-ref path needs update to support uploading cinder api-refs to
developer.openstack.org/api-ref/block-storage

update includes
https://github.com/openstack/cinder/tree/master/api-ref/v2/source
to
https://github.com/openstack/cinder/tree/master/api-ref/source/v2

Depends-on: Id38dd0609c77b0d6bcd0d935a9312b0211134802

Implements: bp api-reference-to-rst

Change-Id: If3a058a95f1eaa8481a926183591ff0f4b23e97b
2016-08-27 16:58:10 +02:00
Sven Anderson
0adde01a06 Let setup.py compile_catalog process all language files
Two years ago the translation files have been split into several
files, separating the log messages of different log levels from each
other, like X.pot, X-log-warning.pot, X-log-info.pot, and so on.
However, the setup.py command `compile_catalogs`, that comes from the
babel package and compiles the corresponding .po files into .mo
files, only supported one file per python package.  This means that
during packaging `compile_catalogs` never compiled the X-log-*.po
files, so the corresponding translations were always missing.

Since babel 2.3 the domain can be set to a space separated list of
domains.  This change adds the the additional log level files to the
domain list.

The obsolete check that .po and .pot files are valid is removed from
tox.ini.

Change-Id: I149c2254cb04297e598cfd3ca73b24efd0c8ef18
Closes-Bug: #1536226
2016-08-24 14:56:04 -04:00
Tom Barron
7517be2f5c Use constraints for api-ref environment
Change [1] updated tox.ini to use constraints rather than
specific, non-constrained install commands but neglected
to update the api-ref environment.  In addition to fixing
that omission we also remove a bit of ugly trailing
whitespace.

Co-authored-By: Gorka Eguileor <geguileo@redhat.com>

[1] I879f34129c0d958be0a73bfdc7f641e178d4efe9

Change-Id: I58ec756cd3de2e501c77a4d947e680d0959d22ec
2016-08-23 13:14:59 -04:00
Tom Barron
66023c52f8 Use constraints for all tox environments
Infra is now ready to use constraints everywhere [1], so adapt
releasenotes, coverage, and venv to use the default install command
instead of a specific non-constrained install command.

With this change, all tox environments now use constraints.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html

Change-Id: I879f34129c0d958be0a73bfdc7f641e178d4efe9
2016-08-15 17:24:40 -04:00
Eric Harney
20d8ff5545 Add functional-py35 to tox
This allows running functional tests in a py35
environment with "tox -e functional-py35".

Change-Id: I0e2a8f2f43ccf06c1e676a52a3843c04056c050a
2016-08-11 10:15:08 -04:00
Jenkins
2197f273d1 Merge "Fix tox pip-missing-reqs" 2016-08-10 17:47:44 +00:00
Gorka Eguileor
5f0f752be8 Prevent doc generation failure on OVO decorators
Currently Sphinx will fail to generate the documentation if we are using
decorators from any OVO in cinder.objects, because the OVOs are only
added to the cinder.objects namespace when the CLI programs are run.

So we need to run `register_all` method during documentation generation
to avoid failures like:

    AttributeError: 'module' object has no attribute 'Volume'

This patch modifies autogenerated doc/source/conf.py file and since now
it is no longer completely autogenerated it is added to PEP8 checks,
which required some minor changes.

Change-Id: Ifeeef61a778f9e3f3daceba8ed05cd2036219499
2016-08-09 18:24:25 +02:00
Eric Harney
7c9139ccd4 Fix tox pip-missing-reqs
tox -e pip-missing-reqs was failing due to a
bug in pip_missing_reqs.  The fixed package
has also been renamed to pip_check_reqs as of
version 2.0.

Change-Id: If91b979fedac8de5c611a87b900a2bcdb80d58b3
2016-08-08 16:25:50 -04:00
Eric Harney
872a46d676 Add functional-py34 to tox
This allows running functional tests in a py3
environment with "tox -e functional-py34".

Change-Id: Ia2cadbcaaea1ea91913a0b6e53009d14c990648b
2016-08-03 12:41:00 -04:00
Sean McGinnis
8148038e92 Add driver list to doc build
We have tox -e gendriverlist that outputs an RST-ish report of all drivers
in the tree. This output can be used in the docs build to automatically
publish the list of drivers to make it easier to find officially supported
drivers.

This effectively removes the existing drivers.html that was generated prior
that did not actually contain any useful information.

Change-Id: I8de78723af76aabcc976733ac4b248db0b8ca16f
2016-07-26 11:09:05 -05:00
Tom Barron
24db697d1b update min tox version to 2.0
The commands used by constraints need at least tox 2.0.  Update to
reflect reality, which should help with local running of constraints
targets.

Note: this is a clone of mriedem's manila change [1].

[1] I27348462a21daab479d76b24b48e3a4e017504cc

Change-Id: I185c8f80285f2e7b4f3d3d45a2e6a7dd5741bc1d
2016-07-12 18:00:40 -04:00
Michał Dulko
348c26094b Delete *.pyc files before running tox tests
When jumping between different review branches .pyc files of newly
introduced .py files are getting left behind. When these are related to
DB migrations all tests start to fail with: "AssertionError: There is no
script for XY version". This is pretty annoying, so why not remove all
*.pyc files before running any tests? This commit implements that.

Change-Id: Iac223fb156af3729b51fc2b11323a7cc30f9c07d
2016-07-11 15:17:02 +02:00
Jenkins
bffc3030da Merge "Added coverage report to tox.ini" 2016-06-27 18:19:34 +00:00
Michał Dulko
c063cbdae9 Don't reuse pep8 env in compliance tox runs
gate-cinder-tox-compliance is broken because run-tox.sh script fails
with ".tox/compliance/bin/pip: No such file or directory". This is
because we're reusing pep8 venv (envdir tox directive), so
.tox/compliance directory never gets created.

To fix hat we can simply stop reusing that venv and let compliance tox
job create it's own venv. It can be done by removing envdir directive
from tox.ini.

Change-Id: Ib809872f5fb1581e1b098775c2ebdc4e724c70e2
Closes-Bug: 1596508
2016-06-27 13:49:03 +02:00
Ryan Selden
2f56ccfb5a Added coverage report to tox.ini
When `tox -e cover` is run, nice coverage reports will be
generated in the cover/ directory.

Change-Id: I8da177475ff50e1ca08c28371ac1f4352ea14736
2016-06-24 23:12:12 +00:00
Sean McGinnis
e7b40242f8 Add driver interface checks
This is the start of an effort to both validate that drivers fully
implement the expected minimum requirements as well as to create a clear
place for driver developers to learn what needs to be implemented and get
documentation explaining what is expected for each method.

This also enables us to create tooling for documenting the available
drivers and their capabilities, to some degree. A follow up patch will
show some of what I'm thinking there, but it will make it possible to write
scripts for different needs.

This is somewhat a cleanup attempt to the ABC work that was started a
while back. This does not aim to replace that effort, but give a
mechanism for some of the things expected out of that effort that ended
up not being possible with how it evolved.

In most cases we do not really care if a driver is inherited from a
certain base class, just that it conforms to the given interface.

The interface/inheritance work really centers around two separate
things:

 * Ensuring drivers conform to an expected interface
 * Allowing code reuse and common implementation

This is really for the first item. Additional work is needed to complete
the ABC work we've done, but that really focuses on the second item, and
is out of scope for the intent of this patch.

Change-Id: I4168225126fe88c31712d94f0a130e9e7ede3446
2016-06-13 15:21:47 +00:00
Jenkins
b230cb743c Merge "Add sample config file to cinder docs" 2016-06-09 23:00:39 +00:00
Sheel Rana
41929d1c7a migrate to os-api-ref
os-api-ref is released now, so we can move to using it instead
of copy pasting in cinder tree.

Change-Id: Id38dd0609c77b0d6bcd0d935a9312b0211134802
2016-06-09 10:35:02 +00:00
Sean McGinnis
fcd7fcea3e Add sample config file to cinder docs
The oslo sphinxconfiggen module was added to the oslo.config
2.3.0 release. This enables config file generation as part of
the sphinx doc generation.

The generated config file will pick up the current config
options from the code base. And as an added bonus, it will
now be published to the docs.openstack.org site for easy
reference or download.

This also puts us inline with what other projects like Nova
are doing for sample config files and is the recommended
method from the Oslo team.

Change-Id: I912a97eb2686d3dc56e50d8641d7bd930179bc18
2016-05-17 08:52:21 -05:00
Daniel Gonzalez
c2f2eeb394 Whitelist 'rm' in api-ref tox environment
Running the api-ref environment currently warns that 'rm' is not
installed in testenv. This patch removes the warning by whitelisting
the rm command.

Change-Id: I4072491713a38b11d9f55b26a8ba0a3cf49445f0
Closes-Bug: #1581778
2016-05-14 11:36:19 +02:00
Sheel Rana
8659ff0ef5 WADL to RST migration in cinder tree
This patch is for converting API Reference to RST and host it
in the Cinder tree.
This patch contains all the RST for cinder to bring over to their
repos to begin building API reference information from within.
This contains .inc files which have all the contents of the .rst files
but are grouped together for easier editing

This is the results of the RST conversion from WADL. It creates a
single index plus a bunch of included files which represent sections
of the API.

Cleaning task will be done once this part is merged.

As we have removed XML API support in Newton release, we need to
remove XML from API reference as well.

But we still have installations which are using XML.
So implementation plan is to first merge this patch in Newton release
keeping XML for now, then I will backport same to stable branches.
After that I will remove same from Newton release to match API
reference with actual API implementation.

Implements: bp api-reference-to-rst
Change-Id: I865ac922538bfa5bd45c24eb4bc49f5e966dc811
2016-05-12 19:07:30 +00:00
Jenkins
01f5f04689 Merge "Run py34 tests with plain 'tox' command" 2016-04-04 23:47:48 +00:00
Christopher J Schaefer
4e8ef51bc7 Remove bandit.yaml in favor of defaults
The most recent version of bandit no longer requires a configuration
file (bandit.yaml) to run the default test suite. Removing the
configuration file and updating the bandit arguments will run all of
the currently available bandit tests in the default test suite.

Change-Id: If144c4017ce9bb58c3e83b62a5e31325353200b5
2016-03-18 11:08:52 -05:00
Tom Barron
1787244db4 Run py34 tests with plain 'tox' command
Now that all cinder unit tests pass on python 3.4 [1], we can run py34
tests by default alongside py27 and pep8.

This commit also addresses the annoyance of py34 tox tests failing with
'db type could not be determined' if py27 tests were initially run in
the workspace.

[1] https://haypo.github.io/openstack_mitaka_python3.html

Change-Id: If60e5d0d3185e78f38fa2bfc7b6bb4840f09d840
Closes-bug: #1489059
2016-03-09 03:07:26 +00:00
Jenkins
077b8593ae Merge "always use pip constraints" 2016-02-29 18:45:29 +00:00
Jenkins
5f2b909ad3 Merge "Add pip-missing-reqs tox env" 2016-02-29 12:59:31 +00:00
Andreas Jaeger
379f272924 always use pip constraints
This flips the tox.ini to always use constraints installation for
all targets. It drops the extra -constraints targets in the process.
This makes it so there is no developer change required to operate in our
new world order.

Change-Id: I2b56835edb2d75b7c871aae030e909578dbb958d
Depends-On: Iecd35d6bc3b1e9e0ed31b48a34e4149d34d55ac2
2016-02-29 13:00:11 +01:00
Victor Stinner
a09143cb31 Enable all unit tests on Python 3.4
* Remove tests-py3.txt: all unit tests now pass on Python 3
* tox.ini: remove specific command for py34 test environment, it now
  inherits testenv commands (same command than Python 2.7)

Partial-Implements: blueprint cinder-python3
Change-Id: I52c68a63740b3ae6097fad419b59a9c29422b525
2016-02-26 14:35:34 +00:00
Ivan Kolodyazhny
7f45046017 Revert changes to use ostestr
ostestr has more user-friendly output including lists of all and failed
tests.

Change-Id: I1f715a5688d702aaf8489928a1b0a39093caa5da
2016-02-17 16:04:40 +02:00
Ivan Kolodyazhny
6bdc836965 Move integrated tests to 'functional' directory
Functional unit tests should not be a part of unit
tests job. This patch adds new tox environment to run
functional tests:
    $ tox -e functional

It also removes logging from functional tests. Some of log calls were
replaced with asserts.

Related Implements: blueprint cinder-integrated-tests

Change-Id: I0ebfef2fe05f502cd5fb08fbc8af09008949e457
2016-02-09 23:28:27 +02:00