66 Commits

Author SHA1 Message Date
Alex Gaynor
cd9d8e45c6 Remove the monkey patching of _ into the builtins
Previous _ was monkey patched into builtins whenever
certain modules were imported. This removes that and
simply imports it when it is needed.

Change-Id: I0af2c6d8a230e94440d655d13cab9107ac20d13c
2013-07-26 07:31:17 -07:00
Joe Gordon
dea5ce73a8 Start using hacking 0.6
So we can start using new hacking 0.6 checks.

Since hacking 0.6 pins pep8, pyflakes and flake8 they can be removed
from test-requires. For any unfixed checks list them in tox.ini

Fix a H103, 'Header does not match Apache 2.0 License notice' that
leaked in

Change-Id: I652fe3270f1c37afdd386bfae7c4b27b6abcfa06
2013-07-25 12:15:17 -04:00
Zhongyue Luo
ca08c4447a Sync sample config file generator with Oslo
The sample generator tool in Oslo is located in tools/config therefore
this patch moves the files in tools/conf to tools/config. The reason
Oslo chose 'config' is because the generator.py module is located in
the openstack.common.config package and update.py looks for the
corresponding directory in tools when it syncs the 'config' package.

./update.sh --module config --base nova --dest-dir $NOVA_HOME --nodep

Since every thing moved to tools/config, changes had to happen on
the tox.ini file and check_update.sh

Change-Id: Icbd467888cc7f16fa3694ed2b93548d0285461b5
2013-07-23 13:03:34 +08:00
Julien Danjou
923a7eeaa3 Check that the configuration file sample is up to date
We added this check recently to Ceilometer, and it helps a lot to keep this
file always up to date and not falling behind.

Change-Id: I5ad2c6366032a39c2e55dd11a16b9f1780ab9ed0
2013-07-18 20:22:43 +02:00
Monty Taylor
e1d9ab2977 Add -U to the command line for pip
Since we have site-packages=True in nova, the existence of system
packages of the wrong version can cause nova to not install what
you expect into the tox virtualenv. Adding -U into the deps list
causes it to get appended into the command line, which causes pip
to do what we want and install what we asked for.

Change-Id: I4f388e07caf31ab870f22d261e856a205ebb36c0
2013-07-11 21:14:43 -04:00
John Garbutt
707a7964f4 Ensure flake8 tests run on all api code
We need to exclude the plugins/xenserver directory
because the python2.4 code can't use as in except clauses.
The existing exclude directory accidentally included some
code in nova/api/openstack/compute/plugins and the related
tests. This code is now included in the flake8 tests,
and the current violations have been fixed up.

Fixes bug 1197858

Change-Id: Ib5f1b65d5d793f0f5854e898ca3ce751e43ef233
2013-07-09 09:52:44 +01:00
Joe Gordon
efe1957237 Fix and gate on E125
E125 continuation line does not distinguish itself from next logical
line.

E125 makes code more readable, and fixing all the existing issues allows
us to gate on this, making this one less thing reviewers have to check
for.

Fix was made using https://github.com/hhatto/autopep8

Change-Id: Ie569fd74fb84d220ceb87e37b1b1f39143bba80f
2013-07-04 08:49:45 +02:00
Devananda van der Veen
f2b4419b9f Add HACKING check for db session param
Add a HACKING check to enforce that public db/api and db/sqlalchemy/api
methods to not accept a 'session' parameter.

This check is initially disabled, since it is failing ~24 times right
now, but will be enabled once bp/db-session-cleanup is complete.

Change-Id: Ib89eea58555032dd142d4e21e62d66e2726f0d06
2013-07-02 03:11:24 -07:00
Dirk Mueller
1bf87dda57 Fix and enable H403 tests
Multi-line doc-strings should end on a new, separate line.

Change-Id: I4cf0cfe92b634ef77971863a4df41ef43531bc20
2013-06-12 22:07:23 +02:00
Jenkins
e9687d8bc7 Merge "Delete unused bin directory" 2013-06-05 18:26:47 +00:00
Joe Gordon
584b2fb4b0 Delete unused bin directory
Delete last bits of bin/.  With the move to entrypoints these aren't
needed anymore.

Update CONF.bindir to default to
  os.path.join(sys.prefix, 'local', 'bin')

Part of blueprint entrypoints-plugins

Change-Id: I95250d3779433e7b85aaa889a873b16c86a7d2be
2013-06-03 11:15:33 -07:00
Chuck Short
a5b981ed65 python3: Add py33 to tox.ini
Introduce py33 to tox.ini to make testing with
python3 easier.

Change-Id: I3f966efc1a845aaa8e317d810bd6a1366acd323c
Signed-off-by: Chuck Short <chuck.short@canonical.com>
2013-06-01 16:46:25 -05:00
Joe Gordon
838b288945 Add posargs support to flake8 call
Add posargs to flake8 call in tox.ini, with this you can pass arguments
directly into flake8 using the following notation
  tox -epep8 -- --FLAKE8-ARG

Change-Id: I1296eac3df46438ef050bf99ca33eb38d3b02efa
2013-05-28 23:04:44 +02:00
Joe Gordon
27948ffa5f Enumerate Flake8 E12x ignores
Instead of ignoring all E12x errors, ignore only the failing ones,
so its easy to fix each one separately.

Change-Id: Id365f4d61be56b51823ead8c7c1948e034ea609b
2013-05-28 23:04:44 +02:00
Joe Gordon
d534b290a1 Fix and enable flake8 F823
F823 local variable 'blah' (defined in enclosing scope on line xx)
referenced before assignment

Change-Id: I721e1339ef023558032980bbf59e8dc44b391d52
2013-05-28 23:04:44 +02:00
Joe Gordon
d9c145c7b8 Fix and enable flake8 F812
F812 list comprehension redefines

List comprehension don't have scope so redefining variables can lead to
unexpected behaviors

Change-Id: I85484dfcd4fa391195ea7be75aadc7897da4739c
2013-05-28 23:04:35 +02:00
Joe Gordon
7ef1dcf18b Enumerate Flake8 Fxxx ignores
Instead of ignoring all Fxxx errors, ignore only the failing ones, so
its easy to fix each one separately.

Change-Id: I4ccc1d38247fb76b6dad4388b76c6e8d696f081f
2013-05-28 13:28:36 +02:00
Joe Gordon
e87f486322 Enable flake8 E721
We already pass the test, so just remove from ignore list

Change-Id: I28bfeb99d9841532133a3825f7336e7b712d6561
2013-05-28 13:28:36 +02:00
Ruby Loo
5740920bb3 Rename requires files to standard names.
Rename tools/pip-requires to requirements.txt and tools/test-requires
to test-requirements.txt. These are standard files, and tools in the
general world are growing intelligence about them.

Change-Id: I68ece8406fb1d4e082a42db8e76e17b1aaa7e775
Fixes: bug #1179008
2013-05-22 20:12:08 +00:00
Monty Taylor
a333be8fe4 Switch to flake8+hacking.
Remove a bunch of local custom scripts. Replace with configurable
external tools.
Use local hacking checks for nova specifics.

Change-Id: I75a01375ba0ec36d2ff05abc47abe0a3f225eda5
2013-05-17 12:54:12 -07:00
Clark Boylan
dbbbbf4ecb Enable tox use of site-packages for libvirt.
Enable the use of site-packages in tox which will allow the use of the
system install of libvirt while testing.

Hardcode the libvirt host UUID for tests that check this UUID when
system libvirt is being used. Without this hardcoding eight tests
would fail when using the system libvirt install.

Partially fixes bug #1113181

Change-Id: I59c5fbd45639962c0963298203c39759b6ca2d11
2013-03-19 12:11:10 +02:00
Dan Prince
0b2e8d6e06 Update tox.ini to support RHEL 6.x.
In order to support running unit tests on RHEL 6.x we need to patch
eventlet with contrib/redhat-eventlet.patch. We already
have support for this in the tools/install_venv_common.py but we need
to make a couple changes to allow tox to consume this:

1) Sync in the latest intall_venv_common.py from oslo. This changes
 patch to use the -N option (ignore already applied patches) and makes
 it safe to call the patching function more than once.

2) Add a new patch_tox_venv.py script in tools.

3) Update tox.ini to call patch_tox_venv.py before it runs tests and
coverage.

Change-Id: I3e2b07c3f718e4aede5c5f231ff0cdb7721ec885
2013-03-07 11:25:25 -05:00
Mark McLoughlin
706a137005 Use oslo-config-2013.1b4
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.

Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.

Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.

Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.

Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.

Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).

Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
2013-02-19 21:16:32 -08:00
Rick Harris
0a93b563e9 Harmonize PEP8 checking between tox and run_tests.sh
Tox and run_tests.sh were running PEP8 checks against different
file-sets. This patch refactors the logic to determine which files to
run PEP8 checks on into `tools/run_pep8.sh` where it can be called by
both tox and `run_tests.sh`.

Additional fixes:

Some of our Python XenAPI Dom0 plugins don't end in *.py but should
still be checked by PEP8. This patches fixes the hacking.py violations
in the files and adds them back to the srcfiles list.

Merged tools/unused_imports.sh into tools/run_pep8.sh

Change-Id: Id5edd1acb644ab938beffc3473494a179d9d8cda
2013-02-15 00:39:58 +00:00
Attila Fazekas
5dd5625d97 Module import style checking changes
* Implementing the * import detection (it is disabled for now)
* New style relative import testing based on syntax rules
* Old style relative import testing based on module search
* Inspection based solution replaced by PYTHONPATH search
  in order to avoid module compile and initialization steps
  (code execution) in a syntax checking phase.

This solution is faster and safer, but does not able to recognize
modules added dynamically to the module scope.

Change-Id: Ifc871f4fdbcd4a9a736170ceb4475f4f2cbe66bc
2013-02-13 17:05:09 +01:00
Anita Kuno
769cb5b674 Added the build directory to the tox.ini list pep8 ignores.
Change-Id: Ie3c0d5ce021058de8ac868c736660fa0c75b5af5
2013-02-07 21:44:17 +00:00
Joe Gordon
8af6c6b885 Make sure there are no unused import
Remove all currently unused imports
Prevent future unused imports

Change-Id: I6ac26d5c71b79952a7732db300355a00310c712e
2013-01-28 21:37:32 +00:00
Joe Gordon
8f394ee716 Enable N302: Import modules only
Fix all N302 issues, and re-enable.

Change-Id: Ic94d144c915b228b7ff2fd9c5951875e159ffcdd
2013-01-25 12:11:54 -05:00
Joe Gordon
4845fc2720 Fix hacking N302 import only modules
* Includes some general tools/hacking cleanup
* Fix several N302 cases
* Disable N302 until all cases are fixed

Change-Id: Iddba07ff13e10dc41a6930749044bb8c0572d279
2013-01-24 09:52:58 -05:00
Clark Boylan
cc88380dfe Fix nova coverage.
Nova coverage was broken because additional coverage tests were added
that interfere with collecting coverage during unit testing. These new
tests were not properly excluded by the coverage unit test regex. Modify
the regex to exclude any tests of the form '.*test.*coverage.*'. This
allows `tox -ecover` and `run_tests.sh --coverage` to run properly.

Change-Id: Id35f50ec335707c9eadb0e7fb3aa0833661f7333
2013-01-21 19:05:51 -08:00
Joe Gordon
2704a40e57 Fix multi line docstring tests in hacking.py
* Fixes N403, along with docstring test
* Adds N404, multi line start
* Disable N403 and N404 until all cases are fixed
* Remove obsolote '--repeat' flag from tox.ini

Change-Id: Ibf6371efc1cdc764e66f6f7c5c4362440101f963
2013-01-17 13:24:31 -05:00
Jenkins
3c6daa9a0b Merge "Use testrepository setuptools support." 2013-01-16 17:19:37 +00:00
Monty Taylor
a0c3e67cb5 Use testrepository setuptools support.
The previous false-positive bug was fixed upstream in 0.0.13.

Change-Id: Iceb07a511992249c104bbc7c34b7c35e01bd9fbd
2013-01-14 23:17:12 -08:00
James E. Blair
9576c6fdeb Revert "Use testr setuptools commands."
This reverts commit c945b71929d7dc872ea27103ca8b6ec3d5cc7cd5
which caused the unit test command to always exit sucessfully.

Change-Id: I410cbc878675503f0ce6e50bddc4fd0552370c46
2013-01-14 14:05:50 -08:00
Sean Dague
313279f99a enable hacking.py self tests
pep8 and hacking.py (by inheritance) have the ability to run self
tests via docstrings, if we format the comments correctly, and change
the docstring test regex. Include self testing in our normal pep8
tests.

fix the comments to actually be test case compliant

fix nova_import_alphabetical to pass on short import lists (only 2
lines, starting at the beginning of a file)

comment where it doesn't appear that hacking.py tests are working
(though there could be deeper black magic in some of them).

fix a doc string in pep8 self tests that breaks N301

fix the fact that pep8 error strings are supposed to start with
\w\d\d\d, we were just lucking out on some formatting because the
string NOVA happened to be only 4 chars long.

Add a couple of escapes so that we don't recursively fail on our
own fail self tests for comments. (boy that's fun)

Change-Id: I14d5e04cfc092b37ec4967cd7485e9975a5eb820
2013-01-14 14:39:49 -05:00
Monty Taylor
c945b71929 Use testr setuptools commands.
Replace the embedded shell scripts with setuptools commands provided by
upstream. Cleans up a few warnings that this cause tox to show.

Change-Id: I9eb47142eafb6ed038b358b94984462ca1632542
2013-01-12 13:33:42 -08:00
Sean Dague
edca0c2a3d make runtests -p act more like tox
now that all the N402 fixes are in, start enforcing it

Change-Id: I9d495849bf698e5af142d0ccafe551ac0e8a600c
2013-01-10 08:57:49 -05:00
Jenkins
fa4696ff03 Merge "fix N401 errors, stop ignoring all N4* errors" 2013-01-09 01:01:57 +00:00
Sean Dague
37bfdd3b38 fix N401 errors, stop ignoring all N4* errors
We had previously been ignoring all our custom N4xx hacking.py
errors. This fixes all the N401 errors "doc strings
should not start with a space" and reduces the ignore set down
to N402 only "single line docstrings should end with period".

It also fixes the N401 parser to catch only docstrings, and
not tripple quoted string blocks used later on in a function.

Clean up a few of the more crazy uses of """ in our code

Clean up additional funky comments to make indents a bit more
consistent, and pull in lines when possible.

Change-Id: I9040a1d2ca7efda83bd5e425b95d1408b5b63577
2013-01-07 21:39:58 -05:00
Clark Boylan
7224fde0f0 Invert test stream capture logic for debugging.
Previously the default test behavior was to capture stdout and stderr.
Make the new default to not capture stdout and stderr then set
.testr.conf to always capture these streams when running tests under
testr.

The motiviation behind this change is that you will want these streams
to be captured when running under testr, but when not running under
testr you may not want to capture them. An example of this would be
running `python -m testtools.run test_name` with a change to test_name
to invoke the python debugger (capturing stdout and stderr interferes
with normal debugger functionality).

Also, only invoke the test timeout by default when running under testr.
This is done for the same reason as above. When running a test under the
debugger the timeout interferes with debugging.

Change-Id: I42cbbdadb2f221ec439e92a6800d14e8436bb77b
2013-01-07 11:03:30 -08:00
Clark Boylan
4abc8cc64f Use testr to run nova unittests.
Convert nova from using nosetests to testr for its test runner. Some
tests had to be modified to get them to run properly under testr.

run_tests.sh has been updated to run testr instead of nosetests.

Coverage is collected by running subunit.run under coverage.py when the
coverage environment is selected.

Note that you will need to rebuild your virtualenvs as nose is being
removed from the dependency lists and is being replaced by testr. Tests
will run in different processes once this test is merged so you cannot
use test classes to pass information between tests. Each test should be
a proper independent unit. Additionally the -x and -d flags to
run_tests.sh have been removed as there are currently no decent
approximations for those functions.

Change-Id: I019ca098972ca749b195f59968cf21edd5ba9109
2012-12-14 14:22:20 -08:00
Joe Gordon
a44ace3196 Fix pep8 exclude logic for 1.3.3
* With pep8 1.3.3 the exclude logic monkey patch broke causing
  tools/hacking to run on openstack/common code
* Also add .git to exclude list

Fix bug 1086410

Change-Id: Ifc50ac1963c50c338432bbb4b1e15cdf58ed2128
2012-12-06 11:19:17 +02:00
Joe Gordon
65ee45c641 Add pyflakes option to tox
* Uses pyflakes wrapper to ignore gettext errors
* Based on: https://bugs.launchpad.net/pyflakes/+bug/844592

Change-Id: I4fd947176f5258f89f517acf5dda2ded4a2da15e
2012-12-04 11:53:29 +02:00
Chuck Short
560e5cd84e Pin pep8 to 1.3.3
Apart of making pep8 version standard across all openstack
projects.

With this change we ignore E712 since it is normal to use
"column == True" in sqlalchemy.

Change-Id: I73a162847a79558cb158112878033edf18039805
Signed-off-by: Chuck Short <chuck.short@canonical.com>
2012-11-19 21:20:42 -06:00
Yun Mao
add01f874a Upgrade pylint version to 0.26.0
Upgrade pylint to reduce false positives in lintstack.

Change-Id: I10ab7a46c6a70daf9151f03cd9fea3c4851e6ebb
2012-11-12 10:14:35 -05:00
Dan Prince
f0437edd8b Make tox.ini run pep8/hacking checks on bin.
This updates the pep8 hacking checks in our tox.ini file so that
we are also scanning the bin directory for HACKING violations.

Change-Id: I4b010ef4dd799f0afe0fcd56c823a611021e8f60
2012-11-09 12:28:46 -05:00
Joe Gordon
c39dbb53bc Fix and enable pep8 E502, E712
pep8 E502 and E712 were temporarily disabld when upgrading to pep8 1.2

Change-Id: If5158304cb8e73ef844dea9f726df6f8fe305269
2012-10-22 13:47:36 -07:00
Joe Gordon
fbc4568eb0 Update tools hacking for pep8 1.2 and beyond
* b9f72b1601
  broke tools/hacking.py
* Upgrade pep8 to 1.2 and disable the following new tests: E12,E711,E712,E721,E502

Change-Id: I32a8808c6c9cccfedcc4d2a26649333aca1cd713
2012-10-01 16:33:35 -07:00
James E. Blair
6e9f3bb10a Revert "Add full test environment."
This reverts commit 30e513309fe25af358b4d9ed47da2c7574fc0f06.

The python mysql module was removed from test-requires in favor
of test-options, but the work to actually use that was never
completed.  Revert the change so that mysql testing is actually
performed again.

Change-Id: Iaf35811cc748272229260f2dab95520995a69b44
2012-09-28 13:04:53 -07:00
Joe Gordon
0876cf5bc5 Do not run pylint by default
pylint creates some files, and leaves git in a detached HEAD.

pylint can still be run via 'tox -epylint'

Change-Id: Ic0193f2488f1c6eb4a9f0fabbe24ac9869239890
2012-08-28 15:14:11 -07:00