252 Commits

Author SHA1 Message Date
ForestLee
d02a66d6bf Add a hacking rule for string interpolation at logging
String interpolation should be delayed to be handled by
the logging code, rather than being done at the point
of the logging call.
See the oslo i18n guideline
* https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#adding-variables-to-log-messages
and
* https://github.com/openstack-dev/hacking/blob/master/hacking/checks/other.py#L39
Closes-Bug: #1596829

Change-Id: Iba231be2476dcbeeb0edd76d6a921e549d183758
2017-07-18 14:11:28 +08:00
zlyqqq
6ac9254bc4 Remove duplicated hacking rule
Remove the hacking rule: CheckForAssertingNoneEquality and
enable H203.

http://git.openstack.org/cgit/openstack-dev/hacking/tree/HACKING.rst

Change-Id: Id5c76e4befde163cfce3badbcd329e94485daed3
2017-06-28 07:48:03 +00:00
Lance Bragstad
d28f4cd97c Cleanup policy generation
This commit only adds the sample policy file to the projects
.gitignore file in the interest of keeping environments cleaner.
This also adds the genpolicy tox environment to the tox environment
list to be consistent with how we maintain other tox environments.

Change-Id: Idf538c97f5f7383d8d6e5c18c985d9715033926b
2017-05-02 15:08:41 +00:00
Colleen Murphy
4b77186a75 Move policy generator config to config-generator/
etc/ is for config files that should be installed on disk to support a
running keystone. config-generator/ is for files that
oslo-config-generator uses to generate sample config files and are not
needed by keystone in a live deployment.

Change-Id: I0614709ded739d77ec620150bcb6c2f456024b6f
2017-04-21 21:47:32 +00:00
Anthony Washington
9bb1556817 Add policy sample generation
This adds the entry point and config file necessary for using the
oslo.policy sample generation script. It also adds a tox target to
simplify the usage of it.

Change-Id: I7feb522b2ea5f56e48982169c7ebbb2ec2ef2cb3
Partially-Implements: bp policy-in-code
2017-03-29 15:48:57 +00:00
Luigi Toscano
d9f896d430 Use ostestr instead of the custom pretty_tox.sh
Switch to os-testr as wrapper for testr, instead of the custom
script pretty_tox.sh.

Change-Id: If10ac96ac32813714be2b83f895071820126dbd8
2017-02-09 16:03:59 +01:00
Eric Brown
a2a06d0531 Remove references to Python 3.4
Now that there exists only a gate job for Python 3.5 and not 3.4,
we should remove those references to the 3.4 that is untested.

Change-Id: Ia017d1199f8ca31808680b91c708823711127659
2017-01-06 07:30:00 +00:00
Davanum Srinivas
b4012e8085 Fix issues with keystone-dsvm-py35-functional-v3-only on py35
keystone/token/providers/fernet/token_formatters.py
* decode payload[2] from bytes to string before comparing
  with a string (CONF.identity.default_domain_id)

keystone_tempest_plugin/services/identity/clients.py
keystone_tempest_plugin/services/identity/v3/auth_client.py
keystone_tempest_plugin/services/identity/v3/identity_providers_client.py
* decode the response body from bytes to string before we
  try to parse the json using json.loads

Change-Id: I98053bc498d78c5f0076a66e725ff2d634f5b663
2017-01-05 21:59:57 +00:00
Steve Martinelli
8a66ef6354 Remove support for PKI and PKIz tokens
This is the first step of several to remove PKI token support in
keystone. A large issue in removing PKI support is support for the
revocation list must be maintained.

This patch removes support for the token format, it's surrounding tests
and examples that are generated. Additionally, some wording has been
changed around the CLI and config options to make the distinction
between keys and certs used for PKI tokens and those used for getting
the revocation list (a list of tokens that are revoked, which is signed).

Future patches will:

- Remove the keystone-manage commands for generating certs

- Modify the revocation list (at /auth/tokens/OS-PKI/revoked) to return
a 403 if pki is not configured (instead of raising a 500). We cannot
remove the API as that would break an API contract.

- Options to configure PKI will be marked as deprecated

- If PKI is configured a normal signed list will be returned (same
behavior as today)

- Follow up patch to keystonemiddleware will make sure auth_token does
not rely on the revocation api at all.

Related-Bug: 1626778
Related-Bug: 1626779

Co-Authored-By: Boris Bobrov <bbobrov@mirantis.com>
bp removed-as-of-ocata
Change-Id: Icf1ebced44a675c88fb66a6c0431208ff5181574
2016-11-01 22:05:01 +00:00
Jenkins
59f5054e25 Merge "Add bindep environment to tox" 2016-11-01 12:11:30 +00:00
Kristi Nikolla
75e8cd1538 Add structure for Devstack plugin
This review creates the structure for the Devstack plugin and
prints to the console to ensure its execution in the gate.

Follow-up reviews will do more useful stuff like setting up
the environment for our functional testing (ldap, federation).

Change-Id: I820ae355ae8f3183fee2b8207e3c17e8bd10dc17
2016-10-31 10:03:16 -04:00
Jeremy Liu
a4fdb40527 Add bindep environment to tox
This is helpful for listing any missing system requirements [1].

[1] http://docs.openstack.org/infra/manual/drivers.html#package-requirements

Change-Id: I54e3955ff0befae35eb1aada478023fa00afc5e7
2016-10-29 14:57:23 +00:00
Steve Martinelli
3b17b277bf remove legacy driver tox target
Another follow up from 810e15689b200f92980e04e233523d72d885d4bd.

Change-Id: I5b16b30a3891a27359b3c238f10a4a13f5281866
Depends-On: Ic4b067b472dad3b8231cb2397abbb92a52939c31
2016-10-05 11:17:08 -04:00
Steve Martinelli
810e15689b Remove stable driver interfaces
bp removed-as-of-ocata

Change-Id: I4672cf7d9d72ef725212085972dbcd90db0e47cf
2016-09-28 17:18:37 +00:00
Steve Martinelli
85ae2454c9 Constraints are ready to be used for tox.ini
Per email[1] from Andreas, we don't need to hack at install_command
any longer.

[1] http://openstack.markmail.org/thread/a4l7tokbotwqvuoh

Change-Id: Ic224c1e20693410c485e45cab5bdaa5d96192f09
2016-08-12 14:01:22 -04:00
Tin Lam
6b52756f09 Fix warning when running tox -e api-ref
Running `tox -e api-ref` prints warning:

  WARNING:test command found but not installed in testenv
    cmd: /bin/rm
    env: /opt/stack/keystone/.tox/api-ref
  Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.

Change-Id: I4c1718dac22c374f048d9e04f42051e10021bb31
2016-08-08 23:51:18 -05:00
lilintan
97a63416ee Don't include openstack/common in flake8 exclude list
The directory openstack/common was used to keep codes from
oslo-incubator, we have retired oslo-incubator, so don't use
this directory any more.

Change-Id: I0637c17f5c7aecf5bda87c8e7a6ef834be299baa
2016-08-02 20:52:32 +08:00
Andreas Jaeger
a6c77639a1 Use constraints for coverage job
OpenStack CI supports now constraints in the coverage job - as a first
job to check that the recent changes for zuul-cloner work correctly.

Use constraints for cover environment.

Change-Id: I3812776ab228bf28df9934273df7fe8ee0880660
Depends-On: I2c0fb1e9b5a51be409687d9cb85ee3ae70c1ce94
2016-07-15 19:59:12 +02:00
Jenkins
d7a08e05fc Merge "Include doc directory in pep8 checks" 2016-07-02 04:40:40 +00:00
Eric Brown
67a50b5bcf Include doc directory in pep8 checks
Currently, flake8 runs against doc related directories such as
releasenotes and api-ref. Might as well remove doc from the
flake8 exclude list.  Each of these directories has only one
python file (conf.py).

Change-Id: I0445ad083d8d9167e0309950c200c9abb766bc1a
2016-06-30 14:14:13 -07:00
David Stanek
d18bb02741 Update the nosetests test regex for legacy tests
The default regex, '(?:^|[\b_\./-])[Tt]est', will match methods that are
not test methods. For example, I created a new method
'skip_test_overrides' and it was mistakenly run as test. By convention
all of our tests start with 'test_' so this is a safe change.

Change-Id: I4f5917de53681c950c72479f16f88e76f49f6bfc
2016-06-29 19:48:12 +00:00
Jenkins
8b415b645f Merge "Let setup.py compile_catalog process all language files" 2016-06-07 10:35:52 +00:00
Dolph Mathews
81e5d8ef12 PEP257: Ignore D203 because it was deprecated
PEP257's D203 check ensures that you have a blank line before class
docstrings. This rule directly conflicts with D211 (no blank lines
before class docstrings), which is intended to supersede D203.

The original language in PEP257 which D203 was based on was actually
removed from PEP257 by Guido here:

  https://hg.python.org/peps/rev/9b715d8246db

Change-Id: Icc048b947acea8f655d00540c221123b906e7545
2016-06-03 08:45:45 -05:00
Samuel de Medeiros Queiroz
9a5395f1b2 Migrate identity /v3 docs from api-ref repo
Docs at [1] have already been converted from WADL (SGML / XML) to RST
using fairy-slipper [2].

This commit polish the results from the conversion and migrate the docs
to our repository under 'api-ref/source' directory. In addition, it
added missing descriptions for access_token_id and consumer_id to
'parameters.yaml'.

Polishing the generated RST files include:
- Removing unnecessary blank lines;
- Removing empty references.

Polishing the generated RST files do not include:
- Modifying their content;
- Modifying file names;
- Wrapping lines at the maximum of 79 chars.

Updating the documentation will be done after this migration step.

This change also adds a tox environment to build the docs using sphinx
under 'api-ref/build', which in turn is added to '.gitignore'.

Lastly, 'os-api-ref' is added as a test requirement. It provides the
sphinx stanzas rest_method and rest_parameter, used to define OpenStack
APIs in RST docs.

[1] https://github.com/openstack/api-site/tree/master/api-ref/source/identity/v3
[2] https://github.com/russell/fairy-slipper

Change-Id: If1b9a3e1b2e4ea7211c337071254c26b881893a3
2016-05-24 09:58:23 -03:00
Morgan Fainberg
94391a3240 Remove helper script for py34
Remove the helper script for py34, it prevents the use of arguments
being passed down e.g. tox -epy34 -- --failing due to the lack of
passing {posargs} to the script. Since py34 has no special exceptions
now, the specific env for py34 with a special pretty_tox script is
not needed.

Change-Id: I4d960d82e2844c0c82a14be5e92230f1698e11b3
2016-05-21 23:46:58 +00:00
Sven Anderson
16d0cdba47 Let setup.py compile_catalog process all language files
Two years ago the translation files have been split into several
files, separating the log messages of different log levels from each
other, like X.pot, X-log-warning.pot, X-log-info.pot, and so on.
However, the setup.py command `compile_catalogs`, that comes from the
babel package and compiles the corresponding .po files into .mo
files, only supported one file per python package.  This means that
during packaging `compile_catalogs` never compiled the X-log-*.po
files, so the corresponding translations were always missing.

Since babel 2.3 the domain can be set to a space separated list of
domains.  This change adds the the additional log level files to the
domain list.

The obsolete check that .po and .pot files are valid is removed from
tox.ini.

Change-Id: I1f0bfb181e2b84ac6dd0ce61881cd2cc4400bdcb
Closes-Bug: #1536226
2016-05-20 17:55:27 +02:00
Navid Pustchi
ce574c31ad Fixing D105, D203, and D205 PEP257
Currently tox ignores D105, D203, and D205.
D105: Missing docstring in magic method.
D203: 1 blank required before class docstring.
D205: Blank line required between one-line summary and description.
This change removes it and make keystone docstring compliant with it.

Change-Id: Id916612dd246e02159907d286e448545120f3655
2016-05-03 20:44:14 +00:00
Jenkins
f7b33213f1 Merge "Fix D400 PEP257" 2016-04-21 19:37:08 +00:00
Navid Pustchi
2c4f948db0 Fix D400 PEP257
Currently tox ignores D401.
D400: First line should end with a period.
This change removes it and make keystone docstring compliantwith it.

Change-Id: I9a9520e69701718ff471eebbcc52199dacdd9c68
2016-04-21 15:25:34 +00:00
David Stanek
a5968655e8 Bump the required tox version to 2.3.1
Other projects are already starting to make this change and it should
help with some of the strange issues we commonly get asked in IRC. When
those issues come up the first step is always update to the latest tox
and pip, so we might as well make that recommendation more formal.

Change-Id: Ic320881797b99747357ac14c6f84636e5070523d
Related-Bug: #1572202
2016-04-20 00:10:23 +00:00
Jenkins
23bb657369 Merge "Remove eventlet support" 2016-04-19 07:29:42 +00:00
Jenkins
82613ddc4e Merge "Remove comment from D202 rule" 2016-04-18 18:44:29 +00:00
Steve Martinelli
ac039414ce Remove eventlet support
Eventlet has been deprecated since the Kilo release and is
being removed in Newton.

A follow on patch will be proposed to remove the [ssl] section
since it is now redundant.

Co-Authored-By: Grzegorz Grasza <grzegorz.grasza@intel.com>
Partially implements: bp removed-as-of-newton

Change-Id: I963d94bbd188dbb6eba68623a42c5bc3f2289da4
2016-04-18 18:07:28 +00:00
Andreas Jaeger
2535f22e61 Fix post jobs
The tox venv environment is run during post jobs and thus cannot use
constraints.

See:
http://logs.openstack.org/db/db7bdf9aa0cb0ba5fbae5ae07ecdb9f024213deb/post/keystone-docs/92d1e87/
http://logs.openstack.org/db/db7bdf9aa0cb0ba5fbae5ae07ecdb9f024213deb/post/keystone-branch-tarball/eb685ad/

We run for docs jobs:
"tox -e venv python setup.py build_sphinx"

thus, the docs environment is not used.

For branch tarball, the infra scripts use:
"tox -e venv python setup.py sdist"

And infra does not setup constraints for post jobs currently as this is
not working with current tools

Fix tox.ini for this

Change-Id: I048368981e4be739c66073fdd9bc8a9663498a80
2016-04-17 22:07:00 +02:00
Davanum Srinivas
db7bdf9aa0 Keystone jobs should honor upper-constraints.txt
Some targets don't respect upper-constraints like
cover and releasenotes, so make sure don't use
the same install_command for those jobs.

Temporarily add Babel to avoid problem with the
keystone-coverage-db coverage job.

Change-Id: I8636e7c86c6c5c608429fab88e181108ae615db9
2016-04-17 13:53:41 -04:00
Navid Pustchi
aabc213040 Fix D401 PEP8 violation.
Currently tox ignores D401 (401: First line should be in imperative mood).
This change removes it and make keystoneauth docstring compliantwith it.

Change-Id: I136cf810f47c4c19f29216907a63f226930b5082
Partial-Bug: 1570049
2016-04-14 20:08:52 +00:00
Jenkins
dee5d8b2f3 Merge "Add py3 debugging" 2016-04-06 21:18:18 +00:00
Rodrigo Duarte
4a8023aef2 Remove comment from D202 rule
The rule is no longer ignored in the list below the comment.

Change-Id: Ib8e4094493e1c7841bda0d0fc7251189b90072a9
2016-04-04 19:08:06 -03:00
Tom Cocozzello
9a92c471e4 Add patch_cover to keystone
`patch_cover` checks to make sure your patch has
adequate test coverage.

How it works is it stashes your changes, runs `tox -e cover`,
saves the results then stash pops your change and runs cover
again.  It then reports to show if your code in your patch
has 100% test coverage or not.

The intent of this job is to be a non-voting / warning like
job that can inform reviewers if the review clearly needs
more testing.

Since cover.sh is copied from [0] the copyright should
reflect the owners of the code.

[0]: https://github.com/openstack/rally/blob/master/tests/ci/cover.sh

Change-Id: I66572072aa4d9563b6e1f73a8f8b3437116ef3c4
2016-03-28 21:26:01 +00:00
Tom Cocozzello
6bd2da1d3c Add py3 debugging
Add a tox testing environment that allows users to
debug with python 3.

Change-Id: Iaf757de29936e1361fd720710154963da76a7a10
2016-03-18 15:46:33 -05:00
Eric Brown
e28fb14a7c Explicitly exclude tests from bandit scan
Now that the bandit linter runs without a bandit.yaml specified,
it should exclude the tests directory similar to how it did before
in the bandit.yaml.

The reason this has not caused a problem yet is because bandit is
still finding a config bandit.yaml in .tox/pep8/etc/bandit/bandit.yaml.
But in a newer version of bandit this will not be the case.

Change-Id: Ie81d052cd57318a987f0ebfe718e1c01bdbffcd3
2016-03-08 16:27:05 -08:00
Brant Knudson
049fbe459b Fix warning when running tox
When running tox it would print out a warning:

  WARNING:test command found but not installed in testenv
    cmd: /usr/bin/find
    env: /opt/stack/keystone/.tox/py27
  Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.

This is because we forgot to specify a dependency.

Change-Id: I4270c52213159cefa04a63979ee0bb2e1a2fa35f
2016-03-07 18:43:00 -06:00
Sean Dague
9bdf6593e5 remove pyc files before running tests
pyc files for old code files and test files won't get removed from the
local tree when those files are removed. They will still be discovered
and used when running tox tests.

This removes pyc files before running tests to ensure that test
results are repeatble by end users even if they are updating older git
trees.

Closes-Bug: #1551312

Change-Id: I9987cc57dc9ac2bd0331af6cb59e2c5e137d9849
2016-03-01 22:20:04 +00:00
Brant Knudson
63741c9c6b Switch to configless bandit
The 0.17.3 release of bandit supports running with no config,
enabling all the tests with a default configuration for them, switch
to that since it's easier to maintain.

Change-Id: I507b7a11e8d1d69a6d9ec88493cbefe48be5ddcf
2016-02-22 13:10:18 -06:00
Tin Lam
2bad130bf4 Removing H405 violations from keystone
Keystone's tox.ini contains an "ignore" entry for H405 violations:
multi line docstring summary not separated with an empty line.
All violations of H405 should be fixed so that H405 can be removed
from the ignore list.

Change-Id: I1b2aae0cabc20909cf3b0a405d5e31c5d91148b2
Closes-Bug: #1482773
2016-02-21 03:47:55 -05:00
Eric Brown
1a4c502dec Add back a bandit tox job
The bandit project uses an integration test to ensure it doesn't
break other projects by introducing new changes. To run this
integration, it necessary to have a common tox target of 'bandit'.

Change-Id: I07c66fa2535211536d0218abd86bea0ae8644065
2016-02-17 14:09:08 -08:00
David Stanek
8f15724dc7 Stop using nose as a Python3 test runner
There are several reasons for this change:
- removes the need for the custome py3 test dependencies
- makes the broken tests more visible because they are listed
  instead of the working ones
- using a blacklist means that new test files are tested by default in
  py3

bp python3

Change-Id: I7472eec05d80cdd99c0ee376159578e0d7306f51
2016-02-11 06:40:24 +00:00
Andreas Jaeger
9f58f97cae Make pep8 *the* linting interface
According to the PTI (=Python Test Interface,
http://governance.openstack.org/reference/cti/python_cti.html), pep8
is the interface for codestyle checks. Move all tests from linters to
pep8.

This basically reverts Ic9efdcb13978d2f29312a4a1f0fdd859d55c8398.

Depends-On: I9346e4bf8c3a50de088c6a157f0d871c0a9333a1
Change-Id: I99df6de57d345c40e39801d66575d35d0d8a36a9
2016-02-10 20:54:32 +00:00
David Stanek
7a0874f6f6 Disable the ipv6 tests in py34
Eventlet 0.18.2 breaks our py34 tests. Disabling these test makes it
work again.

Change-Id: Ie69342ae26dd031e3121e23404b31a133e53cf0e
2016-02-10 01:10:13 +00:00
Jenkins
a27f5337b5 Merge "test_credential.py work with python34" 2016-02-06 00:44:13 +00:00