Commit Graph

267 Commits

Author SHA1 Message Date
Vishakha Agarwal
a022e27307 Drop py35 jobs
Python 3.5 was the target runtime for the Rocky release.
The current target py3 runtime for Stein is Python 3.6,
so there is no reason to keep testing against the older
version.

https://governance.openstack.org/tc/reference/runtimes/stein.html#python-runtime-for-stein

Change-Id: I618c0d263d4c0f497aef59f24215b60169948ea9
2019-03-05 10:56:57 +05:30
Colleen Murphy
76165b3f56 Add py37 tox env
With python3.7 jobs added to the Zuul jobs, it makes sense to add this
to the tox config too so that developers can run python3.7 tests as
well.

Change-Id: I87ba933d57de746651427386d76d5b70b7cf45b8
2019-02-25 12:07:00 +01:00
wangxiyuan
090886902c Use pycodestyle in place of pep8
The lower version of pycodestyle lib (aliased to pep8) doesn't work
for py36 env. This commit unblocks the py36 gate by adding a
dependency on pycodestyle and using that for style checks.

Bump the "hacking" lib version to v1.1.0 which depends
on a higher verion of pycodestyle.

Change-Id: I3fc591e09c1e25a3bd2a3922880772ea9617f1e3
2018-11-20 17:16:01 +00:00
Colleen Murphy
fb67da8016 Clean up python3.5 usage in tox.ini
We already have a debug env that uses python3 as the base python, it's
not that useful to have another one specific to 3.5. Also relax the
functional test environment to allow any python3 so that we have
flexibility in the future if we want to update it.

Change-Id: Ib43b41e0a1ce427d161b51227d0ffe0d4dec0a98
2018-11-06 12:54:32 +01:00
Colleen Murphy
605da1015f Add py36 tox environment
We already run python3.6 unit tests in CI. Add the py36 environment to
the tox file so that developers with python3.6 available locally can opt
into running that version too.

Change-Id: I207db150716139bd55d31e4a4c6eb7e7f48916b0
2018-11-06 12:52:15 +01:00
Nguyen Van Trung
104956c65e Don't quote {posargs} in tox.ini
Quotes around {posargs} cause the entire string to be combined into one
arg that gets passed to stestr. This prevents passing multiple args
(e.g. '--concurrency=16 some-regex')

Change-Id: Ia3353073bbdf9d18bbd5bd97425b3519f3d5aad0
2018-10-09 10:14:09 +07:00
Tony Breeds
114cdeb7eb Move use of constraints out of install_cmd
If you have a -c in the install_cmd it gets used with all the deps
supplied this means that the lower-constraints job actually install from
upper-constraints :(

You can see what I mean in [1]
Note both lower-constraints.txt and upper-constraints.txt are used ; and
---
Collecting oslo.log===3.39.0 (from -c /home/zuul/src/git.openstack.org/openstack/requirements/upper-constraints.txt (line 247))
---

With this fixed we find a few minimums that needs to be bumped:

 * oslo.policy >= 1.33.0
   keystone uses the scope_types[2] kwarg to RuleDefault which was
   introduced in 52c82ff9ab04dd78ff7045cb30d2f5de535dd7da which is
   contained in 1.32.0  ; also we need the 'policy-in-code' feature
   which is in 1.33.0
 * oslo.log >= 0.38.0
   keystone used the ROCKY[3] constant for deprecations which was
   introduced in d68a895ee8e61b5c9d4ef368e7f04252e84649e9 which is
   contained in 3.38.0
 * msgpack >= 0.5.0
   the 0.4.x versions have been removed from pypi so we have to bump the
   minimum :(
 * SQLAlchemy >= 1.0.13
   identity_provider_id in token payload is byte in python3 which
   triggers a sqlalchemy bug[4]. The bug has been fixed in 1.0.13
 * keystonemiddleware >= 5.1.0
   unified limit feature uses system scope feature which is supported
   in keystonemiddleware after 5.1.0
   

We also add correct some errors in bindep.txt related to use on Fedora

[1] http://logs.openstack.org/47/599447/2/check/openstack-tox-lower-constraints/bbc912b/tox/lower-constraints-1.log
[2] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/policies/access_token.py#n24
[3] http://git.openstack.org/cgit/openstack/keystone/tree/keystone/conf/default.py#n50
[4] http://docs.sqlalchemy.org/en/latest/changelog/changelog_10.html#change-a14dd2e73d889d065acc07a77b1ee7cb

Change-Id: Ic0de6799fddd86a70abae2c87c92d565072ebdb9
2018-09-12 03:54:38 +00:00
Doug Hellmann
f5017d9ccd fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I4660a35ff63f3f0212978dda668ba2cb08774e3a
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-06 16:06:00 -04:00
Nguyen Hai
a04a1dc782 Follow the new PTI for document build
- Follow new PTI for docs build
- Add sphinxcontrib.apidoc to replace pbr autodoc

REF:
https://governance.openstack.org/tc/reference/project-testing-interface.html
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128594.html

Depends-On: https://review.openstack.org/#/c/559334/
Change-Id: I1d5f01cf49f36567f5a891c69c7269d3e966b9c5
2018-04-09 01:13:58 +09:00
Doug Hellmann
6cb71c36c4 add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: Ide4d192e390ac78f860634014f2437dd6ea7b919
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-03-23 08:40:54 -04:00
Zuul
ccc9893fa5 Merge "Migrate functional tests to stestr" 2017-12-22 16:47:26 +00:00
qinglin.cheng
07bba320ae Create doc/requirements.txt
For compliance with the Project Testing Interface as described in:

https://governance.openstack.org/tc/reference/project-testing-interface.html

Refer to:

http://lists.openstack.org/pipermail/openstack-dev/2017-November/124815.html

Change-Id: Ic05a55fb14741bb0802c217c81b0035b6e1f7de2
2017-12-19 16:17:26 +01:00
Gage Hugo
3b209ed938 Migrate functional tests to stestr
The functional tests in tox.ini were still set to use testr,
this change makes it use the new stestr.

Change-Id: I2ca4e2ae5d8395ae27f8d14cce8038c737aa65f2
2017-12-14 09:28:48 -06:00
Lance Bragstad
c5f5c2c219 Use stestr directly instead of ostestr
This is inline with the direction from:

  http://lists.openstack.org/pipermail/openstack-dev/2017-September/122135.html

This also makes it easier to invoke tests based on a regular
expression. Previously, it was possible to do:

  $ tox -e py27,py34 -- keystone.tests.unit.test_module.TestCase

But that isn't supported in the way we use ostestr in tox with the
new stestr changes. Instead of refactoring our ostestr usage we can
just replace it with stestr directly which allows us to invoke tests
using a single regex and is inline with the direction of moving away
from ostestr.

Change-Id: I83eb6ce905c45c9c388a8d5a3c9369f5c5f6710e
2017-09-29 20:35:13 +00:00
Gage Hugo
71e543152e Migrate to stestr
With the latests changes to os-testr[0] it now expects
a .stestr.conf file instead of a .testr.conf file. This
change migrates the testing suite from using testr to
use stestr instead.

[0] http://lists.openstack.org/pipermail/openstack-dev/2017-September/122135.html

Change-Id: Ie0de862f2192491fa7752af03c8c03b3ee36337a
2017-09-22 11:07:09 -05:00
ForestLee
d02a66d6bf Add a hacking rule for string interpolation at logging
String interpolation should be delayed to be handled by
the logging code, rather than being done at the point
of the logging call.
See the oslo i18n guideline
* https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#adding-variables-to-log-messages
and
* https://github.com/openstack-dev/hacking/blob/master/hacking/checks/other.py#L39
Closes-Bug: #1596829

Change-Id: Iba231be2476dcbeeb0edd76d6a921e549d183758
2017-07-18 14:11:28 +08:00
zlyqqq
6ac9254bc4 Remove duplicated hacking rule
Remove the hacking rule: CheckForAssertingNoneEquality and
enable H203.

http://git.openstack.org/cgit/openstack-dev/hacking/tree/HACKING.rst

Change-Id: Id5c76e4befde163cfce3badbcd329e94485daed3
2017-06-28 07:48:03 +00:00
Lance Bragstad
d28f4cd97c Cleanup policy generation
This commit only adds the sample policy file to the projects
.gitignore file in the interest of keeping environments cleaner.
This also adds the genpolicy tox environment to the tox environment
list to be consistent with how we maintain other tox environments.

Change-Id: Idf538c97f5f7383d8d6e5c18c985d9715033926b
2017-05-02 15:08:41 +00:00
Colleen Murphy
4b77186a75 Move policy generator config to config-generator/
etc/ is for config files that should be installed on disk to support a
running keystone. config-generator/ is for files that
oslo-config-generator uses to generate sample config files and are not
needed by keystone in a live deployment.

Change-Id: I0614709ded739d77ec620150bcb6c2f456024b6f
2017-04-21 21:47:32 +00:00
Anthony Washington
9bb1556817 Add policy sample generation
This adds the entry point and config file necessary for using the
oslo.policy sample generation script. It also adds a tox target to
simplify the usage of it.

Change-Id: I7feb522b2ea5f56e48982169c7ebbb2ec2ef2cb3
Partially-Implements: bp policy-in-code
2017-03-29 15:48:57 +00:00
Luigi Toscano
d9f896d430 Use ostestr instead of the custom pretty_tox.sh
Switch to os-testr as wrapper for testr, instead of the custom
script pretty_tox.sh.

Change-Id: If10ac96ac32813714be2b83f895071820126dbd8
2017-02-09 16:03:59 +01:00
Eric Brown
a2a06d0531 Remove references to Python 3.4
Now that there exists only a gate job for Python 3.5 and not 3.4,
we should remove those references to the 3.4 that is untested.

Change-Id: Ia017d1199f8ca31808680b91c708823711127659
2017-01-06 07:30:00 +00:00
Davanum Srinivas
b4012e8085 Fix issues with keystone-dsvm-py35-functional-v3-only on py35
keystone/token/providers/fernet/token_formatters.py
* decode payload[2] from bytes to string before comparing
  with a string (CONF.identity.default_domain_id)

keystone_tempest_plugin/services/identity/clients.py
keystone_tempest_plugin/services/identity/v3/auth_client.py
keystone_tempest_plugin/services/identity/v3/identity_providers_client.py
* decode the response body from bytes to string before we
  try to parse the json using json.loads

Change-Id: I98053bc498d78c5f0076a66e725ff2d634f5b663
2017-01-05 21:59:57 +00:00
Steve Martinelli
8a66ef6354 Remove support for PKI and PKIz tokens
This is the first step of several to remove PKI token support in
keystone. A large issue in removing PKI support is support for the
revocation list must be maintained.

This patch removes support for the token format, it's surrounding tests
and examples that are generated. Additionally, some wording has been
changed around the CLI and config options to make the distinction
between keys and certs used for PKI tokens and those used for getting
the revocation list (a list of tokens that are revoked, which is signed).

Future patches will:

- Remove the keystone-manage commands for generating certs

- Modify the revocation list (at /auth/tokens/OS-PKI/revoked) to return
a 403 if pki is not configured (instead of raising a 500). We cannot
remove the API as that would break an API contract.

- Options to configure PKI will be marked as deprecated

- If PKI is configured a normal signed list will be returned (same
behavior as today)

- Follow up patch to keystonemiddleware will make sure auth_token does
not rely on the revocation api at all.

Related-Bug: 1626778
Related-Bug: 1626779

Co-Authored-By: Boris Bobrov <bbobrov@mirantis.com>
bp removed-as-of-ocata
Change-Id: Icf1ebced44a675c88fb66a6c0431208ff5181574
2016-11-01 22:05:01 +00:00
Jenkins
59f5054e25 Merge "Add bindep environment to tox" 2016-11-01 12:11:30 +00:00
Kristi Nikolla
75e8cd1538 Add structure for Devstack plugin
This review creates the structure for the Devstack plugin and
prints to the console to ensure its execution in the gate.

Follow-up reviews will do more useful stuff like setting up
the environment for our functional testing (ldap, federation).

Change-Id: I820ae355ae8f3183fee2b8207e3c17e8bd10dc17
2016-10-31 10:03:16 -04:00
Jeremy Liu
a4fdb40527 Add bindep environment to tox
This is helpful for listing any missing system requirements [1].

[1] http://docs.openstack.org/infra/manual/drivers.html#package-requirements

Change-Id: I54e3955ff0befae35eb1aada478023fa00afc5e7
2016-10-29 14:57:23 +00:00
Steve Martinelli
3b17b277bf remove legacy driver tox target
Another follow up from 810e15689b.

Change-Id: I5b16b30a3891a27359b3c238f10a4a13f5281866
Depends-On: Ic4b067b472dad3b8231cb2397abbb92a52939c31
2016-10-05 11:17:08 -04:00
Steve Martinelli
810e15689b Remove stable driver interfaces
bp removed-as-of-ocata

Change-Id: I4672cf7d9d72ef725212085972dbcd90db0e47cf
2016-09-28 17:18:37 +00:00
Steve Martinelli
85ae2454c9 Constraints are ready to be used for tox.ini
Per email[1] from Andreas, we don't need to hack at install_command
any longer.

[1] http://openstack.markmail.org/thread/a4l7tokbotwqvuoh

Change-Id: Ic224c1e20693410c485e45cab5bdaa5d96192f09
2016-08-12 14:01:22 -04:00
Tin Lam
6b52756f09 Fix warning when running tox -e api-ref
Running `tox -e api-ref` prints warning:

  WARNING:test command found but not installed in testenv
    cmd: /bin/rm
    env: /opt/stack/keystone/.tox/api-ref
  Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.

Change-Id: I4c1718dac22c374f048d9e04f42051e10021bb31
2016-08-08 23:51:18 -05:00
lilintan
97a63416ee Don't include openstack/common in flake8 exclude list
The directory openstack/common was used to keep codes from
oslo-incubator, we have retired oslo-incubator, so don't use
this directory any more.

Change-Id: I0637c17f5c7aecf5bda87c8e7a6ef834be299baa
2016-08-02 20:52:32 +08:00
Andreas Jaeger
a6c77639a1 Use constraints for coverage job
OpenStack CI supports now constraints in the coverage job - as a first
job to check that the recent changes for zuul-cloner work correctly.

Use constraints for cover environment.

Change-Id: I3812776ab228bf28df9934273df7fe8ee0880660
Depends-On: I2c0fb1e9b5a51be409687d9cb85ee3ae70c1ce94
2016-07-15 19:59:12 +02:00
Jenkins
d7a08e05fc Merge "Include doc directory in pep8 checks" 2016-07-02 04:40:40 +00:00
Eric Brown
67a50b5bcf Include doc directory in pep8 checks
Currently, flake8 runs against doc related directories such as
releasenotes and api-ref. Might as well remove doc from the
flake8 exclude list.  Each of these directories has only one
python file (conf.py).

Change-Id: I0445ad083d8d9167e0309950c200c9abb766bc1a
2016-06-30 14:14:13 -07:00
David Stanek
d18bb02741 Update the nosetests test regex for legacy tests
The default regex, '(?:^|[\b_\./-])[Tt]est', will match methods that are
not test methods. For example, I created a new method
'skip_test_overrides' and it was mistakenly run as test. By convention
all of our tests start with 'test_' so this is a safe change.

Change-Id: I4f5917de53681c950c72479f16f88e76f49f6bfc
2016-06-29 19:48:12 +00:00
Jenkins
8b415b645f Merge "Let setup.py compile_catalog process all language files" 2016-06-07 10:35:52 +00:00
Dolph Mathews
81e5d8ef12 PEP257: Ignore D203 because it was deprecated
PEP257's D203 check ensures that you have a blank line before class
docstrings. This rule directly conflicts with D211 (no blank lines
before class docstrings), which is intended to supersede D203.

The original language in PEP257 which D203 was based on was actually
removed from PEP257 by Guido here:

  https://hg.python.org/peps/rev/9b715d8246db

Change-Id: Icc048b947acea8f655d00540c221123b906e7545
2016-06-03 08:45:45 -05:00
Samuel de Medeiros Queiroz
9a5395f1b2 Migrate identity /v3 docs from api-ref repo
Docs at [1] have already been converted from WADL (SGML / XML) to RST
using fairy-slipper [2].

This commit polish the results from the conversion and migrate the docs
to our repository under 'api-ref/source' directory. In addition, it
added missing descriptions for access_token_id and consumer_id to
'parameters.yaml'.

Polishing the generated RST files include:
- Removing unnecessary blank lines;
- Removing empty references.

Polishing the generated RST files do not include:
- Modifying their content;
- Modifying file names;
- Wrapping lines at the maximum of 79 chars.

Updating the documentation will be done after this migration step.

This change also adds a tox environment to build the docs using sphinx
under 'api-ref/build', which in turn is added to '.gitignore'.

Lastly, 'os-api-ref' is added as a test requirement. It provides the
sphinx stanzas rest_method and rest_parameter, used to define OpenStack
APIs in RST docs.

[1] https://github.com/openstack/api-site/tree/master/api-ref/source/identity/v3
[2] https://github.com/russell/fairy-slipper

Change-Id: If1b9a3e1b2e4ea7211c337071254c26b881893a3
2016-05-24 09:58:23 -03:00
Morgan Fainberg
94391a3240 Remove helper script for py34
Remove the helper script for py34, it prevents the use of arguments
being passed down e.g. tox -epy34 -- --failing due to the lack of
passing {posargs} to the script. Since py34 has no special exceptions
now, the specific env for py34 with a special pretty_tox script is
not needed.

Change-Id: I4d960d82e2844c0c82a14be5e92230f1698e11b3
2016-05-21 23:46:58 +00:00
Sven Anderson
16d0cdba47 Let setup.py compile_catalog process all language files
Two years ago the translation files have been split into several
files, separating the log messages of different log levels from each
other, like X.pot, X-log-warning.pot, X-log-info.pot, and so on.
However, the setup.py command `compile_catalogs`, that comes from the
babel package and compiles the corresponding .po files into .mo
files, only supported one file per python package.  This means that
during packaging `compile_catalogs` never compiled the X-log-*.po
files, so the corresponding translations were always missing.

Since babel 2.3 the domain can be set to a space separated list of
domains.  This change adds the the additional log level files to the
domain list.

The obsolete check that .po and .pot files are valid is removed from
tox.ini.

Change-Id: I1f0bfb181e2b84ac6dd0ce61881cd2cc4400bdcb
Closes-Bug: #1536226
2016-05-20 17:55:27 +02:00
Navid Pustchi
ce574c31ad Fixing D105, D203, and D205 PEP257
Currently tox ignores D105, D203, and D205.
D105: Missing docstring in magic method.
D203: 1 blank required before class docstring.
D205: Blank line required between one-line summary and description.
This change removes it and make keystone docstring compliant with it.

Change-Id: Id916612dd246e02159907d286e448545120f3655
2016-05-03 20:44:14 +00:00
Jenkins
f7b33213f1 Merge "Fix D400 PEP257" 2016-04-21 19:37:08 +00:00
Navid Pustchi
2c4f948db0 Fix D400 PEP257
Currently tox ignores D401.
D400: First line should end with a period.
This change removes it and make keystone docstring compliantwith it.

Change-Id: I9a9520e69701718ff471eebbcc52199dacdd9c68
2016-04-21 15:25:34 +00:00
David Stanek
a5968655e8 Bump the required tox version to 2.3.1
Other projects are already starting to make this change and it should
help with some of the strange issues we commonly get asked in IRC. When
those issues come up the first step is always update to the latest tox
and pip, so we might as well make that recommendation more formal.

Change-Id: Ic320881797b99747357ac14c6f84636e5070523d
Related-Bug: #1572202
2016-04-20 00:10:23 +00:00
Jenkins
23bb657369 Merge "Remove eventlet support" 2016-04-19 07:29:42 +00:00
Jenkins
82613ddc4e Merge "Remove comment from D202 rule" 2016-04-18 18:44:29 +00:00
Steve Martinelli
ac039414ce Remove eventlet support
Eventlet has been deprecated since the Kilo release and is
being removed in Newton.

A follow on patch will be proposed to remove the [ssl] section
since it is now redundant.

Co-Authored-By: Grzegorz Grasza <grzegorz.grasza@intel.com>
Partially implements: bp removed-as-of-newton

Change-Id: I963d94bbd188dbb6eba68623a42c5bc3f2289da4
2016-04-18 18:07:28 +00:00
Andreas Jaeger
2535f22e61 Fix post jobs
The tox venv environment is run during post jobs and thus cannot use
constraints.

See:
http://logs.openstack.org/db/db7bdf9aa0cb0ba5fbae5ae07ecdb9f024213deb/post/keystone-docs/92d1e87/
http://logs.openstack.org/db/db7bdf9aa0cb0ba5fbae5ae07ecdb9f024213deb/post/keystone-branch-tarball/eb685ad/

We run for docs jobs:
"tox -e venv python setup.py build_sphinx"

thus, the docs environment is not used.

For branch tarball, the infra scripts use:
"tox -e venv python setup.py sdist"

And infra does not setup constraints for post jobs currently as this is
not working with current tools

Fix tox.ini for this

Change-Id: I048368981e4be739c66073fdd9bc8a9663498a80
2016-04-17 22:07:00 +02:00
Davanum Srinivas
db7bdf9aa0 Keystone jobs should honor upper-constraints.txt
Some targets don't respect upper-constraints like
cover and releasenotes, so make sure don't use
the same install_command for those jobs.

Temporarily add Babel to avoid problem with the
keystone-coverage-db coverage job.

Change-Id: I8636e7c86c6c5c608429fab88e181108ae615db9
2016-04-17 13:53:41 -04:00