94 Commits

Author SHA1 Message Date
Hervé Beraud
632db25246 Refresh python versions
Introduce support of python 3.8 and move tox and jobs to py38.

jsonschema 3.2.0 [1] support python 3.8

[1] https://github.com/Julian/jsonschema/pull/627

Change-Id: Ibcfa044dd0f3b29fd290559795ea1d98e194e886
2020-05-26 10:36:06 +02:00
Monty Taylor
0c5ae59025 Stop subclassing object
This wasn't actually even needed in python2.7, but habits die hard.

Change-Id: I20f282d285506b1f1dde01be6cde0c0e8c7883c8
2020-05-10 08:29:41 -05:00
Andreas Jaeger
a0d75007a8 Update local hacking checks
Update local hacking checks for new flake8.

Change-Id: I1031badf36857a22efcd6ce6124f9db48bc18995
2020-03-29 11:07:34 -05:00
Monty Taylor
e7237b468f Turn off test log capture for now
We're hitting memory pressure test failures on test nodes.
Hypothesis is that we've got things logging too much and buffering
to RAM. While we track that down, turn off auto-logging.

Change-Id: Ie124f8dbd18600508a4a99ef91a5e0f5df00093c
2020-03-27 13:04:46 -05:00
Monty Taylor
185e55bd77 Add python-requires entry indicating 3.5 is required
We require 3.5 now but we didn't add the metadata. Add it.

While OpenStack broadly only supports 3.6, openstacksdk
has a wider audience, including nodepool which still uses
3.5. So add the 3.5 unit tests back.

Use lower-constraints for py35 unit tests because the
upper-constraints have versions of things that have
droppped py35 support already.

Depends-On: https://review.opendev.org/#/c/715467/
Change-Id: I680e0bab5ba3af7be5e914ece64c6bd25d1f2191
2020-03-27 12:13:24 -05:00
Artem Goncharov
d4ddd5ce99 Lay a foundation for the project cleanup
During Berlin Summit 2018 it was decided to start implementation
of the project cleanup in the SDK. This is a foundation for that
with the required bits and few examples in the services to demonstrate
the interface.

In the followup changes following should be done:
- review cleanup in service proxies (the ones implemented and not)
- describe the feature in the documentation
- start consuming this service from the OSC (need help deciding where
should it be implemented - core OSC, separate project, OSC plugin in
SDK, ...)

Change-Id: Ie29d10475e8d1af1beea3cc5b45984f2be9236ef
2020-03-23 15:30:18 +01:00
Monty Taylor
54a76faf72 Rationalize examples and functional extra config loading
This whole thing is a mess. examples is doing the right thing and
using get_extra_config, but its docs are wrong. functional is
reading config from a non-standard place but is the place that
the examples docs incorrect say it should be read from.

Update both places to read from the RIGHT location and update
the docs as well. Make examples support setting a key to use
and then set that key in tox so that we can just use the
functional settings for both.

Also, fix the clustering functional test so that it consumes
the IMAGE_NAME discovered in functional.base.

Change-Id: Ia9f150bbdd3825cb3e0d4282af3ca8f23dc1d888
2020-03-11 17:25:50 -05:00
Monty Taylor
3bebbd385b Remove extra python2 test jobs
Once these are not used elsewhere we can remove them here.

Change-Id: I1e3ec3a39a43f08bd932312a6dd4616f8a973b36
2019-12-16 11:00:36 -05:00
Monty Taylor
e9107c190b Stop supporting python2
We're peeling python2 from libraries now, so Stop running python2
versions of test jobs. Leaving removing the test jobs themselves
for the moment as they're still used in other repos.

Change-Id: Ic6d56e3636b757201d83649f075005ed3ae1e1a6
2019-12-16 11:00:22 -05:00
Zuul
fa85badc7b Merge "tox: Keeping going with docs" 2019-12-05 16:56:02 +00:00
Matt Riedemann
800cbf7cfd Update deps for tox venv target
This makes the venv target use all of the requirements files
including docs/requirements so you can create a release note
using:

  $ tox -e venv -- reno new <slug>

Change-Id: Ic6e53d3aa10d48d56af2886e9dee59c52a9a2994
2019-12-03 15:09:29 -05:00
gujin
84137a1af2 tox: Keeping going with docs
Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.

[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045

Change-Id: I1ed5fd8b965f4466a20d61d9acd84e7aa2bbcf57
2019-11-01 15:08:21 +08:00
Zuul
95b13d4126 Merge "Update the constraints url" 2019-10-14 10:09:24 +00:00
Zuul
abff986a02 Merge "Use generated list of services instead of metaclass" 2019-10-05 18:32:07 +00:00
Monty Taylor
c9fba05bf6 Use generated list of services instead of metaclass
The metaclass makes it pretty impossible to do type annotations on
anything. Turn the metaclass into a utility script and then generate
the list of mappings.

Change-Id: Iccd5ee34e364cd6f53c0a8934e20442c022844e8
2019-10-04 11:05:21 +02:00
pengyuesheng
eb589af1d0 Update the constraints url
For more detail, see http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html

Change-Id: I862f001e92526b332fc5610d426fb382f2ae6bae
2019-09-23 16:10:48 +08:00
Ian Y. Choi
363852f80d Build PDF docs
This commit adds a new tox target to build PDF documentation.
It's a community goal[0] to have PDF docs available.

Update conf.py for latex building.

[0]
https://governance.openstack.org/tc/goals/selected/train/pdf-doc-generation.html

Co-authored-by: Bogdan Dobrelya <bdobreli@redhat.com>
Change-Id: I411bdabeaf70164ec1020abc2d22fcf1764f7c02
2019-09-05 17:21:19 +02:00
Artem Goncharov
4c0071916d Enable ansible module test for keypair to check return data
With a change in cloud layer to use proxy/resource layer for keypair
Ansible got borked and is not capable in resolving data we return it
back. Add test for it searching for the cause.
Investigation resulted in the fact, that when invoked by Ansible native
implementation of the Resoure.items was not suddifient under Py3, while
under Py2 it is fine. An attempt to integrate all invoked functions into SDK
from Ansible has not brought any success.

Change-Id: I60dec9ba26176efc5b8ad8378b0ef414754a857c
2019-08-03 20:57:22 +02:00
Corey Bryant
832e436891 Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: Ib55d9c890de62bdfe269390c5533144e46e92703
Story: #2005924
Task: #34233
2019-07-05 14:34:50 -04:00
Monty Taylor
737bcb0eca Finish updating links to point to opendev
The redirects work, but it's nicer to just use opendev links
directly. Also, update shade, which is in openstack namespace
now.

Depends-On: https://review.opendev.org/654230
Depends-On: https://review.opendev.org/654233
Change-Id: Id559b79c2aefda50fa3ec0feedf6d8c52a687a75
2019-04-21 12:31:44 +00:00
Monty Taylor
8748f4a4f8 Move pep8 requirements in to test-requirements
We had these in tox.ini before to streamline pep8 (it didn't need
to actually install openstacksdk) But now that we're doing local
hacking checks, we have to install the code, which means this
optimization is bong. Put things back in test-requirements like
sane people. While doing that, remove readme, which isn't in
openstack/requirements and just let doc8 check it.

Change-Id: Id1860a1df33610ad7072c13ef9ee61395b785b94
2019-03-10 16:32:03 +00:00
Monty Taylor
a49f2056b2 Get rid of setUpClass and block it for forever
setUpClass is evil. It lures you in to a sense of complacency thinking
that reality exists and makes sense. But it's all just a cruel joke
designed to increase your personal karmic debt.

In order that we might collectively one day reach nirvana, remove all
senseless and misleading instances of setUpClass. Then add local hacking
checks to prevent the introduction of such scourge ever again.

Change-Id: Ifd43958bf47981aedad639bef61769ddb37618d3
2019-03-01 16:40:06 +00:00
Sergii Golovatiuk
d0f9e1b314 Add missing py37 and corrected default envlist.
Change-Id: I408c179e66a58f80972727ad270126319d7ded0b
2019-02-25 14:02:36 +01:00
Artem Goncharov
84116d6125 Make timeouts in functional tests configurable
functional tests are often failing due to timeouts in `wait_for`
functions. It forces us to trigger a "recheck". Let's make it
configurable instead. A top level timeout can be set
with `OPENSTACKSDK_FUNC_TEST_TIMEOUT` env and defaults to 300.
Func tests for each service might override this var
(i.e. for heat `OPENSTACKSDK_FUNC_TEST_TIMEOUT_ORCHESTRATE`).
OPENSTACKSDK_FUNC_TEST_TIMEOUT_LOAD_BALANCER is set to 600 (in tox)
to respect the value previously used in the code.

Change-Id: I7bdcd67e858a1f2c43cb131ebdf549da070fbb4a
2018-11-23 10:21:06 +01:00
ca636b92e3
Test python2 with py27
py3 tests were running with py27. Add ignore_basepython_conflict to
tox.ini so that the basepython of python3 doesn't override the py27 env.

Change-Id: I3ff782fccbc7ffd50f703a006f63b321e673a879
2018-11-06 08:12:04 -06:00
Vieri
3755456cbd Update min tox version to 2.0
The commands used by constraints need at least tox 2.0.  Update to
reflect reality, which should help with local running of constraints
targets.

Change-Id: I9b4c39a0d33fabadb1595389281466ba5d2453b2
2018-11-02 06:54:04 +00:00
Zuul
c400b056eb Merge "Fix bugs in debugging with Tox" 2018-11-01 15:28:04 +00:00
Zuul
9425d07adb Merge "Fix latest flake8 issues" 2018-10-31 19:44:30 +00:00
Corey Wright
a084fe78bc Fix bugs in debugging with Tox
Fix PDB-based test debugging using Tox `debug` environment, ie `tox -e
debug <test>`.

* Tell `oslo_test_helper` what directory holds test.
  * Fixes `ImportError: Start directory is not importable:
    './openstacksdk/tests'`
  * See https://docs.openstack.org/oslotest/rocky/user/features.html#update-tox-ini
* Effectively disable test timeouts when debugging by setting timeout
  to 1 year.
  * Fixes `fixtures._fixtures.timeout.TimeoutException` during PDB
    prompt.

Story: 2004225
Task: 27744
Change-Id: I40fea330e6b6dba78927de66bfe643b9c9a3e5e6
2018-10-31 12:43:03 -05:00
Sean McGinnis
49cf6f914a
Remove setup.py check from pep8 job
Using "python setup.py check -r -s" method of checking the package has
been deprecated with the new recommendation to build the sdist and
wheel, then running "twine check" against the output.

Luckily, there is already a job that covers this that only runs when the
README, setup.py, or setup.cfg files change, making running this in the
pep8 job redundant. This covered by the test-release-openstack-python3
that is defined in the publish-to-pypi-python3 template.

More details can be found in this mailing list post:

http://lists.openstack.org/pipermail/openstack-dev/2018-October/136136.html

Change-Id: I9c760887d90869a44af62f4a4c4c09d3599f6047
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2018-10-30 10:38:41 -05:00
Monty Taylor
d42811c405
Fix latest flake8 issues
The latest flake8 checks strings for escape chars, and also has a new
and better rule about line breaks before binary operators - which is
what knuth says should happen.

The escape char check was catching some asterisks in docstrings that
weren't actually providing much, since we already list type as "kwargs".
Just remove them.

Install latest flake8 so we can gate on this.

Change-Id: I89088adcc6f4ff5a894df5e677ae9b28a58edd9a
2018-10-29 16:44:05 -05:00
Ian Wienand
9804985fc7
Add doc depends to tox releasenotes environment
When I run "tox -e releasenotes" it fails because it doesn't install
any of the sphinx or reno tools.  Add doc dependencies.

Change-Id: Ia8b864fbbe02f0963881d3a393b1a5935a3f16e9
2018-10-26 08:40:52 -05:00
Zuul
0458ec390d Merge "Clean up python3 test and split networking into a job" 2018-10-05 15:10:56 +00:00
Monty Taylor
6befbbe76c
Clean up python3 test and split networking into a job
We want to run with python3 always with python2 being the exception.

Update the functional tests to be python3 by default with a
python2 job to check compat.

Remove python3.5 jobs, since that's covered by python3.6 and the
differences between 3.5 and 3.6 are covered by also having 2.7 in the
gate.

Similarly, make the tips jobs python 3.6 only because they are jobs that
aim to ensure interactions between openstacksdk and keystoneauth and
os-client-config and shade are correct at the unit test level. The
individual unit tests should catch actual python 2.7 issues.

Remove cover jobs because we historically haven't cared about the output
and they take up test nodes. In general it's not that big of a deal, but
we tend to be patch-heavy around here, so it feels extravagant to run a
job we don't actually care about the output of.

Split advanced networking services into a job

We're hitting timeouts like crazy- maybe we're just running low on
resources. Split octavia and designate into their own job. Turn
swift off in that job.

Skip sdk layer dns functional tests when designate is not present. We
should refactor these to deal with both cases.

Change-Id: Ica5a47cc200c8abff1d20af9883c5192fcbb95e1
2018-10-04 11:04:32 -05:00
Monty Taylor
b5c96c5e44
Restore timeout_scaling_factor
In a previous refactor, we lost the part where we actually multiple
TIMEOUT_SCALING_FACTOR to the timeout. Whoops.

Also, turn off keepalive in the underlying session. For devstack
functional tests, we're seeing a lot of "Resetting dropped connection"
so I'm thinking we should just disable that.

Also, increase the individual test timeout, and bump the scaling factor
on trunk tests, which always seem to be timing out.

Change-Id: Ie5b3eb319fed10cabfb2e818f3fcfd52b6fd7b14
2018-09-25 14:05:23 -05:00
Dmitry Tantsur
7fd4c33091 Add a simple baremetal functional job
This creates a new job with a limites set of services, including
ironic. Since many regular tests won't work with the baremetal
backend or will take too much, only baremetal tests are run in it.

Change-Id: Ia8af5ebc9aec46d405180f6894dd4228eb816b59
2018-07-30 20:24:31 +02:00
Rui Chen
722d8136b2 Run ansible tests against specific public cloud
We want to run ansible tests against some public cloud, but
there are always some API policy limits for regular users in
public cloud, like: can not create external network. So we define
some variables that can be changed for different limit scenairoes.

1. Create port in external or internal network.
2. Specified public pool name when launching server.
3. Create simple router, then add interface and external gateway,
   change test step order, so that do not create external network
   when network_external=false.
4. Create subnet with enable_dhcp true or false.
5. Make ANSIBLE_VAR_* environment variables can be passed into virtual
   env.

Change-Id: I69473756b23a6cb525e0f9bb40d09e6ed9880782
2018-07-17 14:26:56 +08:00
Doug Hellmann
d3df639353 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I3edccb24a3f714be8fcc32851342363e44f7ea2f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-06 15:27:00 -04:00
Monty Taylor
83d939dd43
Add ansible functional tests
ansible now directly depends on and uses openstacksdk rather than shade.
Make sure we don't land patches that break ansible.

Remove the installation of ansible from run-ansible-tests as it's
specified in the tox.ini file already, and we'll get it installed via
tox-siblings in the gate.

Depends-On: https://review.openstack.org/570956
Change-Id: I35103b8c75100725caf7a8fd264b81d402bf1973
2018-05-29 10:57:42 -05:00
Zuul
8cd40bf01d Merge "Add python 3.6 jobs" 2018-05-11 15:51:35 +00:00
Zuul
75156b0813 Merge "Bump default timeout values" 2018-05-10 19:31:50 +00:00
Monty Taylor
8447e82b29
Bump default timeout values
We're getting a fair number of timeouts sporadically. The VMs seem to
just be running a bit slower... give ourselves a bit more headroom.

Change-Id: I74ffd304d581e7449b8f850b2412ef1dce6effb1
2018-05-10 11:59:23 -05:00
Monty Taylor
055536ef51 Fix H103 Apache license header check
It's good to have a license header check. Amusingly - this actually
caught a real error in one of the headers. I'll leave it as an exercise
for the reader as to which one was a real error.

Change-Id: I46bee047d798fcb384e711462bc7463ca8745ee8
2018-05-09 16:54:41 +00:00
Monty Taylor
ee1d1ad96b Remove D exclusions from flake8 config
It was unknown why they exist, but removing them doesn't break anything.

Change-Id: I9266ff844407ce4c9e52bda7ee570a7a3e707534
2018-05-09 16:54:36 +00:00
Monty Taylor
9007ab7048 Add comment about W503 being skipped
It's on purpose and we do not want to 'fix' occurances. If anything, a
rule checking the opposite of W503 would be preferrable.

Change-Id: I246b23e7e1ead1682167f3bf076ce563697217e6
2018-05-09 16:54:30 +00:00
Monty Taylor
be87ac1d9b
Add python 3.6 jobs
3.6 is the future. Let's test it today.

Change-Id: I1c240f10e42fadbdaed78b07ff26fde52c11f8cd
2018-05-09 11:53:30 -05:00
Monty Taylor
1ac3a3b1f8 Fix F405 errors
F405 is a bit more aggressive about not being able to tell if a
symbol is valid due to * imports. Just go ahead and fix it since *
imports are, in fact, hard to deal with.

As a result of fixing this, there are two actual bugs that were hidden
that the check found, so the fix seems worthwhile.

Change-Id: If76b0943e16e0c7264b75829838c4a715342ab9d
2018-05-09 16:10:28 +00:00
Monty Taylor
d5fd3e8f1b
Clean up floating ip tests
There are timeout issues as well as sometimes something goes wrong in
the shade floating ip tests. Update the tests to try to make them more
stable.

This also fixes a bug in find_available_ip that was causing ips with
port_ids to be returned that was accidentally working in the functional
tests as a race condition.

While working on this, pycodestyle made a release and broke us. Add pep8
changes to allow landing this patch. Fix two of them, ignore two of them.

Change-Id: I128e85048b91f0508798d6c0c2a7e3aacb1c92c1
2018-05-09 11:09:44 -05:00
Saju
fb0d34ba26
pypy is not checked at gate
We also don't use python 3.4.

Change-Id: I9aa2f3770610570166fc7004df1e05da9c5e0d4a
2018-04-27 08:56:58 -05:00
Doug Hellmann
1f70f692e6 add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I7ec32aefc9caac30f4e93704fc33b9a98cb1c622
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-03-25 09:07:32 -04:00