74 Commits

Author SHA1 Message Date
Yatin Kumbhare
0a56f557d6 Add Python 3.5 classifier and venv
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.

Change-Id: Ifd69012586ecb59faaab2a474f43efde9b244756
2016-07-12 17:20:05 +05:30
Spyros Trigazis
936dd11a16 [install] Add install guide from template for rdo
This guide is based on the installguide-cookiecutter template
and will be published automatically to docs.openstack.org.

[2] is a similar patch and [3] is WIP.

A change to add magnum to the install-guide job will follow.

This is a follow-up of [4].

[1] http://docs.openstack.org/contributor-guide/project-install-guide.html
[2] https://review.openstack.org/#/c/325389
[3] https://review.openstack.org/#/c/317152
[4] https://review.openstack.org/#/c/332161

Change-Id: I145fd30f575dab45b4c947bc4609287b1e300025
Partially-Implements: blueprint magnum-installation-guide
2016-06-22 16:20:05 +02:00
Hongbin Lu
c52e5bea06 Add tox test for k8s coreos bay
* Add a CoreOS test class TestCoreosKubernetesAPIs
* Add a CoreOS test environment in tox.ini
* Create a base class BaseK8sTest and move OS-agnostic k8s testing
  code to that class.
* Increase the disk size from 8G to 10G for m1.magnum and s1.magnum,
  since CoreOS image requires more disk space to boot.
* Set os-distro property for CoreOS image.

Partial-Bug: #1546101
Change-Id: Ie56a9442ecebe05f39c7669bc950f5a6ca11df33
2016-05-09 02:36:26 +00:00
Andreas Jaeger
5364356b99 Remove constraints envs from tox.ini
The constraints environments are not used at all, they have been
obsoleted everywhere in OpenStack. Remove them completely.

Now the normal jobs can use constraints - as magnum already does.

Change-Id: Iecec444fba59923c7894c1fd2a90aebbfdc4724d
2016-04-17 17:09:03 +02:00
Andreas Jaeger
e3fef8412f Fix post jobs
The tox venv environment is run during post jobs and thus cannot use
constraints.

See
http://logs.openstack.org/a6/a6650650f065bc4423568f9318d362957eba7f31/post/magnum-docs/60418d4/
http://logs.openstack.org/a6/a6650650f065bc4423568f9318d362957eba7f31/post/magnum-branch-tarball/415035c/

We run for docs jobs:
"tox -e venv python setup.py build_sphinx"

thus, the docs environment is not used.

For branch tarball, the infra scripts use:
"tox -e venv python setup.py sdist"

And infra does not setup constraints for post jobs currently as this is
not working with current tools

Fix tox.ini for this.

Change-Id: Iad81b47899c4554b75f90dc24f0aa2aff7aef730
2016-04-17 17:07:58 +02:00
Davanum Srinivas
a6650650f0 Use k8sclient library
In Ia400b5b6919753d6241233a6666a2c30c7618bc9, we are
adding a new library to global requirements. This
python-k8sclient library essentially has the same
code that was in Magnum so it's a straight replace.

Change-Id: Ib85bcf532e4163d5c1ac95ef701a1ec3133b69eb
2016-04-17 03:35:39 +00:00
Jenkins
b225ccdbc5 Merge "Add reno to Magnum" 2016-04-05 16:46:36 +00:00
Eli Qiao
c1b63cdac0 Add reno to Magnum
Introduce reno to handle release notes as files in-tree.

Also added documentation on how to add re for changes, after this we
will require changes to add reno if they have some significant changes.

Implements: blueprint add-reno-for-release-management
Change-Id: Ifc18b78ab5ce89375b85f6ac4344ffafa7f1c48b
2016-04-05 08:45:43 +08:00
Niall Bunting
dbcfe35111 Magnum's tox test should respect upper-constraints
This will force pip install to use the upper-constraints.txt specified
version of pip modules. When you don't do this, you are out on the
bleeding edge and become unstable everytime some python library in the
wide world changes in a way that you don't expect.

Change-Id: I962329075b11d3790dc586ba33fe19fc64f1e4f9
Closes-Bug: #1563038
2016-04-04 09:18:44 +00:00
Eric Brown
6253f70a43 Remove bandit.yaml in favor of defaults
* Remove bandit.yaml so bandit just uses internal defaults. Results
  in much less maintenance.
* Raise the severity level of the scan to just medium and high
* Added nosec to line that makes use of file:// scheme. That support
  should probably be droppped at a later time.
* Removed a misleading log message noting what address it was
  serving on. Bandit also flagged this as a potential security issue
  serving on 0.0.0.0.

Change-Id: I3e08e462255f7b4ba8405d69f5843ed3c001d055
2016-03-09 11:20:28 -08:00
Yang Hongyang
fd8ee8d205 Add py34 to tox envlist
We are going to support python3.4, it's better to add py34 to the
envlist so that devs can fix python3 compatible issues.

Change-Id: I8b478f78c5fd2fb11e6fdcb02bffb10929113a18
2016-03-01 02:16:03 +08:00
Andreas Jaeger
d077045d78 Make bandit job voting
Merge bandit job into the voting pep8 target. Pep8 is used here as
general target for lint-like jobs. This will make the bandit job voting.

Looking at recent runs, bandit passes fine, so it's time to enable it.

Once this change is in, the non-voting bandit change can be removed from
OpenStack CI to reduce node usage and the entry in tox.ini can also be
removed.

Change-Id: I9bf1a741336abf830a092f0fb372116d08e6127c
2016-02-09 17:50:17 +01:00
Eli Qiao
022bba9c06 Use magnum-config-generator.conf to generate Magnum config sample file
We hard code configure options in tox.ini to generate Magnum config
sample file, and we keep this config sample file in our repository
before.

It is inconvenient for developer if the add/remove/change some
configuration options cause they need to take care about the
config.sample file.

This patch removes magnum.config.sample instead of letting admin generate it
from magnum-config-generator.conf(picked from tox.ini).

P.S. Make tox.ini cleanup since it is not a good idea we mess up tox.ini
with configuration options.

Closes-Bug: #1538374
Change-Id: I80d019b5c98e0282cd0fe50fd4ad66286287e2cb
2016-01-27 11:28:26 +08:00
Corey O'Brien
d52965d2d0 Improve Tempest credential provider usage
Instead of getting a DynamicCredentialProvider directly, use the
get_credentials_provider function. This allows for configuration via
TEMPEST_ variables, but the default will still be dynamic credentials.
Only create the default keypair for tests if it doesn't already exist.
Ensure that credentials are freed by calling clear_creds in all cases.

Change-Id: I62ad5ed9fe7c9f6675989df459f92194bb8d67a8
Implements: blueprint magnum-tempest
2016-01-22 15:13:42 -05:00
Jenkins
b4abeeafa7 Merge "Add debug testenv in tox" 2016-01-18 08:26:47 +00:00
Jenkins
44a5a41d6f Merge "Revert "Pass environment variables of proxy to tox"" 2016-01-17 06:05:27 +00:00
ting.wang
97e3a6b897 Add debug testenv in tox
Once we add debug testenv, we can use "tox -e debug"
to debug test cases when tox is running. Then tox will
use oslotest which really debug our test cases.

Only we should do is insert pdb into code.It's easy to use.
ref:
http://docs.openstack.org/developer/oslotest/debugging.html

Closes-Bug: #1534030
Change-Id: If10634fd0a8c9b9093ccd64f5ba271cf09cfc37c
2016-01-17 04:22:00 +00:00
Doug Hellmann
3281cd912a Revert "Pass environment variables of proxy to tox"
It looks like you can work around this problem by setting an 
environment variable on your dev system. See 
https://tox.readthedocs.org/en/latest/config.html#confval-passenv=SPACE-SEPARATED-GLOBNAMES

This reverts commit eab8c8aa63bbd08b3e246bb9f41bf12f6eafc4c6.

Change-Id: I2f6650be6c44526b194d0980ec023f3674add8a2
2016-01-15 19:37:45 +00:00
houming-wang
ab14191218 Fix ignored E711 rule and remove this exception
Fix the ignored E711 hacking rule and remove this exclusion in tox.ini.
This will make Magnum code have no pep8 violations and exceptions.

Change-Id: If7bf9f0cd61cebf2c85011156eb187453578faf7
2016-01-14 16:42:08 +08:00
Eli Qiao
69a4f5345e Functional: Add mesos functional bay creation basic testing frame work
This patch adds functional testing case for mesos bay creation testing.

1. Add gate_hook.sh to set MAGNUM_GUEST_IMAGE_URL to use ubuntu image
2. Modify post_test_hook to use ubuntu image when doing functional
testing
3. Add new tox env entry to do mesos functional testing.

Partially implements: blueprint mesos-functional-testing
Change-Id: I8ee53de4d34aa79d037901e599fc6409a7de2e72
2016-01-02 10:18:28 +08:00
houming-wang
2ed5683a1d Remove oslo-incubator code from Magnum
Since oslo-incubator is no longer maintained[1] after
90ae25e38915cc502d9e9c52d59e8fb668a72ae1. And cliutils.py,_i18n.py are
not referenced by Magnum. Let's remove it.
[1] http://lists.openstack.org/pipermail/openstack-dev/2015-November/
079343.html

Change-Id: Iac3758660209a3983fabe55a279ce72d80286f5e
Closes-Bug: #1529030
2015-12-24 09:41:49 -05:00
Jenkins
aa82107e7b Merge "Improve tox to show coverage results" 2015-12-17 22:43:46 +00:00
Manjeet Singh Bhatia
487b8b6953 Improve tox to show coverage results
This patch will improve tox -e cover when it will be run a summary of
coverage report will be shown afterwards.

Change-Id: I5d10a4ec756dcfea74dfb2bc3d43893caf3f6b26
2015-12-15 21:47:29 +00:00
Manjeet Singh Bhatia
0d5234051d use wild card for passing env variables
With this patch it will no longer be required to write each
single proxy variable. It will let pass more variable just
in a single line in future if we add more env variables.

passenv = *_proxy will get all variables like http_proxy,
https_proxy, no_proxy and any_proxy etc.

Change-Id: I8fd137d115380bfeb558706edb52d1ac2d95ca79
2015-12-09 15:51:31 +00:00
Tom Cammann
eb3d5e9a3a Add -constraints sections for CI jobs
Tox does not currently support using factors with sections, and work
towards enabling this has stalled, as such we will need to duplicate
sections adding -constraints to facilitate running sections with the
constrained install_command.

Author: Tom Cammann <tom.cammann@hp.com>
Co-Authored-By: Egor Guz <guz_egor@yahoo.com>
Implements Blueprint: Requirements-Management
Change-Id: Ia36b1794077f29eac189e19552c2d3a4bfe26e5e
2015-11-12 09:03:22 -08:00
Kai Qiang Wu
7b4b49afe0 Make bandit included in test-requirements.txt
We remove old bandit special version txt file and include it
in test-requirements.txt to make sure it can updated by global
automatically.

Change-Id: I03c15eb6d4c1e7a492c37954fce488cbc1fb63d0
2015-11-16 22:05:20 -06:00
Eli Qiao
9f95010246 Add functional test cases for swarm baymodel/bay
This patch adds TestSwarmAPIs test class, aimed to test Docker API/and
swarm related api function, we create a bay while setupClass, and destroy
it after test cases finished. Even without any cased added, at least we can
ensure swarm bay can be created successfully.

This will help to ensure any HOT/yml change won't break swarm's cluster
basic functions.

I leave a todo, we need more works to test container actions.

Since we have created a bay, so we can add more case using magnum client to
test container actions, will do this by followed patch.

Depends-On: Ifa028f6f30b7c2bb7b1dd32b22669379e162c7c5
Partially implements: blueprint swarm-functional-testing
Change-Id: I5c3bc5d7735cce17a45887359d1fb44d12dbba48
2015-11-13 19:29:18 +08:00
Eli Qiao
eb5eaa503d split out k8s and api functional testing
This patch splits out api and functional testing case to different directory,
and modify post_test_hook.sh to passing a parameter to calling different tox
venv.


Depends-On: I10e09b2deea7a0643a9516701994cc47813e1aaa
Partially implements: blueprint split-gate-functional-dsvm-magnum
Change-Id: Ic33ea42f6108c7444c152706458cf1a686ed1756
2015-11-09 10:50:14 +00:00
Jenkins
67c6e4dae7 Merge "Added CORS support to Magnum" 2015-11-04 15:16:23 +00:00
Michael Krotscheck
cc713cd2d3 Added CORS support to Magnum
This adds the CORS support middleware to Magnum, allowing a deployer
to optionally configure rules under which a javascript client may
break the single-origin policy and access the API directly.

For magnum, the CORS middleware was directly added to the setup_app
method, as the last middleware to be wrapped around the application
instance. Common headers were added, in order to avoid magic
configuration moments where a feature does not function without
an operator having to read code.

OpenStack CrossProject Spec:
   http://specs.openstack.org/openstack/openstack-specs/specs/cors-support.html
Oslo_Middleware Docs:
   http://docs.openstack.org/developer/oslo.middleware/cors.html
OpenStack Cloud Admin Guide:
   http://docs.openstack.org/admin-guide-cloud/cross_project_cors.html
DocImpact

Change-Id: I7e6d7e2145e3bf883b8a28071c949e10e5c207e0
Closes-bug: #1508209
2015-11-03 10:20:36 -08:00
Eli Qiao
d802877e6e Improve tox.ini to easy developer's life
This patch is mainly porting from nova repository:

Adds:
1.tools/flake8wrap.sh
  Will let developer use 'tox -epep8 -- -HEAD' to testing changes files
2.tools/pretty_tox.sh
  Give pretty test trace when doing unit/functional testing.

Change-Id: Icedb4ed8b50532531e18784a1584fe63c5a9e017
2015-11-03 17:44:38 +08:00
Tom Cammann
ee52fbbcba Refactor MagnumException to reduce complexity
Reduce cyclomatic complexity of MagnumException to below 10. Remove
exception handling and optimise the logging when the kwargs does not
match the message.

Change-Id: I4fd6ba96be9dff485cdd8bde9750f2d584c1ec63
Closes-Bug: #1501331
2015-10-22 09:52:20 +01:00
Tom Cammann
82003a56e5 Refactor config setup to reduce complexity
The cyclomatic complexity of the setup method was greater than 10 and
reducing this complexity allows for easier understanding of the method.

Change-Id: I45d085d0943fd18a4207fc7a9ba64f2c0f8074ea
Partial-Bug: #1501331
2015-10-22 09:52:20 +01:00
Tom Cammann
64cc141379 Reduce complexity of poll_and_check method
This is method is overly complex and should be broken down. When
reducing the max complexity allowed by flake8 this method was flagged.
Reducing the complexity of this method will allow better testing and
maintenance.

Partial-Bug: #1501331
Change-Id: Id0bdb2d03614ba31de0d96ed09c33695d933b9f7
2015-10-22 09:30:56 +01:00
Tom Cammann
e5dd25cf22 Add doc8 to pep8 job
To allow doc8 testing of the Magnum documentation in CI, doc8 should be
added to the pep8 tox command. After discussion with infra (see change
I6d42cb2092cbaed0cc2ed83f23b7f9ce0b52356f) this has been the suggested
method to allow for doc8 style checking.

As this behaviour is non-obvious I have left the doc8 command in the doc
tox env, this is not run by CI but will be run by users when modifying
and compiling documentation.

Change-Id: I93133e92d792ccc13d61f3ad1281007c2639853b
2015-10-21 17:07:35 +01:00
Jenkins
01f43b47d4 Merge "Reduce complexity of filter methods" 2015-10-02 04:54:09 +00:00
Tom Cammann
f3b6ada57b Reduce complexity of filter methods
Two filtering methods had very high cyclomatic complexity, this change
reduces their complexity.

Partial-Bug: #1501331
Change-Id: Idf5dcf83225c35e91cd0fe71eaa7abe32f7f2c6d
2015-09-30 16:18:54 +01:00
Tom Cammann
f714fd4902 Fix E251 unnecessarily ignored pep8 rule
Fix E251:
Unexpected spaces around keyword / parameter equals

Change-Id: I622f17d2be58af3e5bbfde6054bb7167c09abc9c
Closes-Bug: #1498870
2015-09-29 17:47:34 +01:00
Jenkins
6b2257fda1 Merge "Use --max-complexity flake8 rule" 2015-09-24 18:30:03 +00:00
Jenkins
606a9b0830 Merge "Fix H405 and E131 ignored pep8 rules" 2015-09-24 18:28:56 +00:00
Tom Cammann
d907843ae4 Use --max-complexity flake8 rule
This feature is quite useful to detect over-complex code. According to
McCabe, anything that goes beyond 10 is too complex. See
https://en.wikipedia.org/wiki/Cyclomatic_complexity.

Currently the max complexity is 13:

    ./magnum/service/periodic.py:51:1: C901 'MagnumPeriodicTasks.sync_bay_status' is too complex (11)
    ./magnum/conductor/handlers/bay_conductor.py:220:1: C901 'HeatPoller.poll_and_check' is too complex (12)
    ./magnum/common/magnum_keystoneclient.py:97:1: C901 'KeystoneClientV3._v3_client_init' is too complex (10)
    ./magnum/common/exception.py:192:1: C901 'MagnumException.__init__' is too complex (10)
    ./magnum/common/utils.py:335:1: C901 'temporary_mutation' is too complex (12)
    ./magnum/db/sqlalchemy/api.py:125:1: C901 'Connection._add_bays_filters' is too complex (13)
    ./magnum/db/sqlalchemy/api.py:295:1: C901 'Connection._add_baymodels_filters' is too complex (12)

Setting the max complexity to 14 allows us to not produce anything more
complex. We can work on reducing our complex functions and reducing this
number to 10.

Change-Id: Iaf9c87ecb4873e646d952fd892aafeb7c91fb5e5
Closes-Bug: #1498872
2015-09-23 12:21:09 +01:00
Tom Cammann
bdf880d8fa Fix H405 and E131 ignored pep8 rules
Updated tox.ini and fixed rules.

Fix H405:
Multi line docstring summary not separated with an empty line
Fix E131:
Continuation line unaligned for hanging indent

Change-Id: I20cf75c75cffc434fbdcb05b8e04bffcd4059cd1
Closes-Bug: #1498870
2015-09-23 12:21:09 +01:00
Tom Cammann
6bb1768847 doc8 all documentation and fix doc style
Add doc8 to `tox -edocs` command. doc8 will check syntax/style check
rst documents and fail if not compliant.

Change-Id: Id2e9ed1f96cac27dc5e38f828fd2d824dad31c49
2015-09-16 16:58:11 +01:00
Eli Qiao
d6efb3221e tox: rm all pyc before doing unit test
Delete python bytecode before every test run.

Because python creates pyc files during tox runs, certain
changes in the tree, like deletes of files, or switching
branches, can create spurious errors.

Closes-Bug: #1368661
Change-Id: I7e563875848acc7ec767f97b31fc8a95b23ee880
2015-09-06 10:38:26 +00:00
Zhenguo Niu
4fdd09ab52 Remove H302, H803, H904
These rules were removed from hacking.

H803: https://review.openstack.org/#/c/101498/
H904: https://review.openstack.org/#/c/101701/
H302: https://review.openstack.org/#/c/142586/

Change-Id: I9f7ec4ed8093972335bd5bbfa1686878ca1d833e
2015-07-16 09:14:21 +00:00
ZhiQiang Fan
32374e68c7 port to oslo.service
oslo.service has graduated, let's port to it.

Change-Id: I222e02e6fc6731c128a7857850000ca46b29a2c0
Closes-Bug: #1466851
2015-06-28 14:29:00 +08:00
Jenkins
7d2125a1be Merge "Add oslo.policy namespace" 2015-06-17 13:47:37 +00:00
Kai Qiang Wu(Kennan)
7eb5f9786f Add oslo.policy namespace
As we have used oslo.policy, so we need to add that in
namespace, as oslo.policy has many settings. This followed
other projects genconfig ways.

Partially-implements: blueprint policy-enforce
Change-Id: I83cce25427b65aad6d26f4ea2d6a48170233a4f3
2015-06-17 03:03:14 +00:00
Jenkins
d774f6772e Merge "Add hacking rule framework for magnum" 2015-06-17 02:07:07 +00:00
Eli Qiao
de7da996b5 Add hacking rule framework for magnum
This patch adds hacking rule check framework for magnum, and adds first rule:
policy.enforce_wsgi decorator must be the first decorator on a method.

refer this link for why we need this rule.
`https://review.openstack.org/#/c/190140/`

Closes-Bugs: #1465895
Change-Id: If98e47426b391b75755ca0b559aee1baa93b8503
2015-06-17 09:00:06 +08:00