105 Commits

Author SHA1 Message Date
Natal Ngétal
55a31ee2bd [Configuration] Add missing py37 and corrected default envlist.
Change order of default tox envlist in order of likeliness to fail on a
new change and add py37 and py36.

Change-Id: I7b88f0ebce37a319e66b20d5438852976195f06e
2018-12-18 10:31:18 +01:00
Zuul
3a44481f22 Merge "Update min tox version to 2.0" 2018-12-14 09:56:20 +00:00
Nguyen Hai Truong
1266656162 Add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: I37bd6b96781ad8ca47c5cd57f455c540245f27f0
2018-11-07 11:16:18 +00:00
Nguyen Hai Truong
7eae234408 Update min tox version to 2.0
The commands used by constraints need at least tox 2.0.
Update to reflect reality, which should help with local running of
constraints targets.

Change-Id: Ie74cd61e251d46524d83a05c0b48fd462aeb4a57
2018-11-02 00:04:16 -07:00
Doug Hellmann
6d6e1c85a3 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I916dfe05c88d3960e0239a6b9a7e5affc6d79196
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-09-26 18:55:18 -04:00
Eduardo Gonzalez
989306a98e Temporary ignore B303 in bandit tests
Revert this change once the insecure usages are fixed.

Change-Id: I36712b3c187afd5b1014d8f3b030a794621474be
2018-08-17 14:46:39 +02:00
Liping Mao (limao)
339e9b2051 Fix yaml warning in release notes and enable yamllint strict mode
Fix yaml warning in release notes. And this patch wants to
make sure pep8 jenkins job can cover yaml warning check
in the future. So use yamllint with strict mode[1]
to treat warning as error.

[1] http://yamllint.readthedocs.io/en/stable/configuration.html#errors-and-warnings

Closes-Bug: #1778389

Change-Id: I05af5204c4d5388daa8bd3c8582fd1058df15920
2018-06-24 16:46:23 +08:00
confi-surya
0f2c0d54b8 Following the new PTI for document build
For compliance with the Project Testing Interface [1] as described in
[2]

[1]
https://governance.openstack.org/tc/reference/project-testing-interface.html
[2]
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: Ic0b3f6c24b3229bbee19b39f9264a40884c700c9
2018-04-12 13:48:50 +09:00
Doug Hellmann
956dc0da6a add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I5753b175264c074ff722b47293c65063f0187ffe
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-03-22 14:21:55 -04:00
Nguyen Hai
850bd8afaf Follow the new PTI for document build
For compliance with the Project Testing Interface as described in:
[1] https://governance.openstack.org/tc/reference/project-testing-interface.html
[2] http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: I302aced3b50b038900375e9632ca7ed136f32914
2018-03-16 03:32:55 +09:00
Eduardo Gonzalez
01a5388169 Verify YAML syntax in gates
Test yaml syntax is correct on new changes.
Fixes current warnings and errors

Change-Id: I5888f8e4a9d27a08506036df2c564b9f2081ccee
2018-02-27 21:45:57 +00:00
Jenkins
ef949dafd7 Merge "Fix doc build warnings" 2017-07-18 15:07:46 +00:00
Eduardo Gonzalez
5289c6b428 Fix doc build warnings
Fix warnings:
doc/image-building.rst:415:undefined label: deploy_a_registry
(if the link has no caption the label must precede a section header)

Change-Id: I8d921f3b05acf9304ccf41e172926887088b97d7
2017-07-13 16:26:41 +01:00
Eduardo Gonzalez
cc7aabc12e Add bandit check to all python code
Bandit was not running in all directories
due missing -r parameter.

Also, this change fix bandit errors.
* Add nosec to subprocess module
* Change /tmp/releases to .releases
* Change /tmp/ to tmp/, tmp/ is not checked
  and in unittest is not an issue, simpler that mocking tempfile
  and make more complex tests.

Change-Id: I42bafa4707a9a4d1faeb01cb9b5a843f56280deb
2017-07-06 15:54:58 +02:00
blue55
f517a42752 Enable some off-by-default checks
Some of the available checks are disabled by default, like:
[H106] Don’t put vim configuration in source files
[H203] Use assertIs(Not)None to check for None

Change-Id: I61ce619eeb37dc05f81015d37afc78a32838448a
2017-06-21 15:40:51 +08:00
Surya Prakash Singh
eb664f9d6c Fix oslo_debug_helper not working
Change-Id: Ifd4ea791b22b1bddc1116b89e6589a372cd17bd9
Closes-Bug: #1666560
2017-03-31 07:40:10 +00:00
Mohammed Naser
4c7a4099ca Remove global privilege escalation for deploy_aio.sh
At the moment, deploy_aio.sh is called with sudo which results
in the environment being wiped.  This means that all Zuul
environment variables are removed and therefore when zuul-cloner
runs to checkout the branch of kolla-ansible, it will always
fallback to master as ZUUL_BRANCH is unset.

This patch removes the global usage of sudo in the tox.ini file
as there is existing usage of sudo in the bash script itself. This
will mean that we will only escalate privileges when needed rather
for the whole script and the environment variables should be passed
on properly resulting in the matching release of kolla-ansible being
checked out.

Closes-Bug: #1673756

Change-Id: I477a843e696be7136c020be6408afea01967879d
2017-03-18 14:13:53 -04:00
Jenkins
53668615e0 Merge "Remove support for py34" 2017-03-15 14:55:12 +00:00
lingyongxu
3cbdd4a1a0 Remove support for py34
Change-Id: I522dc769803f2951341db39c868a153d6a69944a
2017-03-15 15:52:32 +08:00
Jeffrey Zhang
443694bcfa Add debian related test env
Change-Id: Ic186d8f563e58680e10d73a188ef63fa53e44c10
2017-03-07 22:54:57 +08:00
zhuzeyu
5c82141a01 Delete python bytecode file
Delete python bytecode including pyo

Change-Id: Ia967e03e5b46ecdf25625a8554d26c3182ed7b6d
2017-02-10 15:45:03 +08:00
Surya Prakash Singh
9004321aaa Code coverage improvement for kolla
This patch add the coverage tool package to test-requirements.txt
which was missing, while kolla support coverage. In addtion to this
tox.ini coverage command is modified to reflect the coverage report
in standard output whenever "tox -e cover" command run for better
representation of coverage report.

Change-Id: I7f77c8d5320a7d180586ce1e8773a4f294846331
Partially-Implements: blueprint coverage-increment-for-kolla
2017-01-17 13:22:07 +05:30
Surya Prakash Singh
7700c6ca53 Cover environment for kolla
Enabled the cover environment for kolla in tox
Closes-bug: #1650269

Change-Id: I732058b07d951227e7011f6f1a3d84fed594cfa7
2017-01-11 15:46:36 +05:30
Jenkins
6f17e3ffc1 Merge "Remove Ansible references from tox.ini" 2016-11-20 15:02:57 +00:00
Jenkins
6262658764 Merge "Use upper constraints for all jobs in tox.ini" 2016-11-20 13:35:34 +00:00
Steven Dake
b7ef7471c9 Remove Ansible references from tox.ini
Change-Id: Id27756e2333e216c9b0947c0af1e77c60ad5d9f0
2016-11-18 08:50:36 -05:00
Jeremy Liu
f03e409f5c Add bindep environment to tox
This is helpful for listing any missing system requirements [1].

[1] http://docs.openstack.org/infra/manual/drivers.html#package-requirements

Change-Id: I7bfa910a09f99354f4a2fb0ff67854a9c89c44fd
2016-11-12 22:32:53 +08:00
Jeffrey Zhang
2392d7e38c Add file validator for *.json.j2 file
All json file in Kolla are json.j2 file, it shold be render as jinja
template, then validate by using json.

Change-Id: Ibabbe435116fd255d68347e00407156db76fedfe
Closes-Bug: #1634447
2016-11-05 01:16:12 +08:00
Andreas Jaeger
e7d4a1ecf6 Move bandit to pep8 as generic linting target
bandit is a security linter and can be made voting now. Instead of
starting another virtual machine, run it as part of the generic linting
target which is pep8.

Change-Id: I6a8f7aa2b60bd96d55b37b6cc2a371069d6d5e6a
2016-10-19 20:40:33 +02:00
Jeffrey Zhang
3e89f987c0 Add validate no new line at end of file
TrivialFix

Change-Id: I817aa52caf56b7d54b266e553db6db1ceb38b773
Signed-off-by: Jeffrey Zhang <zhang.lei.fly@gmail.com>
2016-10-19 17:17:06 +00:00
pawnesh.kumar
bcd3868275 Use upper constraints for all jobs in tox.ini
Openstack infra now supports upper constraints for all jobs.
Updated tox.ini to use upper constraints for all jobs.

Change-Id: I603949f16dc415b0900fafcfbd4cf76ff9e8d746
Closes-Bug: #1628597
2016-10-13 11:35:48 +00:00
Jenkins
76ff764522 Merge "Add support to use pdb with tox" 2016-09-22 07:08:58 +00:00
MD NADEEM
3bf571da06 Add support to use pdb with tox
TrivialFix

Change-Id: I6f32ce7033fe5a8b8f628a8f52dde67b9540083d
2016-09-21 03:25:42 +00:00
MD NADEEM
92c828b99c Remove all bytcode & objectcode before running tox
Currently __pycache__ directory, which also holds
python bytecode is not deleted when running tox.
TrivialFix

Change-Id: I0fdcf9a3f9b01cee813eefdbfe29d208dd67f826
2016-09-20 12:09:30 +05:30
Paul Bourke
fc30d583f9 Fix bandit gate jobs
* Inspected each error and fixed / added nosec where appropriate.
* build-swift-ring.py which was throwing sec errors is no longer used so
  removed it.
* Removed the dev/ directory from being checked.

Closes-Bug: #1617713
Change-Id: I25664cabca4137e5c9f499c1af3f5ce78b86fb56
2016-08-28 08:52:44 +00:00
Christian Berendt
c3d8262ec5 Add doc8 test and improve rst syntax
This will test all rst files inside the doc directory for style issues with
doc8 (an opinionated style checker for rst styles of documentation).

This will fix all syntax issues identified by doc8 and will improve
the syntax.

Change-Id: Id1b9563e07e77e306aef5a0767c98c27f87c5c0e
2016-08-04 15:09:10 +02:00
Yatin Kumbhare
bcebc7cb47 Add Python 3.5 classifier and venv
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.

Change-Id: Ia7b1fb88ac108e1e8fd6ed963967e731ae3ca5a4
Closes-Bug: #1603006
2016-07-14 15:59:37 +05:30
Swapnil Kulkarni (coolsvap)
eb1bfc2c2e Add support to lint the install commands in Dockerfiles
Change-Id: Icb47ba8b13b13889e28bb285321f11344400edc2
Closes-Bug:#1582618
2016-05-26 17:41:42 +05:30
Jeffrey Zhang
7ab705cd4d Add Ubuntu binary build and deploy gate
The Ubuntu Mitaka repo is release, and we have add it.
So it is time to add the Ubuntu binary gate.

Closes-Bug: #1560386
Change-Id: I29d01147595c4ebebfa00798d01d5fd7398123a6
2016-04-02 16:02:58 +00:00
Steven Dake
a789346727 Fix gate to use world writeable docker socket
A recent change related to sudo securepath broke how sudo -E
works.  Now the PATH is reset by sudo as set by tox.  As a result
we can no longer rely on sudo -E anywhere in our gating system
relating to path inheritence from the parent shell.

This patch uses a shell operation in the setup_nodes.yml code
to chmod the docker socket to 666 so docker containers can build
properly.  Now docker operations don't return a permission denied
and we no longer require the sudo -E operation in any of our tox
scripts.  This isn't a security vulnerability because our gate
scripts are only meant to be run in OpenStack infrastructure.
To make this more clear I recommend moving these shell scripts to
tests/gate.

Change-Id: I5b3d6e280e3c12a25defedb14b5589ba642043fa
Closes-Bug: #1562383
2016-03-26 22:23:35 -07:00
Steven Dake
801ef2e843 Add reno support
For proper release note generation on the releases.openstack.org
website, this change will need to be backported into mitaka and
liberty as well.  Then a special one-file reno file will have to
be added for mitaka describing the features and the same story
for liberty.

See log of the discussion here:
http://eavesdrop.openstack.org/irclogs/%23openstack-release/%23openstack-release.2016-03-23.log.html#t2016-03-23T18:00:05

Change-Id: I3c298ae08ec4645f384ea5829cb0851fe82f4557
Partially-Implements: blueprint add-reno
2016-03-23 11:59:35 -07:00
Dave McCowan
39b8150f66 Add tox job for the bandit python security linter
Bandit is a job that parses and analyzes python code to detect
common insecure programming practices.

TrivialFix

Change-Id: Iea23a9bd23b25827043f02aa1a62fb0724f1d0ee
2016-03-08 16:33:59 -05:00
Jenkins
f3048ff8ab Merge "Introduce the hacking module to kolla" 2016-03-03 21:48:44 +00:00
Jeffrey Zhang
4e154f0018 Introduce the hacking module to kolla
Implements: blueprint add-hacking

Change-Id: Ia24a263555b9c1eb11e61b9d3fe89612a89daed6
2016-03-03 04:40:16 +00:00
Paul Bourke
80f0890c81 Add new gates for oraclelinux
Change-Id: Ic5c0e266639c8ed76d0af473d076d9ae6749102a
Implements: bp oraclelinux-gate
2016-03-01 15:34:34 +00:00
Jeffrey Zhang
d95fd11b2d Remove the tox linters env
TrivialFix

Change-Id: Icd69a4d6de8ef222b219e758fa38d58f2c98f505
2016-02-26 19:26:26 +08:00
Andreas Jaeger
8ad0b11065 Make pep8 *the* linting interface
According to the PTI (=Python Test Interface,
http://governance.openstack.org/reference/cti/python_cti.html), pep8
is the interface for codestyle checks. Move all tests from linters to
pep8.

This change will be followed by a change to project-config to use pep8
for testing in the gate.

Change-Id: I1a48f4b1e0b0950640192bcfc55121619a844b50
2016-02-11 09:55:28 +00:00
Jenkins
06b16d577d Merge "Convert gate to file backend rather than partition" 2016-01-26 03:04:43 +00:00
SamYaple
d7d5eecdbd Remove our unused jobs that now run in linters
Change-Id: I5c83b28ce3ef6c4080c68fa2fa353ad118ef7b71
Depends-On: I48faeb61fd3d630e7d7b6c949afcf10f724b0b72
2016-01-25 22:21:49 +00:00
SamYaple
6adb5c0aa5 Convert gate to file backend rather than partition
The existing gate partitioned a disk for use with docker, depending
on the gate it would use the swap disk (RAX) or a spare disk (HP).
However, with the new gates (Bluebox + OVH) there is neither a spare
disk nor a swap disk. This leaves us with one choice: File based loop
device.

This patch creates a file at /swapfile to ensure we have swap. It
creates a file at /docker to ensure we have a loop device for Docker.

Right now the /docker file is 10GB and the /swapfile is 4GB due to
size limitations in the gate across all servers and types. This has
proven to be enough space for all our current tests.

Additionally, reduce the number of threads the gate uses to 4 to
prevent the lockup and hour timeout we have been seeing as more
recently in the gate.

The scripts that setup the gate are moved to the tools directory
rather than the tests directory to match the structure of the other
projects.

Partially-Implements: blueprint functional-testing-gate

Change-Id: I3e370f2382b6df36103d8b2ceda9b21d9b4229d5
2016-01-21 22:59:13 +00:00