48 Commits

Author SHA1 Message Date
ramishra
a263da138f Use merge strategy for EndpointMap
Now someone can override individual endpoints like
below in an environment:

parameter_merge_strategies:
  EndpointMap: merge

parameter_defaults:
  EndpointMap:
    AodhPublic:
      protocol: http
      port: 9999
      host: 121.21.2.1

Closes-Bug: #1897592
Change-Id: Ibf2e0d183dd51421c4feb7467c3c01fb416d2965
2021-06-15 09:15:41 +05:30
Sorin Sbarnea
a013f42e36 Enable ansible-lint
- enable run of ansible-lint, with a temporary set of excludes
- fixes two problems reported by ansible-linter

Change-Id: Ibbe23db8fd5ac1008109f50f514df96686b0fa19
Bug: #1921409
2021-03-30 09:18:15 +01:00
Sorin Sbarnea
2108a5191b Fixed tox executions
- fixed bindep to allow execution under fedora
- fixed flake8 config to use officially recommended use patterns
- assured that git does not report dirty after running tox (both
  pep8 and tht environments where breaking this by creating untracked
  files). tox-extra plugin addition will prevent future regressions.

Change-Id: I4137056853f8c5f1c2304bfb6ec1188a82b12130
Bug: #1921409
2021-03-26 15:37:07 +00:00
Sorin Sbarnea
727d1656df Add support for py39
- Use version agnostic "py" inside tox default envs
- Enable py39 testing jobs (new template contains them)
- Address deprecation warning in setup.cfg regarding use of dashes
  instead of underlines (avoid future breakage with setuptools)

Change-Id: I26be50e029393e01db6718a21ac67d669ccd2e0c
2021-03-24 09:40:57 +00:00
Hervé Beraud
fea7fa2d9c Add doc/requirements
We need to specify doc requirements in doc/requirements.txt
to avoid problems with the pip resolver [1] for the release team [2][3].
Removed specific doc requirements from test-requirements.txt.

The problem here is that this repos haven't doc/requirements.txt file
and by default in this case zuul will use the test-requirements.txt file
to
pull requirements [4].

This requirements file contains extra requirements like flake8 that
collided with those allowed in our job environment and so the new pip
resolver fails to install these requirements and the job exits in error.

This project meet the conditions leading to the bug.

/!\/!\/!\
Notice that I voluntarily added the doc directory even if no docs
are generated here because zuul will try to pull this requirements from
there first and the contained requirements are needed for reno but AFAIK
the releasenotes dir is ignored by zuul. c.f [4] for further details.
/!\/!\/!\

[1] http://lists.openstack.org/pipermail/release-job-failures/2021-January/001500.html
[2] http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019611.html
[3] http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019612.html
[4] https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-sphinx/tasks/main.yaml#L36

Change-Id: I519ef389448acb3dde328a0e61c074e48d7e0466
2021-01-05 09:49:46 +01:00
Kevin Carter
efd1975513 The lower constraint file has been removed
This file, and its tests, are not useful for tripleo so we're
removing because they're now becoming problematic.

Change-Id: I4207e7f543f33dda640bf0784e54311c66ba4e30
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2020-12-08 10:56:59 -06:00
Hervé Beraud
97ddb9496c Replace deprecated UPPER_CONSTRAINTS_FILE variable
UPPER_CONSTRAINTS_FILE is deprecated and TOX_CONSTRAINTS_FILE is
the new environment variable name that replaces it [1].

This allows to use upper-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.

[1] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file
[2] https://review.opendev.org/#/c/722814/

Change-Id: Iffb3f2051a45c77e4d58b54f07b436486ce9e153
2020-11-04 11:11:22 +01:00
Michele Baldessari
87b365afd3 Fix Flakes and lower-constraints errors
With the switch to Ubuntu Focal for tox jobs via https://review.opendev.org/#/c/738322/
our 1.1.0 version of hacking pulls in old modules that are not compatible
with python3.8:
https://github.com/openstack/hacking/blob/1.1.0/requirements.txt#L6

Let's upgrade hacking to >= 3.0.1 and < 3.1.0 so that it supports python3.8
correctly. The newer hacking also triggered new errors which are
fixed in this review as well:
./tools/render-ansible-tasks.py:113:25: F841 local variable 'e' is assigned to but never used
./tools/yaml-validate.py:541:19: F999 '...'.format(...) has unused arguments at position(s): 2
./tools/render-ansible-tasks.py:126:1: E305 expected 2 blank lines after class or function definition, found 1
./tools/yaml-validate.py:33:1: E305 expected 2 blank lines after class or function definition, found 1
./container_config_scripts/tests/test_nova_statedir_ownership.py:35:1: E305 expected 2 blank lines after class or function definition, found 0

Also make sure we exclude .tox and __pycache__ from flake8 as well

We also need to change the lower-constraint requirements to make them
py3.8 compatible. See https://bugs.launchpad.net/nova/+bug/1886298
cffi==1.14.0
greenlet==0.4.15
MarkupSafe==1.1.0
paramiko==2.7.1

Suggested-By: Yatin Karel <ykarel@redhat.com>

Change-Id: Ic280ce9a51f26d165d4e93ba0dc0c47cdf8d7961
Closes-Bug: #1895093
2020-09-10 11:10:54 +02:00
Wes Hayutin
7bc4670f15 remove py27 in tox.ini from tripleo-heat-templates
Remove python 2 from envlist parameter. Check the link:

https://etherpad.openstack.org/p/drop-python2-support
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

The plan is drop the python 2 support from OpenStack in Ussuri release.
Move the basepython parameter.

Change-Id: I794fe6677328d74bef305dcfc15ab81038ae2b87
2020-04-01 21:31:17 -06:00
Wes Hayutin
3370d7b1d3 remove py27 tox env for tht
py27 tox jobs are not running.
clean up tox.ini

Change-Id: Id14f2a2afc55a259737d0279038abff76fc848af
2020-03-24 20:04:36 -06:00
Wes Hayutin
da19fdbccd Add py27 testenv for tox.ini
Since upstream requirements is removing support of py27 but
TripleO jobs still runs on py27 and which is going to break
the deps installation.

In order to fix, we can create a seperate testenv py27
to handle that and it should not depends on upstream
constriants.

Related-Bug: #1861803
Change-Id: I61365503cc1c70644d6520f54646492359fa4e44
2020-02-04 20:38:32 +05:30
Daniel Bengtsson
a465243be1 Stop configuring install_command in tox.
Currently, we are overriding 'install_command' to use 'pip'. This is
considered poor behavior and 'python -m pip' should be used instead:

https://snarky.ca/why-you-should-use-python-m-pip/

It turns out that this is the the default value provided by tox:

https://tox.readthedocs.io/en/latest/config.html#conf-install_command

So we can remove the line and simply use the default value.

Change-Id: Ideb8c351a88f55592d58e6c42709db76de50c82b
2019-11-14 13:04:50 +01:00
Daniel Bengtsson
35ff2c37c6 Use python -m pip instead pip command.
It's recommand to use this way. With tox is not really necessary to do
it but it can be good to use it. See this article:

https://snarky.ca/why-you-should-use-python-m-pip/

For example python -m pip executes pip using the Python interpreter
you specified as python.

Change-Id: Icf90d76eb51c4a52c2cfda0f51f64697bd5cfdb9
2019-11-11 10:18:06 +01:00
Zuul
5504f947ea Merge "tox: Keeping going with docs" 2019-11-07 20:02:09 +00:00
gujin
bfa44f0bbe tox: Keeping going with docs
Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.

[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045

Change-Id: I806842ce6d2522deab060f5e9ef6607a1e919dbf
2019-11-03 02:44:38 +08:00
Carlos Camacho
0de387ee3c Ansible lint check in THT
Until now if there is a nit inside any
Ansible task part of tripleo-heat-templates we will
need to wait until the end-to-end test imports
the actual tasks to fail.

This submission adds the hability to run a syntax
check for any Ansible task part of any role
defined by default in the resource registry.

Also it will allow to reduce the CI CO2 footprint
by not executing unneeded jobs meant to fail.
Let's save the planet and some CI cycles.

Change-Id: I6eb7c873cc9a4b7d765e745dc260475bff7b268a
2019-10-30 04:56:05 -04:00
Takashi Kajinami
e804f7bd82 Remove duplicated ignore list for pep8 check
Change-Id: I0c3b17f858f093c260d4412544b3562c6200655e
2019-09-05 23:18:25 +09:00
Takashi Kajinami
f47dfe1059 Enforce pep8/pyflakes rule on python codes
This change makes sure that we apply pyflake8 checks on all python
codes to improve its readability.

Note that there are some rules applied for other OpenStack projects,
but not yet turned on, which should be enabled in the future.

Change-Id: Iaf0299983d3a3fe48e3beb8f47bd33c21deb4972
2019-09-05 15:40:46 +09:00
Corey Bryant
f4a4af8701 Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: Ie696d60c6c6d53f189ad4259a1682d030478fdf2
Story: #2005924
Task: #34253
2019-07-12 16:39:23 -04:00
jacky06
8d85e61c36 Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I0f522b848c586506d9556b6821d03f9c0c302cd8
2019-06-26 02:43:46 +00:00
Natal Ngétal
0fb353339a [Configuration] Add missing py37 and corrected default envlist.
Change order of default tox envlist in order of likeliness to fail on a
new change and add py37 and py36.

Change-Id: If72e480c12385463088e2e0962801cd7095a6f12
2019-03-27 10:57:19 +01:00
98k
e7d3bd90c1 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: Ifa8bcc1c1c29551a0ff80edc82699028c8305fe6
Closes-Bug:  #1801657
2019-03-14 08:51:55 +00:00
Zuul
4eb89d09d1 Merge "Update the min tox version to 2.0" 2019-03-12 18:45:27 +00:00
Emilien Macchi
160cddda3f Rename docker_config_scripts to container_config_scripts
Change-Id: Iabd65560c2fc28b3aeca07a21efa861c4c583c01
2019-03-06 09:05:50 -05:00
confi-surya
1e3fd7e846 Update the min tox version to 2.0
It is always safe to define the minimal tox version required to run,
if the host tox is less than this the tool will exit with an error
message indicating the user needs to upgrade tox.

For ref: https://tox.readthedocs.io/en/latest/config.html#tox-global-settings

Change-Id: I3b387930451d78e20a69b8c29b0b28290673ef78
2019-02-04 07:38:32 +00:00
Zuul
f47e2f70a4 Merge "fix tox python3 overrides" 2018-09-19 05:14:47 +00:00
Cédric Jeanneret
d37308d63c Fix python3 support in yaml-validate script.
Python3 drops the "iteritems" method for dict in favor to
"items".

Change-Id: I8d21233e917b5d36be385d59f1a7c9a0588aea2d
Closes-Bug: 1783988
2018-07-27 15:20:32 +02:00
Oliver Walsh
58624abf5f Improve nova statedir ownership logic
The nova_compute container kolla config is currently set to recursively change
the ownership of /var/lib/nova to nova:nova on startup.

This is necessary when upgrading from an non-container deployment to a
containerized deployment as the nova uids are not consistent across the host
and container image.

If the nova instancedir is an NFS mount then open filehandles are
broken and every VM using that NFS export fails with I/O errors.

This change re-implements the nova statedir ownership logic to target only the
files/directories controlled by nova.

Requires dist-git change: https://review.rdoproject.org/r/14734

Change-Id: I57d421feb6356d28002e77fb9bfa50a397758cbf
Closes-bug: 1778465
2018-07-09 17:07:30 +01:00
huang.zhiping
f5ab0e760d fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I09bb55e842515307fbedeee6ba6519dfa4b95e24
2018-06-10 00:16:26 +08:00
Doug Hellmann
08b1132f89 add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I67d2ab519d1baf463e4ad253b8a85b5194ad31f6
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-20 16:10:28 -04:00
Ben Nemec
f66df2ec12 Add "clean" tox target
This uses the -c parameter to process-templates.py to clean the
generated templates in the local repo.

Change-Id: Ib105f60a1e54a6d6372d00bb2db6a9b378b2176e
2017-12-13 17:14:36 -06:00
Andreas Jaeger
faf245de60 Remove -U from pip install
'pip install -U' ugrades specified packages, this is not necessary
since we use constraints, remove the parameter '-U' from the line.

With tools/tox_install.sh - which a previous change of mine removed -
the -U was not harmful, but with the current set up, it might cause
upgrades, so remove it.

Change-Id: I18b21a396b8df4224cb21ce5b8fd062f80604167
2017-12-02 19:17:18 +01:00
Andreas Jaeger
3a4698d8e8 Avoid tox_install.sh for constraints support
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.

This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.

Change-Id: I0b36f95fa92f15159156815bc6d8e1add8902182
2017-12-01 07:47:29 +01:00
Jenkins
de3541a248 Merge "Run roles data validation in CI" 2017-09-25 05:00:45 +00:00
Martin André
987412621f Run roles data validation in CI
The roles data validation added in
2eb1476b6e93279d4ee4988c856dba50ab168bc5 was never run due to a typo in
tox.ini.

Change-Id: I0a151c9863086e4447cedf0790a9e4a18f433cfe
Closes-Bug: #1718391
2017-09-20 12:33:06 +02:00
Ben Nemec
1c9553c37a Add pep8 check that generated environments are up to date
This check ensures that if a parameter is changed that would affect
a generated environment then the environment must be updated before
pep8 will pass.  It will also catch any mistaken hand edits to the
generated files.

bp generated-environments

Change-Id: I2d12992ed55f963285422e1282a4cee06e989b6d
2017-09-07 15:30:31 -05:00
Alex Schultz
2eb1476b6e Add in roles data validation
With the merging of Iad3e9b215c6f21ba761c8360bb7ed531e34520e6 the
roles_data.yaml should be generated with tripleoclient rather than
edited. This change adds in a pep8 task to verify that the appropriate
role files in roles/ have been modified to match how our default
roles_data.yaml is constructed.  Additionally this change adds a new tox
target called 'genrolesdata' that will all you to automatically generate
roles_data.yaml and roles_data_undercloud.yaml

Change-Id: I5eb15443a131a122d1a4abf6fc15a3ac3e15941b
Related-Blueprint: example-custom-role-environments
2017-07-07 09:51:40 -06:00
Ben Nemec
f503d1b0e7 Support config dir for env generator input files
We're not going to want to list every single sample environment in
a single file, so let's also take a directory and just read every
yaml file in it.  This commit adds support for that as well as
some initial environments to demonstrate its use.

Change-Id: If2c608f2a61fc5e16784ab594d23f1fa335e1d3c
2017-06-12 15:02:50 -05:00
Ben Nemec
4e24c8cb6a Sample environment generator
This is a tool to automate the generation of our sample environment
files.  It takes a yaml file as input, and based on the environments
defined in that file generates a number of sample environment files
from the parameters in the Heat templates.  A tox genconfig target
is added that mirrors how the other OpenStack services generate
their sample config files.

A description of the available options for the input file is
provided in a README file in the sample-env-generator directory.

In this commit only a single sample config is provided as a basic
example of how the tool works, but subsequent commits will add
more generated sample configs.

Change-Id: I855f33a61bba5337d844555a7c41b633b3327f7a
bp: environment-generator
2017-06-12 15:02:50 -05:00
Alex Schultz
fa941910cc Use upper-constraints for tox
Use the openstack upper-constraints when running tox.

Change-Id: I9eef36eec749beec0effdb2309fe2ceb9bc557f8
Related-Bug: #1691511
2017-05-17 11:01:07 -06:00
Zane Bitter
5e0a7eee8f Validate that endpoint_map.yaml is up to date in the gate
Change-Id: I72aa48c72c825151739cb478c58e9a6c841c9130
2017-02-01 16:06:20 -05:00
Emilien Macchi
5efee26c97 Add ReNo support
Add ReNo support to manage release notes.
http://docs.openstack.org/developer/reno/

Change-Id: Ie5154d909e616e4e7e813052f9c121d6ac5b0875
2017-01-13 14:35:27 -05:00
Dan Prince
b56e666dd5 Make pep8 task run template generation
This patch updates the pep8 task (which is executed in CI) so
that it generates templates locally. This will give us extra
CI coverage to ensure our generated templates produce valid
YAML.

Change-Id: I2287802d44c0ebe404d3fce30f04efcc3c6ab27f
2016-11-30 21:14:52 -05:00
Dan Prince
4b5b24462b Add local template generation tox task
This patch adds a local version of our template processing
routine so that developers can more quickly view the templates
that are actually getting generated. I've noticed multiple developers
now do a full deployment with 'overcloud deploy' only to download
the swift container with the generated templates. This simple task
avoids that step by allowing developers to generate it locally.

It also aims to preserve the ability to use t-h-t templates directly
with Heat (instead of going through Mistral) should users wish to do that.
The new undercloud heat installer requires the ability to generate
templates without requiring Mistral and Swift to do so.

Ideally the Mistral API workflow would use this same code
so perhaps in the future we might modify that routine to:

 -download swift tarball containing the templates
 -run this local routine that lives in t-h-t
 -re-upload the tarball of templates to the swift container

Change-Id: Ie664c9c5f455b7320a58a26f35bc403355408d9b
2016-11-30 16:02:44 -05:00
Ben Nemec
63df8f1a01 Rename tox env to pep8
It turns out the linters rename was a bit premature[1].  Use the
current standard pep8 name so we don't need custom jobs in the
gate to run this test on proposed changes.

Change-Id: I5226d4c5e3d4095d76cba24fcf27f87c59730587
1: http://lists.openstack.org/pipermail/openstack-dev/2016-February/086268.html
2016-02-11 12:29:36 -06:00
Ben Nemec
e97fdbe639 Rename validate tox env to linters
This is the new blessed naming scheme for lint-type jobs such as
pep8 or the yaml validation job we have in this project.  Doing
this rename will allow us to use standard infra job templates
to run validation on proposed changes.

Change-Id: I0a4c4372429a08e0babb4d323f2b027f1d95f3d7
2016-01-20 15:33:23 -06:00
Ben Nemec
607804f242 Add YAML sanity check
Adds a "validate" tox env for basic sanity checking of templates.
Currently it just validates that all of the .yaml files are in fact
valid YAML.  In the future we might want to add more, but this
seemed like a reasonable start.

Change-Id: I8091bbad0003b150e23dae5de4f465053c982229
2015-09-15 15:31:43 -05:00
Chris Jones
80da8e3968 Add release configuration.
We can now release through openstack.org infrastructure.

Change-Id: I6dff6ae4a97db15bdc4ce419e46e9a125bec277c
2013-10-22 17:49:35 +01:00