Commit Graph

68 Commits (master)

Author SHA1 Message Date
Előd Illés 35db4c21e6 Remove virtualenv pinning
virtualenv pinning was added on Yoga final release time because some of
the deliverables' packaging didn't work with the new python module
discovery (see details: [1]). 1 year passed, hopefully most of the
projects have fixed these errors, so let's unpin virtualenv.

Since newest virtualenv introduces some errors, those need to be fixed:
- skipdist=False needs to be removed otherwise openstack_releases won't
  be installed
- basepython=python3 is not necessary anymore since there is no py2
- allowlist_external is needed for tools/* scripts
This patch also adds python3.10 classifier to setup.cfg


Change-Id: Ibc1f538ddd04c0afeb3d5464fe0b1347ee3ea67a
2023-03-23 20:08:34 +01:00
Dr. Jens Harbott e1375e0158 Update tox.ini for tox 4
The whitelist_externals option is deprecated and has been dropped in
tox 4, replace it with allowlist_externals.

Change-Id: I596be20e2db1bea39be23a9bbcd88afa4d3353eb
2023-03-07 16:38:31 +01:00
Előd Illés 13cc92a1b6 Pin setuptools for Yoga release
The latest setuptools releases (>=61.0.0) broke the build of some
packages that have multiple top-level packages discovered in a
flat-layout. To be able to release Yoga, this patch pins the old
setuptools for our validator script. This needs to be done via pinning
virtualenv because it bundles setuptools as well.

Change-Id: Ia8963bd49f0c898ee1b7219193d28f4af0a868ee
2022-03-28 15:36:29 +02:00
Radosław Piliszek 2d587a5423 Fix gate
By using fixed openstack-governance from Git repo.
The upstream was fixed already [1] but the release cannot be made
as this is where it would be done from.

This patch also amends tox.ini to always install deps from
requirements.txt, otherwise the git repository reference
is simply ignored.


Change-Id: Id711996f90687235624f82fa64e79fb0b5204073
2021-09-17 11:19:46 +00:00
Jeremy Stanley 5b2e171f0c Correct the tox option for skipping sdist generation
The tox option to skip source distribution building is skipsdist,
but this seems to be often misspelled skipdist instead, which gets
silently ignored and so does not take effect. Correct it
everywhere, in hopes that new projects will finally stop copying
this mistake around.

and for details.

Change-Id: I56bd83c4d70e751df6576a38111a5fa5ef656dac
2021-06-17 17:08:58 +00:00
zhangboye 477f4aa44b Use py3 as the default runtime for tox
Moving on py3 as the default runtime for tox to avoid to update
this at each new cycle.

Change-Id: I53ea5d6948a1309ca7264d413108ea1a6af698e4
2021-04-20 14:57:51 +08:00
Sean McGinnis 493b7bf33a
Update to latest hacking for pep8 checks
This updates the version of hacking we are using for our linting and
addresses various issues that the latest version flags.

Change-Id: I95ed73411e96451bc447e1b5858b0466fb8f10a9
Signed-off-by: Sean McGinnis <>
2020-07-27 16:33:03 -05:00
Sean McGinnis b19a92adbe
Install wheel in validate venv
During validation we need to build a bdist_wheel which is only available
as an extension when wheel is present in the environment. This adds
wheel and makes sure all requirements are installed for the validation

Change-Id: Ied3b1ff4accf66e358cec4b209f4d3a65ae88d26
Signed-off-by: Sean McGinnis <>
2020-05-10 17:42:14 -05:00
Andreas Jaeger 0c31414677 Update docs building, Python 3 cleanup
Update docs building:
* Remove unneeded doc sections from setup.cfg
* Change constraints to use published document, use
  new variable
* Import mock from unittest, remove imports from future

Change-Id: I8f33d4c0bf5fcba203d410cd021c548219a757ec
2020-04-20 11:16:46 +02:00
Sean McGinnis cbf3cc9338
Parallelize docs build
This makes sphinx-build use the number of cores available to make the
docs build parallel. On my 16 core system, this cut the time by ~50%. We
don't have as many cores available in the gate, but it should provide a
slight improvement in overall job time.

Change-Id: I7ad4d87a3dba10475a0fcf797a9aa2d4b1ad0eb5
Signed-off-by: Sean McGinnis <>
2020-04-11 15:47:13 -05:00
Daniel Bengtsson dc8c0770dd Update the minversion parameter.
Update the minversion parameter to use the python -m pip to install
python packages:

It's recommend to use this. Remove the useless install_command

Change-Id: I25a43f463dbfac59a193774d6f90b0ddb73f5268
2020-02-17 11:19:47 +01:00
Jean-Philippe Evrard 96c324ba73 Ignore line length for bashate
I am not sure we really need to care about a line length that's 90
chars instead of 79. We have so many ugly things here, I am not
sure this will impact positively the readability to force line breaks.

Without this patch, all the shell scripts changes will fail with
a bashate failure, due to the fact that the
recently introduced changes in tools/
have lines of length above 79.

The alternative would be to fix those lines. There are only 3 of them.

Change-Id: I5fc7bde3c60c50ae78e3c7f297f715a7591e486c
2020-01-22 16:10:11 +01:00
Thierry Carrez 09e8ffb3b1 Introduce tool to check PTL/liaison approval
This is the first step into automating PTL/liaison approval.
This tool will check that PTL/liaisons either authored the change or
approved it.

Change-Id: I1bbb371997e9e92f39eff47adb4d3d176af35de7
2019-12-05 17:35:06 +01:00
Zuul 0e2b206d19 Merge "Add Python 3 Train unit tests" 2019-08-08 09:50:12 +00:00
Corey Bryant 49154ccfd0 Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:

Change-Id: I0fb852f85e435c1ec53b4e250255534c708eab0b
Story: #2005924
Task: #34242
2019-07-19 14:43:41 -04:00
Sean McGinnis 6dae687b0d
Drop references to oslosphinx
Change-Id: Ibb8b14ac8c7f3c56958415c3022de31efef12d52
Signed-off-by: Sean McGinnis <>
2019-07-01 07:37:03 -05:00
Tony Breeds 828e285701 Add a 'generic' release-type.
We use release-types as a way to verify that versions are compatible and
if needed reflected accurately in the code (puppet, xstatic).  If one
isn't set explicitly then we assume python-service.

In certain circumstances (anything other than the first release in a
series) we also perform python specific requirements checking on all
'python' types.

Add a new 'generic' type that uses the same rules to validate version
numbers but wont run any python specific checks.

We need this a projects (like monasca-thresh) will fail the requirements
check for 2nd or greater releases[1].

An alternate would be to have the requirements code check if
exists before calling it but that seems like the wrong layer to me.

Change-Id: I3fcde5eb266f954fddb6871ce8690b93b8fd7a8d
2019-04-18 15:27:27 +10:00
Tony Breeds 0d59816c64 Use a template to generate the 'whereto' testing data
As we add more branches and tags we don't want to manually add tests for
them (and we'd need to or the docs build will fail with an 'Untested

Use the redirections data that we use to generate the htaccess file to
also generate the tests.

Also I removed the debug output as it it's assertion is incorrect we can
actually get the .htaccess file from the docs job \o/

Change-Id: Ib5d35169e68cd92b666a35705d8e36942bd28b89
2019-03-02 15:31:01 +11:00
Tony Breeds 752ce5a491 Generate the constraints redirections from the deliverable data
Instead of statically listing the redirections move to a dynamic model.

We move the existing _extras/.htaccess to _templates/htaccess so we have
some control and safety of what goes in there.  Connect 'build-finished'
from to trigger generating the redirects.  Doing
so here avoids re-reading the data as ahas already done
that for us.

Change-Id: If6bd59fd478593a84ebcedc3a50af3720d620d3c
2019-02-28 12:50:37 +11:00
Thierry Carrez 50258742c9 Add script to facilitate MembershipFreeze handling
This script will compute a list of deliverables present in
governance but unknown to release management, for manual

Change-Id: Ibebf777911416d978ecea5ba8d7b25b211e7ae52
2019-01-16 22:34:21 +01:00
Zuul ad49a15ae0 Merge "Don't quote {posargs} in tox.ini" 2018-10-21 05:30:16 +00:00
Zuul 8f026d9072 Merge "configure oslotest for debug output" 2018-10-11 02:35:22 +00:00
98k 5c06c2d5a6 Don't quote {posargs} in tox.ini
Quotes around {posargs} cause the entire string to be combined into one
arg that gets passed to stestr. This prevents passing multiple args
(e.g. '--concurrency=16 some-regex')

Change-Id: Ia7e151422b69f8dc8b0bd0f05eb72343f1221520
2018-10-09 21:25:51 +00:00
Doug Hellmann 7d3fc1b7a1 remove warnings from using deprecated methods of Sphinx application
Use the sphinx wrapper around logging instead of

Add a -v option to sphinx-build to see the output.

Change-Id: Ia698929c252a2a19da812e399011341a41976540
Signed-off-by: Doug Hellmann <>
2018-10-09 11:08:48 -04:00
Doug Hellmann 1fdf4b0bac configure oslotest for debug output
Capture stdout, stderr, and log output so that when a test fails we
have all of the info we need to debug it.

Change-Id: If3cc9e800dc4a166af96a5fd07903b918e5d3499
Signed-off-by: Doug Hellmann <>
2018-09-24 09:15:26 -04:00
Vu Cong Tuan 9580e50be7 Switch to stestr
According to Openstack summit session [1],
stestr is maintained project to which all Openstack projects should migrate.
Let's switch to stestr as other projects have already moved to it.


Change-Id: Ifbcb65237598b99224768a47c18b30473c2eaa3e
2018-07-10 10:38:33 +07:00
Doug Hellmann b8df43f4de store test temporary files under the .tox environment
Change-Id: I5dbb352285d3e3fcb7fff8ddd18939f3d525f0c6
Signed-off-by: Doug Hellmann <>
2018-03-16 15:47:02 -04:00
Doug Hellmann 6fca7a1f07 move deliverable schema checking into its own command
We've run into problems because we weren't maintaining older
deliverable files as we extended the schema. This change moves schema
validation out of the main validation command so it runs as a separate
step and processes all deliverable files by default. The more complex
validation rules are still only checked for files being modified in a
given commit.

Change-Id: I4b69ca3c59da6606cf2b27d05c8846223d52998a
Signed-off-by: Doug Hellmann <>
2018-02-21 16:23:11 -05:00
Dirk Mueller aa3d3bbc66 Upgrade hacking 1.x
Change-Id: Id54894246f3db134a3868609dbbaa60062a92c5e
2018-02-11 18:01:01 +01:00
Doug Hellmann 0875f22e23 redirect shade team page to openstacksdk
Since the shade team is renamed, redirect the old team page location.

Change-Id: I7cc485365ef370ebc724c51557dd2c77b9e8c75e
Signed-off-by: Doug Hellmann <>
2018-01-02 11:01:51 -05:00
Sean McGinnis bccb7dea80
Add doc job requirements.txt
We had previously installed our docs dependencies as extras in
setup.cfg. Doc jobs are now changed to not install the project,
so these dependencies are no longer being installed.

With recent changes to the doc jobs, we need to specificy that python3
is used. Update the zuul config file for this.

Change-Id: Ie925efefb1126dbe7fa0a9f0f1727da01f48c512
Depends-on: Ib882a4d09898ff206b36ac8aef840bee0748368e
2017-12-19 15:37:22 -06:00
Doug Hellmann 248aca1a48 log yamllint output to its own file
Adjust the tox log wrapper so it works with the full path to the
script for running yamllint, then add it to the step for running
yamllint so the results are logged to a separate file.

Change-Id: I98561e17e4fa76aeee9adfd57d9e71a7f229e536
Signed-off-by: Doug Hellmann <>
2017-12-05 23:38:14 -05:00
Doug Hellmann 3054c9cdde log output of validate-requests separately
Make the validate job log to a separate file like list-changes does so
we can find the output more easily instead of reading through all of
the zuul output.

Change-Id: Ib5286384aca2b126b9cad93084aaef873883d20a
Signed-off-by: Doug Hellmann <>
2017-11-21 10:20:40 -05:00
Doug Hellmann 178cc689aa log the output of list-changes to a separate file for easier reading
Having the rather complex output of list-changes mixed in with the
main job log can make it difficult to read. Continue to include the
data in the main log file but also write it to a separate file that is
easier to consume.

Change-Id: If4d3b56c76b2bf513b2d44ffedf5825f562f4bfb
Signed-off-by: Doug Hellmann <>
2017-10-25 19:08:51 -04:00
Doug Hellmann fffdada0bd pass $HOME to tox commands so git can find its config file
list-changes now needs to be able to add a temporary tag to build the
release notes. Adding the tag requires identity information not previously
required for other git commands, and git needs to know where HOME is to
find that info.

Change-Id: Ia9a506bdfa30cf72b48d53a7ec05b0bc939f34a6
Signed-off-by: Doug Hellmann <>
2017-10-16 09:22:21 -04:00
Doug Hellmann bf8de9c316 run commands without buffered I/O
A lot of the commands in this repo use zuul-cloner in a
subprocess. Using unbuffered I/O causes the stderr from zuul-cloner to
appear interleaved properly with the stdout from the main command.

Change-Id: I87c2a6f49e9c9f307ebe42d23ee79cd6344573bf
Signed-off-by: Doug Hellmann <>
2017-08-28 12:00:05 -04:00
Doug Hellmann 8477f4fde9 add wrapper for yamllint
We don't need to run the linter over every single old deliverable file
every time. Just check the "modern" series that are most likely to
be open, and the _independent projects.

Change-Id: I5207600838c37b241d0ab178080be75a8f31a725
Signed-off-by: Doug Hellmann <>
2017-07-31 17:11:02 -04:00
Doug Hellmann 04bb43a585 add py35 to the default list of test jobs to run
Change-Id: I5367a2e081cb167467ee34d07a6ace7db14f0374
Signed-off-by: Doug Hellmann <>
2017-03-15 15:10:10 -04:00
Doug Hellmann 3092b4d51c change basepython to 3
Use python3 by default, allowing tox to find the best fit (instead of
specifying 3.5).

Needed-By: I8b93b88bb4123d2913554b08c2284a7972e291f2
Change-Id: I8b93b88bb4123d2913554b08c2284a7972e291f2
Signed-off-by: Doug Hellmann <>
2017-03-03 15:43:29 -05:00
Doug Hellmann 83fd1ea44c add docs to the list of default tox environments
Build the docs when the user runs "tox" without args.

Change-Id: I16b683cf82715f0351c7f2ca5c0655b36f5fe4f1
Signed-off-by: Doug Hellmann <>
2017-03-03 15:00:04 -05:00
Doug Hellmann 42e8a6fd7f remove unused tool with launchpadlib dependency
The only reason we have a dependency on launchpadlib and
lazr.restfulclient is for the old tool that imported project history
from launchpad. We no longer need the tool, so we can remove it and the
dependency as part of our python 3 port.

Change-Id: Ie87e6172c953df10cb70d34c9bc23ad30e97ce4d
Signed-off-by: Doug Hellmann <>
2017-03-03 14:59:59 -05:00
Doug Hellmann afef890fb6 add tox environment for running aclmanager
Change-Id: Ia10a287e4fe0f8003b5943914cb1dd815521d768
Signed-off-by: Doug Hellmann <>
2017-01-09 10:00:01 -05:00
xhzhf 220dd9b933 H803 hacking have been deprecated
H803 hacking have been removed since hacking 0.10.

Change-Id: I15c5da46273e9fbf1de20f5bc6115e4fe710c555
2016-12-18 10:33:55 +08:00
Doug Hellmann 333f81f738 configure tox to install editable versions of tools
This configuration makes it easier to work on changes to the tools.

Change-Id: Ic4afe2a811d6eb2163a2590a865b0f56d1474eb8
Signed-off-by: Doug Hellmann <>
2016-12-02 08:21:50 -05:00
Doug Hellmann f58b6b7eb9 set up for unit tests
Set up commands to run to execute new unit tests.

Ignore output from testr and coverage.

Rename the dist to match the package name so that the testr/coverage
integration works properly.

Add test-requirements.txt with some common testing dependencies.

Change-Id: Ic6f34df893f3981a9af6122456b9feb9c80b31dd
Signed-off-by: Doug Hellmann <>
2016-11-28 12:47:17 +11:00
Luong Anh Tuan fbaf7f1412 Don't include openstack/common in flake8 exclude list
There is no this directory openstack/common which was used to keep codes from
oslo-incubator, we have retired oslo-incubator. Removing openstack/commonfrom
all OpenStack code in favor of the Oslo libraries is a project wide goal
for the Ocata release. So don't use this directory any more. We should drop it
for improving searching efficiency.

Change-Id: I38429dd29767afdbf395cbb3777879636ea0ad55
2016-10-31 11:51:49 +07:00
Doug Hellmann c043a7a48f pin yamllint to 1.4.1
A new update of yamllint introduced new rules that we're failing. Treat
this linter like we do others and pin it, so we can update it and fix
any issues from the new version at the same time.

Change-Id: I51800955a5d25b52910e9a6a9de8578be8519028
Signed-off-by: Doug Hellmann <>
2016-10-12 08:55:34 -04:00
Doug Hellmann c50bc84923 configure tox to always reinstall the tools
When running commands using "tox -e venv -- $name" while working on the
missing-releases command, I noticed that changes to the program were not
being installed into the tox environment as I made them and re-ran
tox. These changes fix that issue by having tox always re-install.

Change-Id: Iea132f5fecfc47a0cd4bbb3c7777568da841bf53
Signed-off-by: Doug Hellmann <>
2016-08-31 13:23:12 -04:00
Andreas Jaeger f2fadb1fe5 List system dependencies for running common tests
Add an other-requirements.txt file containing a cross-platform list
of dependencies needed for running included tox-based tests. Also
include a tox environment for convenience calling the bindep[*]
utility to list any missing system requirements.

This change is self-testing.

For other-requirements.txt see also


Change-Id: Ib29d02d5ea751f09df26030dcfb9e153ba82661b
2016-06-27 14:05:25 +02:00
Doug Hellmann 97986ff795 invoke sphinx directly instead of through pbr
The pbr feature for invoking sphinx doesn't let us treat warnings as
errors because of a bug in pbr. Switch to invoking sphinx directly,
including removing the setup.cfg instructions for sphinx, so we can use
the flag to treat warnings as errors. Doing this also requires fixing a
few existing warnings, so those changes are included in this patch.

Change-Id: I2aa0d844855d5a18646d0bc3907620544ae71be2
Signed-off-by: Doug Hellmann <>
2016-04-20 13:15:09 -04:00