182 Commits

Author SHA1 Message Date
Hervé Beraud
a8ab91eebc Adding a new tools to search topic on the ML
By default it will research for topics related to the release team but
pattern can be overriden to looking for all kind of topics (c.f the
related doc).

Usages
------

The following example will search for all emails sent openstack-discuss
who match release topics from the creation of the mailing list to the
current day:

```
$ tools/search_emails.py
```

In the following example we search for release countdown emails sent
between May 2020 and today:

```
tools/search_emails.py --topic ".?\[release\] Release countdown.*" --starting-date 2020-5-1
Looking for emails between 2020 May and 2020 September and sent by
Analyzing May 2020: http://lists.openstack.org/pipermail/openstack-discuss/2020-May
Analyzing June 2020: http://lists.openstack.org/pipermail/openstack-discuss/2020-June
Analyzing July 2020: http://lists.openstack.org/pipermail/openstack-discuss/2020-July
Analyzing August 2020: http://lists.openstack.org/pipermail/openstack-discuss/2020-August
Analyzing September 2020: http://lists.openstack.org/pipermail/openstack-discuss/2020-September
15 result(s) have been found
2020-May:
        - [release] Release countdown for week R-1, May 4 - 8 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014577.html
        - [release] Release countdown for week R-0, May 11 - 15 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014780.html
        - [release] Release countdown for week R-20, May 25 - 29 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014999.html
2020-June:
        - [release] Release countdown for week R-17, Jun 15 - Jun 19 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015415.html
        - [release] Release countdown for week R-16, Jun 22 - Jun 26 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015541.html
2020-July:
        - [release] Release countdown for week R-13  July 13 - July 17 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-July/015878.html
        - [release] Release countdown for week R-12  July 20 - July 24 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016056.html
2020-August:
        - [release] Release countdown for week R-10  August 3 - 7 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016243.html
        - [release] Release countdown for week R-8 Aug 17 - 21 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016534.html
        - [release] Release countdown for week R-7 Aug 24 - 28 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016709.html
        - [release] Release countdown for week R-6 Aug 31 - Sept 4 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016894.html
2020-September:
        - [release] Release countdown for week R-5 Sept 7 - 11 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-September/017079.html
        - [release] Release countdown for week R-4 Sept 14 - 18 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-September/017210.html
        - [release] Release countdown for week R-3 Sept 21 - 25 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-September/017399.html
        - [release] Release countdown for week R-2 Sept 28 - Oct 2 - Sean McGinnis
        http://lists.openstack.org/pipermail/openstack-discuss/2020-September/017616.html
```

Change-Id: Ib1fe5f74b408ac489b12c4ef003860ce7aee1532
2020-10-02 15:57:38 +02:00
Sean McGinnis
355cd7debd
Add script to semi-automate bulk releases
The release team has a few cases now where we need to propose release
patches for a large set of deliverables. Due to differences in semver
choice based on actually merged commits, along with other decisions that
need to be made while doing these, this process can't be completely
automated. But this adds a script that will automate the majority of the
process to simplify it as much as possible.

Change-Id: I6ec9fa77baab58df93bdadc0ac3c3fa5d3e18804
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-09-11 14:48:55 -05:00
Sean McGinnis
493b7bf33a
Update to latest hacking for pep8 checks
This updates the version of hacking we are using for our linting and
addresses various issues that the latest version flags.

Change-Id: I95ed73411e96451bc447e1b5858b0466fb8f10a9
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-07-27 16:33:03 -05:00
Sean McGinnis
08cd1fb33c
Switch to PyYaml safe_load
The load() call from PyYaml is considered a higher security risk in that
it uses the FullLoader. safe_loade() is considered more safe by using
the SafeLoader instead.

Since the 5.1 release of PyYaml added warning output when using load(),
this switches over to safe_load() to avoid the unnecessary noise.

Change-Id: I1949deed094822d2c2c56659eadb1fc5ea6a59e5
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-07-27 16:18:19 -05:00
Zuul
ca7d97f823 Merge "Allow list_unreleased_changes to format results at json & yaml formats" 2020-07-10 16:05:02 +00:00
Hervé Beraud
4912f7d5d0 Allow list_unreleased_changes to format results at json & yaml formats
Rewrite list_unreleased_changes as python format and add new features.

By default it will behave as the previous version of this command (the
shell script).

Few new feature have been added in these changes to allow us to more
easily handle outputs in scripts.

Features added:
- allow user to retrieve results in json format
- allow user to retrieve results in yaml format
- allow user to ignore project not yet released

The shell script entry-point (tools) is still there but it will call
the python command in a venv instead of directly implement features.

Also the python version allow us to more surround this tools with unit tests.

Change-Id: Iaf86ecb1589c40102acb621b23ea12d71ed453bb
2020-07-07 19:07:25 +02:00
Hervé Beraud
339af5321a Fix tox command in tool help
Also prefer to use retrive command name dynamically.

Change-Id: Iffe7fe3216256ddb52001e4fcfa49cc3ea82d6e8
2020-07-07 17:39:24 +02:00
Sean McGinnis
62c2fc4317
Use updated git cache dir default path
Change-Id: I7c3ed1e08c7ef1a62845df0d9c5b22f400d8d90b
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-13 06:13:07 -05:00
Sean McGinnis
27d4f7db98 Update add_reviewers to add to all open reviews
This updates the script to not require a specific topic to make it
easier to automate a job that periodically makes sure all open reviews
have had the PTL and liaison added.

Also refactored the adding of teams because bash array handling can be a
bit wonky sometimes and not always append values. This makes sure no
teams are skipped.

This is slightly less optimized for large reviews like the final series
release since it loops through teams based on the deliverable files. As
a slight optimization, it will skip a team if it was the same team as
the last deliverable file. It just might be possible that the same team
will be processed multiple times if their deliverable files are not in
alphabetical order, but there is no harm in that other than taking
slightly longer to complete.

Change-Id: I9a9194ea276aa0120aef21a321f48a93419fb848
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-09 07:15:06 -05:00
Hervé Beraud
ae48bf825b Display more help to the new-release wrapper
If we miss something with this command we need to get the
python script's help to get how to use this tools, these changes
add the command to execute through the wrapper to get this help.

Change-Id: I1e9fceece6fee40722e21e49ca5689faacefdf93
2020-05-05 22:37:17 +02:00
Thierry Carrez
d40c4ca491 Rely on Zuul to provide project name in announces
If we run in CI, rather than prioritizing using 'python setup.py --name'
as the project name to be used in release announcements, use the
repository name as provided by Zuul.

This should fix the issue reported in:
http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014359.html

Change-Id: Ia0026356676561a6b3a55ab05db5ed7a58b646b6
2020-04-24 12:41:09 +02:00
Sean McGinnis
a7a5ad2a94
Add script for adding PTL and liaisons to reviews
This adds a script that will go through all release reviews under a
given review topic, find the PTL and release liaisons for the owning
teams of those releases, and add them as reviewers to the patch.

Change-Id: I6294c1d9da7d6a977df6d8460aa085d2cc7e72a5
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-23 07:40:58 -05:00
Sean McGinnis
56f6ffaa83
Update upstream cloning URL
Change-Id: I38d07ddefc7b3b6faf0156adc0f3fe5faf86b72c
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-21 14:25:41 -05:00
Hervé Beraud
1be644f0d4 trivial - fix nits in script
Change-Id: Ic4687dfea7c71a193922d5286106d1783e9a6bdf
2020-02-20 17:40:09 +01:00
Julia Kreger
5d3a49e61d Remove coreos-image-builder references
We are taking down coreos-image-builder since it
was never really used, and left abandoned.

More information on this can be found at:
https://review.opendev.org/#/c/707181/

Change-Id: Idb42353f86f192cc8ff2defbbae71e7df76702c6
2020-02-11 08:58:09 -08:00
Zuul
c276874854 Merge "governance/releases consistency check tool" 2020-02-05 21:38:09 +00:00
Tony Breeds
60e786d724 Add tools/bulk_review.sh and get-contacts
get-contacts:
To extract the PTL/Liaison contact information from the governance data
/ release team data

tools/bulk_review.sh:
To manipulate a dirty working-tree and post as a single gerrit topic,
one change per team.  Changes are unordered so any change can be merged
when it's ready/ approved by the appropriate team

Also enhance get-deliverable-owner: to detect file paths

Story: 2005704
Task: 31028

Change-Id: Ia319e8a7b4da195cb4bc861c51025a41adc43bb3
2020-01-23 11:52:59 +00:00
Thierry Carrez
b54fc8defe governance/releases consistency check tool
Introduce a new tool to list inconsistencies between projects.yaml
in governance repository and deliverables in the releases repository.

Sometimes projects.yaml defines release-managed deliverables, but the
release team is not handling them. Sometimes, the release management
team handles deliverables which are not officially produced by an
OpenStack project team, and should probably not be listed on releases.o.o.

This tool lists both types of inconsistencies, and can be integrated
to release management process so that they are handled before becoming
an issue.

NB: Once ready, this tool could be used to replace the "membership
freeze check" tool which only looked at some of those inconsistencies.

Change-Id: Id263f0c91f872111fe997e2b8a1088b61033d114
2020-01-23 12:40:43 +01:00
Zuul
bdd5e522e7 Merge "Address review comments for make_missing_releases" 2020-01-13 19:50:53 +00:00
Zuul
ee2df08000 Merge "Make check_approval not require openstack_governance" 2020-01-10 14:15:49 +00:00
Zuul
2e99e0f951 Merge "Add wrapper script that can be used to update deliverables" 2020-01-09 18:41:47 +00:00
Sean McGinnis
b1ed70c939
Address review comments for make_missing_releases
Fixes some comments from the initial review and minor cleanup.

Change-Id: I17f18f63cb7e9da4bc7504405472e3ac39151c3e
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-01-09 11:56:25 -06:00
Sean McGinnis
bcb61fece9 Fix library shortname identification in announce script
Git repo origins are now removed in CI jobs, so our detection of project
short name from looking at the git remote now fails. This resulted in
release announcements linking to "null" instead of the actual project
location [0].

This updates the script logic to detect if that is the case and fall
back to getting the repo name from the directory, which should be safe
in a CI job since there is no manipulation of the clones repo directory
name.

Also cleans up an unused variable that was left from a previous
refactoring.

[0] http://lists.openstack.org/pipermail/release-announce/2019-December/008306.html

Change-Id: I84e2cf258e9bc4541b7edde55a22a4be24bdb232
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-12-20 12:33:40 -06:00
Thierry Carrez
53fba5642b Make check_approval not require openstack_governance
Since the job will have to run on the Zuul executor, it needs to
only rely on stdlib, yaml and requests (which are already installed
on the executor).

Change-Id: Id4e5a88d352e95402d0899deceade6fed8146619
2019-12-17 18:21:29 +01:00
Thierry Carrez
09e8ffb3b1 Introduce tool to check PTL/liaison approval
This is the first step into automating PTL/liaison approval.
This tool will check that PTL/liaisons either authored the change or
approved it.

Change-Id: I1bbb371997e9e92f39eff47adb4d3d176af35de7
2019-12-05 17:35:06 +01:00
Hervé Beraud
2064f3e29f Remove unused imports in tools and commands
For some non obvious reasons pep8 check doesn't detect these unused imports.

Change-Id: Ie98af5a0fcf4fb8bcd05051de75a84755fbc0b88
2019-11-27 17:45:34 +01:00
wangfaxin
8d3a0b0384 Fix ssh host addr link.
Change-Id: I2298ef11e15ce7ade4c232899df1c1d2225c9c5d
2019-11-20 18:16:54 +08:00
Tony Breeds
7045ff71b0 Add wrapper script that can be used to update deliverables
At various stages we want to update lots of deliverables.  This acts as
a tool to consolidate what we mean to supplement the docs and etherpad

Story: 2005707
Task: 31033

Change-Id: Ied1f6d06a33baf3d3ef3d789bbe5c7f70d967d62
2019-11-18 12:24:00 +11:00
Sean McGinnis
44d6736a5e
Be explicit about python3 executable
Per best practices, explicitly use the python3 executable rather than
assuming the platform with have a "python" executable that maps to
python3.

https://www.python.org/dev/peps/pep-0394/

Change-Id: I39b8a1013a891f4570f374d8faa3cfa2ecaf3347
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-10-16 09:45:29 -05:00
Sean McGinnis
73d6e8b1a0
Update lib release instructions for freeze
This updates the instructions for client and non-client lib freeze to
make sure all changes are captured in a final release. Also adds a
helper tool to separate out the two types of libs since they have
different deadlines.

Change-Id: I29ea73bcc5a9649d7d9c502ebc0a05c840e41484
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-09-19 16:04:06 -05:00
Sean McGinnis
123864e507
Fix bashate errors
Running our default tox environments locally, the bashate tests fail
with several errors due to line length and indentation levels.

Change-Id: I15ec6a9cb1dbdb2200294c5aa2d2eb6074f00a25
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-07-08 11:46:30 -05:00
melissaml
e700061762 Rename review.openstack.org to review.opendev.org
There are many references to review.openstack.org, and while the
redirect should work, we can also go ahead and fix them.

Change-Id: I0d46b1a4e00c1775ea5e38d39e87bed99f6bbb2e
2019-05-12 04:33:25 +08:00
Sean McGinnis
8b1a9ada18 Add script to transition entire series to EM or EOL
Probably mostly useful for extended maintenance, but this adds a script
that can add a $series-em tag to all deliverables of a series as we
claimed we would do in the extended maintenance policy changes [0].

[0] https://docs.openstack.org/project-team-guide/stable-branches.html#extended-maintenance

Change-Id: Ibeec88bf4e81b97c523f416669e07a10aa86a53b
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-04-11 06:44:55 -05:00
Doug Hellmann
915a9a9d33 add a tool to show changes in rc-based repos
Simplify the instructions by providing a special script and then
update the instructions to refer to it.

Change-Id: Ibbcd422c06dc4aeb3df503f3c0f084b39d3bf0f7
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2019-03-27 13:54:24 -04:00
Zuul
e1b557915a Merge "Fix announce email paths" 2019-03-27 11:00:33 +00:00
Sean McGinnis
ba0843c051
Fix announce email paths
Our announce email included a few errors in the paths provided. If the
git remote.origin.url ended with ".git" the tarball path was wrong. We
were also using the package name for git and doc paths incorrectly. This
fixes the name used so the resulting links are correct.

Story: #2005279
Task: #30131

Change-Id: I7a64761778b7391c0f3188b1177f03a7e712bd2a
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-03-25 11:20:13 -05:00
Ian Wienand
8c030f0eaf Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I648dbeadf4ff11e3d9f23521be530b36186927d9
2019-03-25 08:27:17 -04:00
Doug Hellmann
3f24d9dd78 look at teams in the branch being scanned
Teams come and go, so we want to iterate over the set of teams in the
branch that we are scanning for unreleased changes instead of assuming
the group from ocata is correct.

Change-Id: Iacf3933cbfc14f9a3e7b404171247108c27083f3
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2019-03-14 08:43:57 -04:00
Előd Illés
e577664cdd Put the lists under a directory
With this script now the txt files with the lists are stored under
<releas>-<date>-<time> directory.

Also small modification to show the links and commit messages of the
open changes in one line.

Change-Id: I3563445e7693448b3308ef4df8739532d1fc3f99
2019-03-12 20:06:17 +01:00
Jean-Philippe Evrard
196c4f2df2 Use env python instead of /usr/bin/python
If running with tox venv, the python version (where all the
requirements are installed) might be different than /usr/bin/python.

We should use the one in the env.

Change-Id: Ic69282b289fefeafa5a63b6da284f44c4b3c4b15
2019-03-04 20:20:59 +01:00
Zuul
8262f18660 Merge "Allow user to generate urls for the given results." 2019-02-07 10:29:40 +00:00
Zuul
b94c3c119d Merge "Skip repos if not handled by release management" 2019-02-06 22:12:03 +00:00
Hervé Beraud
6b3259ee21 Allow user to generate urls for the given results.
Introduce the flag `--url` who allow user to generate urls
for the given result.

Introduce the option `--distgit` to allow user to specify
the base url to use to generate urls.

Change-Id: I10d4f92f64a68f283c70c2c880c27a069f123762
2019-02-01 15:10:23 +01:00
Zuul
a806227734 Merge "Improve script by counting results found." 2019-02-01 13:52:19 +00:00
Hervé Beraud
062f3a2ddc Improve script by counting results found.
Display the number of results at the end of the output.

Change-Id: Ic603c8557dbf8c8436d64f3910c0e0cf3d86ea84
2019-02-01 10:52:23 +01:00
Thierry Carrez
c1b2b78b07 Skip repos if not handled by release management
Leverage the newly-introduced 'release-management' key from
projects.yaml to skip repositories in deliverables that are
not handled by the release-management team.

No longer maintain a local list of repository exceptions.

Change-Id: I0e5eab7a2aeb3bd09acf7bf01356a9f664530b0f
2019-01-31 16:43:18 +01:00
Hervé Beraud
bd589e76e3 Allow users to format output results to the yaml format.
Allow users to format output results to the yaml format
by using the flag --yaml.

Passing this flag prefixing each results with dash in the stdout.

Change-Id: I460c17704c5311d6fb422d5e56be8080344ab986
2019-01-31 16:34:10 +01:00
Thierry Carrez
50258742c9 Add script to facilitate MembershipFreeze handling
This script will compute a list of deliverables present in
governance but unknown to release management, for manual
processing.

Change-Id: Ibebf777911416d978ecea5ba8d7b25b211e7ae52
2019-01-16 22:34:21 +01:00
Doug Hellmann
510e8c9a5d add a script to get a list of unreleased changes for a team
Change-Id: I061cea61dd5d0a7711faae9e06809993caf36de9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-11-19 12:57:18 -05:00
Sean McGinnis
32afc4160d Update tooling to support cycle-with-rc model
This updates commands and tools to work with the cycle-with-rc model.

Change-Id: I8df85df1c84ae6d8fb37a5206e155f84f8fac947
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2018-10-24 08:41:13 -05:00