Commit Graph

164 Commits

Author SHA1 Message Date
Hervé Beraud
fb08afc1fa Fix gitutils to use release-id
With the new release identification / naming schema [1] (like:
2023.1 Antelope) new stable branch naming was introduced (like:
stable/2023.1). This was not handled in check_branch_sha method
in validate.py, as the method still searches stable/$series.

This patch fixes this by reading the 'release-id' field from
series_status.yaml if present and uses it as stable/<release-id>
for the branch search.

[1] https://governance.openstack.org/tc/reference/release-naming.html

Change-Id: I014b1d6dc561be4db0fc8faa85fb3133e851acfc
2023-03-10 17:57:53 +01:00
Előd Illés
32c9bc69eb Fix validator for tagless deliverables
With the new release identification / naming schema [1] (like:
2023.1 Antelope) new stable branch naming was introduced (like:
stable/2023.1). This was not accepted in the validator command.
Previously the validator was fixed for 'std' and 'std-with-versions'
branching typed deliverables, but missed the 'tagless' deliverables
case. This patch fixes this by accepting the new branch naming style,
stable/<release-id>, for tagless deliverables, too.

[1] https://governance.openstack.org/tc/reference/release-naming.html

Change-Id: I6ba51454213e095e6b76e0813dab3ce44b1457ba
2023-02-27 17:16:25 +01:00
Előd Illés
27c4dcced7 Fix validator for std-with-versions
For projects that has branch type 'std-with-versions' (like ironic,
because they cut bugfix/<MAJOR>.<MINOR> bugfix branches) the new stable
branch name style (like stable/2023.1) does not pass validator. This
patch extends the validator to accept these branch names, too.

Change-Id: If597684b1a4ea741707ee786e8e01e9e8f3d2cb4
2023-01-20 11:40:59 +01:00
Előd Illés
2b8ee3aaec Fix upper-constraints redirect for new release naming
The new release identification / naming schema [1] (like:
2023.1 Antelope) broke the redirection rules for branch specific upper
constraint files as the stable branch names are from now on based on
the new release identification (like: stable/2023.1).

This patch fixes the redirections by introducing the release ID field
to series status, to translate a release name to 'stable/<release-id>'
where this ID exists.

[1] https://governance.openstack.org/tc/reference/release-naming.html

Change-Id: Iab885d4e36f64903b323e16c74d8315c759584de
2023-01-14 10:29:27 +01:00
Hervé Beraud
032f60e961 Fix stable branch naming validation
This fix aim to solve issues with the new naming convention related to
SLURP releases.

Without that our validation will fail to validate stable branches named
like `stable/2023.1`.

Change-Id: Ic45d4a13f63846e5b60bde800492da7c851addf8
2022-12-02 14:33:14 +01:00
Hervé Beraud
9504e96b5d Fix validation to ensure that first release of series isn't a bugfix
We recently observed that this check was skipped. Indeed it was
possible to propose a bugfix version on the first release of a series
without facing validation errors [1].

The described situation was allowed by checks that we introduced
during Wallaby [2]. Those was designed to ensure that we can
build sdist from the proposed tag. Indeed, they temporarly created
the new git tag to build sdist from it. However we never delete this
tag after our tests so this temporary tag is see as an existing tag
and so that led us to the observed bug.

Our validation check that the first version number of series isn't a
bugfix after it tried to build the sdist from the new tag, so, without
this patch the tag newly created by the sdist check remains available
in the repo, and so a couple of next checks are skipped, including the
validation of the version number, because these checks are decorated
to ignore existing tags.

Indeed the function decorated with `skip_existing_tags` detect this
temporary tag as an existing tag, so, the decorated functions are
skipped.

We should notice that it surely impacted other checks in the same manner.
Indeed, all validation functions decorated with this function
`skip_existing_tags` are impacted by this bug.
Hopefully it didn't have too much side effects.

These changes delete the temporary tag after usage to ensure to restore
the initial environment and to stop skipping next checks.

[1] https://review.opendev.org/c/openstack/releases/+/795836
[2] 80652b5232

Change-Id: I5d9024528d08487a7582e0a0e73576b22708a6cf
2021-06-17 13:41:12 +02:00
Sean McGinnis
493b7bf33a
Update to latest hacking for pep8 checks
This updates the version of hacking we are using for our linting and
addresses various issues that the latest version flags.

Change-Id: I95ed73411e96451bc447e1b5858b0466fb8f10a9
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-07-27 16:33:03 -05:00
Hervé Beraud
4912f7d5d0 Allow list_unreleased_changes to format results at json & yaml formats
Rewrite list_unreleased_changes as python format and add new features.

By default it will behave as the previous version of this command (the
shell script).

Few new feature have been added in these changes to allow us to more
easily handle outputs in scripts.

Features added:
- allow user to retrieve results in json format
- allow user to retrieve results in yaml format
- allow user to ignore project not yet released

The shell script entry-point (tools) is still there but it will call
the python command in a venv instead of directly implement features.

Also the python version allow us to more surround this tools with unit tests.

Change-Id: Iaf86ecb1589c40102acb621b23ea12d71ed453bb
2020-07-07 19:07:25 +02:00
Sean McGinnis
804e560832
Add std-with-versions stable-branch-type
This adds a new std-with-versions branch type. This is used to control
validation logic when branching to allow the Ironic team to create
intermediary stable branches based on major.minor version numbers in
addition to our normal expected stable/$series branches.

https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/new-release-model.html

Change-Id: Ic482c77a2c177162ffe37643a455ac1724a658b3
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-06-02 14:34:32 -05:00
Andreas Jaeger
0c31414677 Update docs building, Python 3 cleanup
Update docs building:
* Remove unneeded doc sections from setup.cfg
* Change constraints to use published document, use
  new variable
* Import mock from unittest, remove imports from future

Change-Id: I8f33d4c0bf5fcba203d410cd021c548219a757ec
2020-04-20 11:16:46 +02:00
Sean McGinnis
7449c29ce9
Don't allow tagging em or eol with no branch
Change-Id: I2f113d8519338665606bf267084917b1fdde1a44
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-03-07 07:13:37 -06:00
Sean McGinnis
05f5aed7fa
Make sure -em tag is on last release
We don't want to allow tagging EM if there were no releases. We also
want to make sure the -em tag is on the last official release that was
done.

Change-Id: I7afb3a52cf2ec47d8e0154b51825b500806aa590
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-03-06 17:08:09 -06:00
Thierry Carrez
655b62e84c Introduce 'abandoned' release model
Introduce an 'abandoned' release model for cycle-independent
deliverables. It should only be applied to deliverables in the
_independent directory. No new release should be accepted for
deliverables with this release-model.

Change-Id: I65c163888c37f7a7f77273abf3ca0633923a0fe2
2019-12-18 17:57:06 +01:00
Tim Burke
e392f0bd48 Redirects to master should always be temporary
We *know* that requirements will have a stable/train branch soon; make
sure clients don't go caching the redirect to master.

Change-Id: I8507d86e4f553b698163be61665267d42e033d91
Related-Change: Ia2e8c46c27ac97217576afdd1677efba4b99fc37
2019-10-03 15:10:33 -07:00
Sean McGinnis
e5362d5d2c
Skip minor version increment validation for branchless
Tempest plugins do not branch, therefore we do not need to enforce that
the minor version is incremented to leave room for stable releases.

Change-Id: I22ba5cb42070b5a450617872de2da337b327de1c
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-08-08 12:19:33 -05:00
Thierry Carrez
3f7c9e782f Add stable-branch-mode:none option
Some "other" deliverables like tempest and patrole never
create stable branches. This allows to clearly mark such
deliverables and add a corner case in validation tests.

Change-Id: I2f6414d0f71baad58335702743f2180f8da3273f
2019-07-15 12:27:33 +02:00
Thierry Carrez
1bcdeaaa9f Do not allow final without RC in cycle-with-rc
Currently validation considers it valid to propose final versions
on a cycle-with-rc model without RC first:

https://review.opendev.org/#/c/613444/2

Relying on human review to catch that sounds risky. Let's fix it.

Change-Id: Id06794f165f0c852fb62dcacd53058bdc756899e
Task: 27762
2019-06-20 14:54:50 +02:00
Kendall Nelson
59c38e6708 Update Liaison Scripts
Now that the release liaison info lives in data/release_liaisons.yaml,
we don't need to pull from the wiki. This patch also adds a schema
to verify the release_liaions.yaml is formatted correctly.

Story: 2005702
Task: 33639
Task: 33733

Change-Id: Ic4de16c26bb1a4fb878b86d2bd9c59bf5f54d11d
2019-06-19 16:22:29 -07:00
Tony Breeds
5801fbdc77 Switch constraints files over to the new opendev infrastructure
Currently constraint redirectiones that point to EOL releases don't work
as expected due to gitea needed to differentiate between tags and
branches.  Rather than fix, and rely on multiple redirects lets just
craft 301's that point to the correct gitea url.

This means we now need to know when a target is a branch or tag but
that's pretty simple to intuit given our deliverable structure.

Change-Id: Ife030f8ee7b5d204b054f99e920a675f7d92da69
2019-05-22 14:33:37 +10:00
Tony Breeds
828e285701 Add a 'generic' release-type.
We use release-types as a way to verify that versions are compatible and
if needed reflected accurately in the code (puppet, xstatic).  If one
isn't set explicitly then we assume python-service.

In certain circumstances (anything other than the first release in a
series) we also perform python specific requirements checking on all
'python' types.

Add a new 'generic' type that uses the same rules to validate version
numbers but wont run any python specific checks.

We need this a projects (like monasca-thresh) will fail the requirements
check for 2nd or greater releases[1].

An alternate would be to have the requirements code check if setup.py
exists before calling it but that seems like the wrong layer to me.

[1] http://logs.openstack.org/54/652854/1/check/openstack-tox-validate/80df01c/job-output.txt.gz#_2019-04-16_06_08_00_636538
Change-Id: I3fcde5eb266f954fddb6871ce8690b93b8fd7a8d
2019-04-18 15:27:27 +10:00
Sean McGinnis
7450ab3a71 Drop references to git.openstack.org
Updates to use opendev.org or published documentation.

Change-Id: I0bd488b25d1259bce3f723a6c58283fdf670e721
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-04-15 14:25:18 -05:00
Tony Breeds
752ce5a491 Generate the constraints redirections from the deliverable data
Instead of statically listing the redirections move to a dynamic model.

We move the existing _extras/.htaccess to _templates/htaccess so we have
some control and safety of what goes in there.  Connect 'build-finished'
from _exts.deliverables.py to trigger generating the redirects.  Doing
so here avoids re-reading the data as deliverables.py ahas already done
that for us.

Change-Id: If6bd59fd478593a84ebcedc3a50af3720d620d3c
2019-02-28 12:50:37 +11:00
Doug Hellmann
63d7394a36 update to use openstack_governance library
Remove the governance code in this repository and use the new
openstack_governance library instead.

Depends-On: https://review.openstack.org/614605
Change-Id: Ia7ffff3945462f4b568b55287dfdf45fe73a35d9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-11-21 15:10:34 -05:00
Andreas Jaeger
c9152cc789 Switch back to publish-to-pypi
We updated publish-to-pypi so that it has the same content as
publish-to-pypi-python3 - and then replaced publish-to-pypi-python3 with
publish-to-pypi. Thus, publish-to-pypi does what publish-to-pypi-python3
has done previously.

Since all repos have been updated to use publish-to-pypi, check for that
one again.

Change-Id: I09ed2370a4d1f1bfb94cf73a951d0f1f6af36be6
Depends-On: https://review.openstack.org/615239
2018-11-02 19:41:36 +01:00
Sean McGinnis
32afc4160d Update tooling to support cycle-with-rc model
This updates commands and tools to work with the cycle-with-rc model.

Change-Id: I8df85df1c84ae6d8fb37a5206e155f84f8fac947
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2018-10-24 08:41:13 -05:00
Zuul
a467622d66 Merge "drop support for driverfixes branches" 2018-10-11 11:50:28 +00:00
Zuul
fcf6b31eaa Merge "stop supporting legacy release jobs" 2018-10-11 11:49:13 +00:00
Zuul
65c449c542 Merge "clarify debug logging for test fixtures" 2018-10-11 02:39:21 +00:00
Doug Hellmann
2c2e61b76f drop support for driverfixes branches
This change removes support for driver fixes branches. It will prevent
any releases from projects with those branches listed in the deliverable
file in any series, which should prevent us from re-creating branches
that we are trying to remove when we do releases.

Change-Id: Ied35b936af1345e893da1384762f30547601ae8f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-10-08 19:09:14 -04:00
Doug Hellmann
5fb6f55fd3 stop supporting legacy release jobs
Drop the legacy release jobs for python projects and require the use of
the new python3 job.

Change-Id: I3c6040983dd00f45c4322aa0fb3dd16fff6eef11
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-10-05 11:23:15 -04:00
Doug Hellmann
27da7e8486 clarify debug logging for test fixtures
Have the fixtures report more details about what they're doing so when
we see the processutils output we can tell why the commands are being run.

Change-Id: I17374484e3e7457cf836c2dae695f0ca5a46ea8e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-09-24 09:19:20 -04:00
Sean McGinnis
bb81a80fd0 Simplify ref checkout in validate cmd
When the current code needs the repo checked out to a specific commit it
calls clone_repo. The clone_repo call makes sure the repo has been
cloned locally, performs a fetch to get all tags and prunes any
references no longer on the remote, then finally checks out the
requested ref.

The validation logic starts with cloning all necessary repos and
performing these actions, so after that point, we should be able to just
checkout the ref that we need for the given check. This will greatly
improve the execution time of the validate job for repos that have had a
large number of releases.

Change-Id: Ie444c8ef8e91832c38ee69307382ddb5fb8e1b08
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2018-09-22 22:11:55 -05:00
Doug Hellmann
3013ead10c convert warning for missing repo to error
If a release does not include all of the repositories in the
deliverable report an error instead of a warning.

Change-Id: I7980b204995647fc77657123fa8d7bb565b55247
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-09-19 09:24:54 -04:00
Jeremy Stanley
b84749b4e2 Support StoryBoard project names in place of IDs
Now that storyboard-webclient no longer displays the project id
number in project URLs by default, it is harder for people to look
them up without referring to the API. Conversely, we could just use
the newer name-based project query support in the API instead. As a
transitional step, support both. Also switch openstack-infra/shade
from id to name to prove that the validate tool change is effective.

Change-Id: I9da97a1af40bb3527c1c7e8a66a267c76b9db564
2018-09-08 20:58:58 +00:00
Doug Hellmann
11bb8e04f4 require stable branches be made from latest release
At the end of rocky we had a couple of cases where a forced branch was
created from a release that was not the most current. This validation
change should prevent that from happening again in the future.

Change-Id: Ie8635b74ccbce988b5f4e53c0848daf50625b239
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-08-16 16:20:01 -04:00
Doug Hellmann
3b993850c1 add validation for pre-release version progression
Change-Id: I162baf43c1f782de314af95c525476f80f32a9bd
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-08-09 12:19:36 -04:00
Sean McGinnis
22f923d104
Add stein to docs landing page
Include stein in the index page list. This also adds support for
a "future" status since validation was add/changed since the
start of the last cycle.

Change-Id: Ie1fc997c1f5f0f3c6af1d048fcc4b25a4608e70f
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2018-07-30 12:26:35 -05:00
Thierry Carrez
eb4c7ac4ef new_release: go back in history and compute depth
make get_last_release() go further back in release history,
compute the depth at which it finds a release.

Properly increment feature version based on the depth at
which the last release was found.

Change-Id: I5880611a7d9005cee8fe4c2d7920f84d880eeeb8
Task: 22351
2018-07-04 18:40:25 +02:00
Thierry Carrez
e7ac6d6ab8 new_release: work from a release history object
The calculation of the last release was working from release
information from targeted series, and used specific code to
retrieve the data from the previous series if that failed.

In preparation for landing code to look further backwards in
history, refactor that code to retrieve and work from a
release history object containing list of releases for each
considered series.

To preserve current behavior, for the moment, we only retrieve
history for the targeted series and the series before that.

This allows to get rid of mocks in get_last_release() tests and
simplify testing from a history of releases.

Change-Id: I48ff9c46c1e9250fbdfc5dd5344f18c7e046735a
2018-07-04 18:15:18 +02:00
Doug Hellmann
c28884d766 add support for $series-em tags
Add validation support for $series-em tags, using the same rules as
$series-eol tags for now.

Story: #2001879
Task: #22613
Change-Id: I689fc8fee0ded41da202cc4e84cfed6a9daa9846
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-28 16:08:07 -04:00
Zuul
9339106a3c Merge "allow setting tarball-base once in the repository-settings" 2018-06-20 09:25:08 +00:00
Zuul
5e8c510828 Merge "bypass validation checks when tags exist" 2018-06-20 09:25:07 +00:00
Zuul
7079b1359e Merge "update validation to support eol tags" 2018-06-20 09:23:03 +00:00
Doug Hellmann
1d9c2f854d allow setting tarball-base once in the repository-settings
Rather than requiring tarball-base be set every time in the release,
allow it to be set in the repository-settings section. The release
value acts as an override.

Change-Id: I48e424b9977a2acbfa46221da80ad0d698de8e88
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-19 10:05:19 -04:00
Doug Hellmann
cdf80e1469 bypass validation checks when tags exist
Create a decorator we can use to bypass checks when tags exist and
apply it to the validation functions that make sense.

Change-Id: I91a26abd2934b6a65ae1c1c15d125fd183cd4a23
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-18 17:08:31 -04:00
Doug Hellmann
0c9e8b7f3b support reading zuul settings from within a repo
As part of the python3 goal for stein we are going to move the zuul
settings out of the project-config repository and into each project
tree. We need to update the validation to look at the files in the
repo being released in addition to project-config when we are looking
for the project-templates to verify the release jobs.

Change-Id: Ib64d59b1b6646d779112c9fcad47ae7d2c3c74d4
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-18 16:45:22 -04:00
Doug Hellmann
a97418e706 move test fixtures to their own file
This will make it easier to reuse the git repo fixture.

Change-Id: I8922029fcd60fa94dad73cb187982904289d9a3b
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-18 16:45:22 -04:00
Doug Hellmann
be69bc3581 update validation to support eol tags
Update the version string validation to allow a series-eol
tag. Require that the EOL tag for a deliverable match the series and
be applied to all repositories that are part of the deliverable.

Extend the Release data object to understand whether its tag is an EOL
tag and if so to return the series component from it as eol_series.

Story: #2001879
Task: #14344
Change-Id: I40b6822c942b559dc6f18a4b6be6abb5407c4d7c
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-14 11:01:45 -04:00
Doug Hellmann
ef1b0d6574 add was_forced property to Release data object
Change-Id: I5cfb9df09b598bb677b365fc1581eb96dd9b7233
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-08 17:35:28 -04:00
Doug Hellmann
9bb5fec5f8 change API to pass directory containing status file
Instead of passing the root directory, pass deliverables directory
like we do for the Deliverables class.

Change-Id: I3f6dfe8936ccbce62d0735c591694b8831604437
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-20 15:58:57 -04:00