openstack/releases
Code Issues Proposed changes

Update cycle process based on PTG review

Browse Source 
This updates our process guide to incorporate the updates and changes we
identified while going through all steps during the Denver PTG.

Change-Id: Ib91bea5f874880f37d4d0e90c111ae0b89efa9d4
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This commit is contained in:
Sean McGinnis 2019-05-06 13:31:24 -05:00
parent 4936cae262
commit 804df78394
No known key found for this signature in database
GPG Key ID: CE7EE4BFAF8D70C8
1 changed files with 160 additions and 139 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

299
doc/source/reference/process.rst
View File

@ -8,56 +8,63 @@ all of the steps related to preparing the release.
Before PTG (after closing previous release) Before PTG (after closing previous release)
=========================================== ===========================================
1. Set up the release schedule for the newly opened cycle by creating #. Set up the release schedule for the newly opened cycle by creating
the required pages in openstack/releases. the required pages in openstack/releases.
2. Update the link to the documentation on the newly opened cycle page #. Update the link to the documentation on the newly opened cycle page
to point to the right place on docs.openstack.org. to point to the right place on docs.openstack.org.
3. Create the $series-relmgt-plan and $series-relmgt-tracking #. Create the $series-relmgt-tracking etherpad using ``tools/list_weeks.py``.
etherpads. For example::
4. Use ``init-series`` to create stub deliverable files based on the tools/list_weeks.py t 2019-04-15 2019-10-16
#. Use ``init-series`` to create stub deliverable files based on the
contents of the previous release. contents of the previous release.
Between Summit and Milestone-1 Between Summit and Milestone-1
============================== ==============================
1. Establish liaisons by having them update #. Establish liaisons by having them update
https://wiki.openstack.org/wiki/CrossProjectLiaisons with their https://wiki.openstack.org/wiki/CrossProjectLiaisons with their
contact information. contact information.
2. Email PTLs directly one time to explain the use of the "[release]" #. Email PTLs directly one time to explain the use of the "[release][ptl]"
email tag on the openstack-discuss list. email tag on the openstack-discuss list.
3. Encourage liaisons to ensure that their release model is set #. Encourage liaisons to ensure that their release model is set
properly before the first milestone. properly before the first milestone.
4. Start weekly countdown emails, sent on Thursday afternoon (US) #. Start weekly countdown emails, sent right after the team meeting,
or Friday morning (EU/APAC) with information needed about the with information needed about the
following week (deadlines, instructions, etc.). following week (deadlines, instructions, etc.).
5. The week before Milestone-1, include a reminder about completing #. The week before Milestone-1, include a reminder about completing
the responses to community-wide goals in the countdown email. the responses to community-wide goals in the countdown email.
Milestone-1 Milestone-1
=========== ===========
1. Generate release requests for all cycle-with-intermediary libraries #. Generate release requests for all cycle-with-intermediary libraries
which had changes, but did not release since the previous release. which had changes, but did not release since the previous release.
That patch will be used as a base to communicate with the team: That patch will be used as a base to communicate with the team:
if a team wants to wait for a specific patch to make it to the library, if a team wants to wait for a specific patch to make it to the library,
someone from the team can -1 the patch to have it held, or update someone from the team can -1 the patch to have it held, or update
that patch with a different commit SHA. that patch with a different commit SHA.
2. Run tools/aclissues.py to detect potential leftovers in Gerrit ACLs #. To catch if there are acl issues in newly created repositories,
allowing official deliverables to directly tag or branch without run tools/aclissues.py to detect potential leftovers in Gerrit ACLs
allowing official deliverables to be directly tagged or branched without
going through openstack/releases. You need to specify the location going through openstack/releases. You need to specify the location
of up-to-date checkouts for the governance and the project-config of up-to-date checkouts for the governance and the project-config
repositories. For example:: repositories. For example::
tools/aclissues.py ../project-config ../governance tools/aclissues.py ../project-config ../governance
If the tool reports any violation, you can re-run it with ``--patch`` to
generate needed changes in ../project-config to align ACLs with governance,
and propose the changes for review.
Between Milestone-1 and Milestone-2 Between Milestone-1 and Milestone-2
=================================== ===================================
@ -76,41 +83,54 @@ Between Milestone-1 and Milestone-2
#. Mention the upcoming MembershipFreeze deadline in the countdown emails. #. Mention the upcoming MembershipFreeze deadline in the countdown emails.
#. Ahead of MembershipFreeze, run membership_freeze_test to check for #. Ahead of MembershipFreeze, run ``membership_freeze_test`` to check for
any new deliverable in governance that has not been released yet:: any new deliverable in governance that has not been released yet::
tox -e membership_freeze_test -- $series ~/branches/governance/reference/projects.yaml tox -e membership_freeze_test -- $series ~/branches/governance/reference/projects.yaml
Those should either get a release management exception (see Those should either be tagged as a release management exception if they do
release-management key in the governance projects.yaml file) or an not need to be released (see ``release-management`` key in the governance
empty deliverable file should be added to the series so that we can projects.yaml file) or an empty deliverable file should be added to the
properly track it. Leftovers are considered too young to be released series so that we can properly track it. Leftovers are considered too young
in the next release and will be reconsidered at the next cycle. to be released in the next release and will be reconsidered at the next
cycle.
Milestone-2 Milestone-2
=========== ===========
1. Generate release requests for all cycle-with-intermediary libraries #. Generate release requests for all cycle-with-intermediary libraries
which had changes, but did not release since milestone-1. which had changes, but did not release since milestone-1.
That patch will be used as a base to communicate with the team: That patch will be used as a base to communicate with the team:
if a team wants to wait for a specific patch to make it to the library, if a team wants to wait for a specific patch to make it to the library,
someone from the team can -1 the patch to have it held, or update someone from the team can -1 the patch to have it held, or update
that patch with a different commit SHA. that patch with a different commit SHA.
2. Run tools/aclissues.py to detect potential leftovers in Gerrit ACLs #. To catch if there are acl issues in newly created repositories,
allowing official deliverables to directly tag or branch without run ``tools/aclissues.py`` to detect potential leftovers in Gerrit ACLs
allowing official deliverables to be directly tagged or branched without
going through openstack/releases. You need to specify the location going through openstack/releases. You need to specify the location
of up-to-date checkouts for the governance and the project-config of up-to-date checkouts for the governance and the project-config
repositories. For example:: repositories. For example::
tools/aclissues.py ../project-config ../governance tools/aclissues.py ../project-config ../governance
If the tool reports any violation, you can re-run it with ``--patch`` to
generate needed changes in ../project-config to align ACLs with governance,
and propose the changes for review.
Between Milestone-2 and Milestone-3 Between Milestone-2 and Milestone-3
=================================== ===================================
#. In the countdown email immediately after Milestone-2, include a #. In the countdown email immediately after Milestone-2, include a
reminder about the various freezes that happen around Milestone-3. reminder about the various freezes that happen around Milestone-3.
Remind PTLs a heads up to start thinking about what they might want to
include in their deliverables file as cycle-highlights
and that feature freeze is the deadline for them.
#. Check with the election team about whether the countdown email
needs to include any updates about the election schedule.
#. For intermediary-released service projects that have not done a #. For intermediary-released service projects that have not done a
release by milestone-2, propose a change from cycle-with-intermediary release by milestone-2, propose a change from cycle-with-intermediary
to cycle-with-rc. Engage with PTLs and release liaisons to either to cycle-with-rc. Engage with PTLs and release liaisons to either
@ -119,26 +139,31 @@ Between Milestone-2 and Milestone-3
#. Two weeks before Milestone-3, include a reminder about the final #. Two weeks before Milestone-3, include a reminder about the final
library release freeze coming the week before Milestone-3. library release freeze coming the week before Milestone-3.
1. Run the command from milestone-2 again to get a list of libraries:: #. Run the command from milestone-2 again to get a list of libraries::
tools/list_library_unreleased_changes.sh tools/list_library_unreleased_changes.sh
2. Include list of unreleased libraries in the email to increase visibility. #. Include list of unreleased libraries in the email to increase visibility.
#. Two weeks before Milestone-3, prepare other teams to the final release #. Two weeks before Milestone-3, prepare other teams to the final release
rush. rush.
1. Ask the release liaisons for the affected teams to update the #. Ask the release liaisons for the affected teams to audit the
contents of their $project-stable-maint groups, as that group contents of their ``$project-stable-maint`` groups, as that group
will control the stable/$series branch prior to release. They will control the ``stable/$series`` branch prior to release. They
should reach out to the stable-maint-core group for additions. should reach out to the ``stable-maint-core`` group for additions.
2. Notify the Infrastructure team to `generate an artifact signing key`_ #. Include a reminder about the stable branch ACLs in the countdown email.
#. Notify the Infrastructure team to `generate an artifact signing key`_
(but not replace the current one yet), and (but not replace the current one yet), and
begin the attestation process. begin the attestation process.
.. _generate an artifact signing key: https://docs.openstack.org/infra/system-config/signing.html#generation .. _generate an artifact signing key: https://docs.openstack.org/infra/system-config/signing.html#generation
#. Include a reminder in the weekly countdown email reminding PTLs of the
feature freeze deadline for cycle-highlights.
Final Library Release (week before Milestone-3) Final Library Release (week before Milestone-3)
=============================================== ===============================================
@ -149,27 +174,25 @@ Final Library Release (week before Milestone-3)
library, someone from the team can -1 the patch to have it held, or update library, someone from the team can -1 the patch to have it held, or update
that patch with a different commit SHA. that patch with a different commit SHA.
#. Release libraries as quickly as possible this week to ensure they .. note::
are all done before the freeze. Consider relaxing the "not on
Friday" release rule if absolutely necessary.
#. Remind liaisons to prepare releases for client libraries at At this point, we want *all* changes in the deliverables, to ensure
Milestone-3. that we have CI configuration up to date when the stable branch
is created later.
#. Release libraries as quickly as possible this week to ensure they
are all done before the freeze.
#. Update the feature list and allowed stable branch names in #. Update the feature list and allowed stable branch names in
devstack-gate for the new stable branch. For devstack-gate for the new stable branch. For
example, https://review.openstack.org/362435 and example, https://review.openstack.org/362435 and
https://review.openstack.org/363084 https://review.openstack.org/363084
#. Allow the stable/$series branch to be requested with each library final #. Allow the ``stable/$series`` branch to be requested with each library final
release if they know they are ready. Do not require branching at this point release if they know they are ready. Do not require branching at this point
in case of critical issues requiring another approved release past the in case of critical issues requiring another approved release past the
freeze date. freeze date.
#. For stable libraries that did not have any change merged over the
cycle, create a stable branch from the last available release.
Milestone-3 Milestone-3
=========== ===========
@ -179,11 +202,22 @@ Milestone-3
to make it to the library, someone from the team can -1 the patch to have to make it to the library, someone from the team can -1 the patch to have
it held, or update that patch with a different commit SHA. it held, or update that patch with a different commit SHA.
#. Freeze changes to ``openstack/requirements`` by applying -2 to all #. Evaluate any libraries that did not have any change merged over the
open patches. Ensure that reviewers do not approve changes created cycle to see if it is time to `transition them to the independent release
by the proposal bot. model <https://releases.openstack.org/reference/release_models.html#openstack-related-libraries>`__.
#. Allow the stable/$series branch to be requested with each client library If it is OK to transition them, move the deliverable file to the ``_independent`` directory.
If it is not OK to transition them, create a new stable branch from the latest release
from the previous series.
#. Remind the requirements team to freeze changes to
``openstack/requirements`` by applying -2 to all
open patches. Ensure that reviewers do not approve changes created
by the proposal bot, but do approve changes for new OpenStack deliverable
releases.
#. Allow the ``stable/$series`` branch to be requested with each client library
final release if they know they are ready. Do not require branching at this final release if they know they are ready. Do not require branching at this
point in case of critical issues requiring another approved release past the point in case of critical issues requiring another approved release past the
freeze date. freeze date.
@ -191,6 +225,9 @@ Milestone-3
#. Remind PTLs/liaisons that master should be frozen except for bug #. Remind PTLs/liaisons that master should be frozen except for bug
fixes and feature work with FFEs. fixes and feature work with FFEs.
#. Email openstack-discuss list to remind PTLs that cycle-highlights are due
this week so that they can be included in release marketing preparations.
#. Remind PTL/liaisons to start preparing "prelude" release notes as #. Remind PTL/liaisons to start preparing "prelude" release notes as
summaries of the content of the release so that those are merged summaries of the content of the release so that those are merged
before their first release candidate. before their first release candidate.
@ -200,114 +237,92 @@ Milestone-3
constraint or requirement changes will be held until after the freeze constraint or requirement changes will be held until after the freeze
period. period.
#. Include a reminder about completing the responses to community-wide .. note::
goals in the countdown email.
#. Run tools/aclissues.py to detect potential leftovers in Gerrit ACLs Do not release libraries without a link to a message to openstack-discuss
allowing official deliverables to directly tag or branch without requesting a requirements FFE and an approval response from that team.
going through openstack/releases. You need to specify the location
of up-to-date checkouts for the governance and the project-config
repositories. For example::
tools/aclissues.py ../project-config ../governance
#. Email openstack-discuss to give PTLs a heads up to start thinking about
what they might want to include in their deliverables file as cycle-highlights
and that RC1 is the deadline for them.
Between Milestone-3 and RC1 Between Milestone-3 and RC1
=========================== ===========================
#. Warn cycle-with-intermediary projects that have releases more than #. Warn cycle-with-intermediary services that have releases more than
2 months old that we will use their existing release as a point for 2 months old that we will use their existing release as a point for
branching if they have not prepared a newer release by the final RC branching if they have not prepared a newer release by the final RC
deadline. deadline.
#. Propose stable/$series branch creation for all client and non-client #. Propose ``stable/$series`` branch creation for all client and non-client
libraries that had not requested it at freeze time. The following command libraries that had not requested it at freeze time. The following command
may be used:: may be used::
tox -e venv -- propose-library-branches --include-clients tox -e venv -- propose-library-branches --include-clients
#. Include a reminder in the weekly countdown email reminding PTLs of the
RC1 deadline for cycle-highlights.
RC1 week RC1 week
======== ========
#. Early in the week, generate RC1 release requests (including the #. Early in the week, generate RC1 release requests (including the
stable/$series branch creation) for all cycle-with-rc deliverables. ``stable/$series`` branch creation) for all cycle-with-rc deliverables.
That patch will be used as a base to communicate with the team: That patch will be used as a base to communicate with the team:
if a team wants to wait for a specific patch to make it to the RC, if a team wants to wait for a specific patch to make it to the RC,
someone from the team can -1 the patch to have it held, or update someone from the team can -1 the patch to have it held, or update
that patch with a different commit SHA. that patch with a different commit SHA.
#. Email openstack-discuss list to remind PTLs that cycle-highlights are due
this week so that they can be included in release marketing preparations.
#. By the end of the week, ideally we would want a +1 from the PTL and/or #. By the end of the week, ideally we would want a +1 from the PTL and/or
release liaison to indicate approval. However we will consider the absence release liaison to indicate approval. However we will consider the absence
of -1 or otherwise negative feedback as an indicator that the automatically of -1 or otherwise negative feedback as an indicator that the automatically
proposed patches can be approved at the end of the RC deadline week. proposed patches can be approved at the end of the RC deadline week.
#. After the minimum set of projects used by devstack have been branched, the #. After all the projects enabled in devstack by default have been branched,
devstack branch can be created. Devstack doesn't push a tag at RC1 it is remind the QA PTL to create a branch in the devstack repository. Devstack
just branched off of HEAD doesn't push a tag at RC1 it is just branched off of HEAD.
#. After devstack is branched a grenade branch can be created. As with #. After devstack is branched, remind the QA PTL to create a branch in the
devstack it will branch from HEAD instead of a tag. grenade repository. As with devstack, it will branch from HEAD instead of a
tag.
#. Update the default branch for devstack in the new stable #. Remind the QA PTL to update the default branch for devstack in the new
branch. For example, https://review.openstack.org/#/c/493208/ stable branch. For example, https://review.opendev.org/#/c/493208/
#. Update the grenade settings in devstack-gate for the new branch. For #. Remind the QA PTL to update the grenade settings in devstack-gate for the
example, https://review.openstack.org/362438. new branch. For example, https://review.opendev.org/362438.
.. note:: .. note::
As soon as grenade is updated for the new branch (see the RC1 As soon as grenade is updated for the new branch (see the RC1
instructions that follow), projects without stable branches may instructions that follow), projects without stable branches may
start seeing issues with their grenade jobs because without the start seeing issues with their grenade jobs because without the
stable branch the branch selection will cause the jobs to run stable branch the branch selection will cause the jobs to run
master->master instead of previous->master. At the end of Ocata master->master instead of previous->master. At the end of Ocata
this caused trouble for the Ironic team, for example. this caused trouble for the Ironic team, for example.
#. For translations, create stable-$series versions in the Zanata #. Remind the I18n PTL to update the translation tools for the new stable
translation server on https://translate.openstack.org for all series.
projects that the translation team wants to handle. Create new
translation-jobs-$series periodic jobs to import translations from
the Zanata translation server and propose them to projects, add
these jobs to all projects that have a stable-$series version.
Note this work is done by translation team. #. After all cycle-with-rc projects have their branches created, remind the
requirements PTL to propose an update to the deliverable file to create the
#. After all cycle-with-rc projects have their branches ``stable/$series`` branch for ``openstack/requirements``. Then announce that
created, someone from the requirements core team (preferably the the requirements freeze is lifted from master.
requirements PTL) needs to propose an update the deliverable file to
create the stable/$series branch for ``openstack/requirements``.
Then announce that the requirements freeze is lifted from master.
.. note:: .. note::
We wait until after the other projects have branched to We wait until after the other projects have branched to
create the branch for requirements because tests for the stable create the branch for requirements because tests for the stable
branches of those projects will fall back to using the master branches of those projects will fall back to using the master
branch of requirements until the same stable branch is created, branch of requirements until the same stable branch is created,
but if the branch for the requirements repo exists early the but if the branch for the requirements repo exists early the
changes happening in master on the other projects will not use it changes happening in master on the other projects will not use it
and we can have divergence between the requirements being tested and we can have divergence between the requirements being tested
and being declared as correct. and being declared as correct.
#. In the tempest repo, create new branch specific jobs for our two branchless #. Remind the QA PTL to create new branch specific jobs for our two branchless
projects, devstack-gate and tempest. Configure tempest to run them on all projects, devstack-gate and tempest, in the tempest repo. Configure tempest
changes, voting. Configure tempest to run them as periodic bitrot jobs as to run them on all changes, voting. Configure tempest to run them as
well. All this can be done in one tempest patch, like for example, see periodic bitrot jobs as well. All this can be done in one tempest patch,
https://review.openstack.org/521888. for example, see https://review.openstack.org/521888.
Configure devstack-gate to run the new jobs in check pipeline only, Configure devstack-gate to run the new jobs in check pipeline only,
non-voting, for example see https://review.openstack.org/545144. non-voting, for example see https://review.openstack.org/545144.
#. Add the new branch to the list of branches in the periodic-stable job #. Remind the QA PTL to add the new branch to the list of branches in the
templates in openstack-zuul-jobs. For example, see periodic-stable job templates in openstack-zuul-jobs. For example, see
https://review.openstack.org/545268/. https://review.openstack.org/545268/.
Between RC1 and Final Between RC1 and Final
@ -324,14 +339,14 @@ Between RC1 and Final
.. note:: .. note::
Try to avoid creating more than 3 release candidates so we are not Try to avoid creating more than 3 release candidates so we are not
creating candidates that consumers are then trained to ignore. Each creating candidates that consumers are then trained to ignore. Each
release candidate should be kept for at least 1 day, so if there is a release candidate should be kept for at least 1 day, so if there is a
proposal to create RCx but clearly a reason to create another one, proposal to create RCx but clearly a reason to create another one,
delay RCX to include the additional patches. Teams that know they will delay RCX to include the additional patches. Teams that know they will
need additional release candidates can submit the requests and mark need additional release candidates can submit the requests and mark
them WIP until actually ready, so the release team knows that more them WIP until actually ready, so the release team knows that more
candidates are coming. candidates are coming.
#. Ensure that all projects that are publishing release notes have the #. Ensure that all projects that are publishing release notes have the
notes link included in their deliverable file. See notes link included in their deliverable file. See
@ -367,7 +382,7 @@ Between RC1 and Final
is included in the metadata that goes onto the signed tag. is included in the metadata that goes onto the signed tag.
#. The week before final release test the release process using the #. The week before final release test the release process using the
openstack/release-test repository. ``openstack/release-test`` repository to ensure our machinery is functional.
#. Notify the documentation team that it should be safe to apply #. Notify the documentation team that it should be safe to apply
their process to create the new release series landing pages for their process to create the new release series landing pages for
@ -379,29 +394,47 @@ Between RC1 and Final
Final Release Final Release
============= =============
1. Approve the final release patch created earlier. #. Approve the final release patch created earlier.
2. Run the missing-releases script to check for missing tarballs on the .. note::
This needs to happen several hours before the press release
from the foundation (to give us time to handle failures) but not
too far in advance (to avoid releasing the day before the press
release).
#. Run the ``missing-releases`` script to check for missing tarballs on the
release page before the announcement:: release page before the announcement::
tox -e venv -- missing-releases --series $SERIES tox -e venv -- missing-releases --series $SERIES
3. Mark series as released on releases.o.o, by updating doc/source/index.rst If there are any missing deliverables, fix them.
#. Mark series as released on releases.o.o, by updating doc/source/index.rst
and doc/source/$series/index.rst. and doc/source/$series/index.rst.
See https://review.openstack.org/#/c/381006 for an example. See https://review.openstack.org/#/c/381006 for an example.
4. Update the default series name in .. note::
This item can be staged as a patch on top of the final release patch.
#. Update the default series name in
``openstack/releases/openstack_releases/defaults.py`` to use the ``openstack/releases/openstack_releases/defaults.py`` to use the
new series name. new series name.
5. Send release announcement email to .. note::
This item can be staged as a patch on top of the previous patch.
Only workflow when previous step *fully* ready
#. Send release announcement email to
``openstack-announce@lists.openstack.org``, based on ``openstack-announce@lists.openstack.org``, based on
``templates/final.txt``. Coordinate the timing of the email with ``templates/final.txt``. Coordinate the timing of the email with
the press release from the Foundation staff. the press release from the Foundation staff.
6. Send an email to the openstack-discuss list to point to the official #. Send an email to the openstack-discuss list to point to the official
release announcement, and declare ``openstack/releases`` unfrozen for release announcement from the previous step, and declare
releases on the new series. ``openstack/releases`` unfrozen for releases on the new series.
Post-Final Release Post-Final Release
================== ==================
@ -417,17 +450,5 @@ Post-Final Release
tox -e venv -- init-series $SERIES $NEXT_SERIES tox -e venv -- init-series $SERIES $NEXT_SERIES
cycle-trailing Final Release #. Remind PTLs of cycle-trailing projects to prepare their releases.
============================
#. A week before the cycle-trailing deadline, use
``propose-final-releases --all`` to tag the existing most recent release
candidates as the final release for the cycle-trailing projects.
#. Ask liaisons and PTLs of cycle-trailing projects to review and +1
the final release proposal from the previous step so their approval
is included in the metadata that goes onto the signed tag.
#. On the cycle-trailing deadline approve the final release patch created
earlier.