=================== Release Processes =================== This document describes the relative ordering and rough timeline for all of the steps related to preparing the release. Before Summit (after closing previous release) ============================================= 1. Set up the release schedule for the newly opened cycle by creating the required pages in openstack/releases. 2. Create the $series-relmgt-plan and $series-relmgt-tracking etherpads. 3. Use ``init-series`` to create stub deliverable files based on the contents of the previous release. Between Summit and Milestone-1 ============================== 1. Establish liaisons by having them update https://wiki.openstack.org/wiki/CrossProjectLiaisons with their contact information. 2. Email PTLs directly one time to explain the use of the "[release]" email tag on the openstack-dev list. 3. Encourage liaisons to ensure that their release model is set properly before the first milestone. 4. Start weekly countdown emails, sent on Thursday afternoon (US) or Friday morning (EU/APAC) with information needed about the following week (deadlines, instructions, etc.). 5. The week before Milestone-1, include a reminder about completing the responses to community-wide goals in the countdown email. Milestone-1 =========== 1. Include the deadline for the milestone in the countdown emails but do not send extra reminders to liaisons. Missing the first milestone isn't important in terms of the release, so we want to encourage everyone to pay attention on their own. Between Milestone-1 and Milestone-2 =================================== 1. Follow up with PTLs and liaisons for projects that missed the first milestone. 2. Use the countdown emails to list which projects have not done any stable release yet, to encourage them to do so. 3. Use the countdown emails to list which intermediary-released (or independent) projects haven't done a release yet. Remind teams that we need at least one library release before milestone-2. 4. The week before Milestone-2 include a reminder of the deadline in the countdown emails. Mention the MembershipFreeze deadline as well. List teams that still haven't done a library release yet and remind them of the milestone-2 deadline. Milestone-2 =========== n/a Between Milestone-2 and Milestone-3 =================================== 1. Follow up with PTLs and liaisons for projects that missed the second milestone, or still haven't done their library releases yet. 2. In the countdown email immediately after Milestone-2, include a reminder about the various freezes that happen around Milestone-3. 3. Two weeks before Milestone-3, include a reminder about the final library release freeze coming the week before Milestone-3. 4. Two weeks before Milestone-3, set up the gerrit ACLs for the yet-to-be-created stable/$series branches. (The timing for this is meant to postpone it until all projects that will be included in the current release are accepted but not put it off for so long that we have to rush to merge the changes in order to avoid having the release blocked.) 1. Update ACLs for refs/heads/stable/$series so that members of $project-release-branch can approve changes. The patch can be generated (for all release:cycle-with-milestones deliverables) with:: tox -e aclmanager -- acls /path/to/project-config Double-check that all references to the previous stable/$old are all removed from project-config gerrit ACLs, as some may have changed release models and kept stale entries:: grep stable/$old /path/to/project-config/gerrit/acls/openstack/* 2. Once the above patch merges (and only then!), set the population of all $project-release-branch groups to the "Release Managers" group and $project-release. This can be done (for all release:cycle-with-milestones deliverables) by running ``aclmanager.py``:: tox -e aclmanager -- groups pre_release $user ($user being your Gerrit username). It will prompt you for your Gerrit HTTP password, which you can see/generate at https://review.openstack.org/#/settings/http-password. 3. Ask the release liaisons for the affected teams to update the contents of their $project-release groups. For new projects in some cases they will need to get the group created by Infra. 4. Notify the Infrastructure team to `generate master instead of previous->master. At the end of Ocata this caused trouble for the Ironic team, for example. RC1 === 1. Ensure all RC1 tag requests include the info to have the stable/$series branch created, too. Branches for cycle-trailing and cycle-with-intermediary projects should be created when the PTL/liaison are ready, and not necessarily for RC1 week. 2. After the minimum set of projects used by devstack have been branched, the devstack branch can be created. Devstack doesn't push a tag at RC1 it is just branched off of HEAD 3. After devstack is branched a grenade branch can be created. As with devstack it will branch from HEAD instead of a tag. 4. Update the default branch for devstack in the new stable branch. For example, https://review.openstack.org/#/c/493208/ 5. Update the grenade settings in devstack-gate for the new branch. For example, https://review.openstack.org/362438. 6. For translations, create stable-$series versions in the Zanata translation server on https://translate.openstack.org for all projects that the translation team wants to handle. Create new translation-jobs-$series periodic jobs to import translations from the Zanata translation server and propose them to projects, add these jobs to all projects that have a stable-$series version. Note this work is done by translation team. 7. After all cycle-with-milestone projects have their branches created, update the deliverable file to create the stable/$series branch for ``openstack/requirements``. Then announce that the requirements freeze is lifted from master. Note that we wait until after the other projects have branched to create the branch for requirements because tests for the stable branches of those projects will fall back to using the master branch of requirements until the same stable branch is created, but if the branch for the requirements repo exists early the changes happening in master on the other projects will not use it and we can have divergence between the requirements being tested and being declared as correct. 8. Create new branch specific jobs for our two branchless projects, devstack-gate and tempest, and configure Zuul to run them on all changes to those projects to protect against regressions with the stable branches and these tools. For example, see https://review.openstack.org/375110. 9. Create periodic bitrot jobs for the new branch in Jenkins Job Builder and add them to Zuul's periodic pipeline. For example, see https://review.openstack.org/#/c/375092. 10. Add periodic bitrot jobs to tempest. For example, see https://review.openstack.org/#/c/375271. Between RC1 and Final ===================== Try to avoid creating more than 3 release candidates so we are not creating candidates that consumers are then trained to ignore. Each release candidate should be kept for at least 1 day, so if there is a proposal to create RCX but clearly a reason to create another one, delay RCX to include the additional patches. Teams that know they will need additional release candidates can submit the requests and mark them WIP until actually ready, so the release team knows that more candidates are coming. 1. Ensure that all projects that are publishing release notes have the notes link included in their deliverable file. See ``tools/add_release_note_links.sh``. 2. Encourage liaisons to merge all translation patches. 3. When all translations and bug fixes are merged for a project, prepare a new release candidate. 4. Ensure that the final release candidate for each project is prepared at least one week before the final release date. 5. After final releases for release:cycle-with-intermediary projects are tagged, create their stable branches. 6. On the morning of the deadline for final release candidates, check the list of unreleased changes for milestone projects and verify with the PTLs and liaisons that they are planning a release or that they do not need one. In the release-tools repository working directory, run:: $ ./list_unreleased_changes.sh stable/newton $(list-repos --tag release:cycle-with-milestones) 2>&1 | tee unreleased.log 7. As soon as the last release candidate is tagged and the freeze period is entered, use ``propose-final-releases`` to tag the existing most recent release candidates as the final release for projects using the cycle-with-milestone model. 8. Ask liaisons and PTLs of milestone-based projects to review and +1 the final release proposal from the previous step so their approval is included in the metadata that goes onto the signed tag. 9. The week before final release test the release process using the openstack/release-test repository. 10. Notify the documentation team that it should be safe to apply their process to create the new release series landing pages for docs.openstack.org. Their process works better if they wait until most of the projects have their stable branches created, but they can do the work before the final release date to avoid having to synchronize with the release team on that day. Final Release ============= 1. Approve the final release patch created earlier. 2. Reset gerrit ACLs 1. Update all of the $project-release-branch groups to have $project-stable-maint as members instead of "Release Managers" and $project-release. This can be done (for all release:cycle-with-milestones deliverables) by running:: tox -e aclmanager -- groups post_release $user ($user being your Gerrit username) 2. Remove the refs/heads/stable/$series from the project gerrit ACLs. This can be done by reverting the original ACL patch. 3. Add documentation links on the series page on releases.o.o. See https://review.openstack.org/#/c/381005 for an example. 4. Mark series as released on releases.o.o, by updating doc/source/index.rst and doc/source/$series/index.rst. See https://review.openstack.org/#/c/381006 for an example. 5. Update the default series name in ``openstack/releases/openstack_releases/defaults.py`` to use the new series name. 6. Send release announcement email to ``openstack-announce@lists.openstack.org``, based on ``templates/final.txt``. Coordinate the timing of the email with the press release from the Foundation staff. 7. Declare ``openstack/releases`` unfrozen. Post-Final Release ================== 1. The week after the final release, process any late or blocked release requests for deliverables for any branch (treating the new series branch as stable). 2. The week after the final releases for milestone-based projects are tagged, use ``propose-final-releases --all`` to tag the existing most recent release candidates as the final release for projects using the cycle-trailing model. 3. Ask liaisons and PTLs of cycle-trailing projects to review and +1 the final release proposal from the previous step so their approval is included in the metadata that goes onto the signed tag. cycle-trailing Final Release ============================ 1. Two weeks after the final release for milestone-based projects, approve the final release patch created earlier. 2. Reset gerrit ACLs 1. Update all of the $project-release-branch for cycle-trailing groups to have $project-stable-maint as members instead of "Release Managers" and $project-release. This can be done (for all release:cycle-with-milestones deliverables) by running ``aclmanager.py groups post_release $user`` ($user being your Gerrit username) 2. Remove the refs/heads/stable/$series from the project gerrit ACLs. This can be done by reverting the original ACL patch.