stable: document the 'proactive' backporting workflow
Neutron team follows the process for a while (since Liberty release). I hear more projects are interested in adopting the process. Let's document it in project team guide so that different teams stay consistent in how they approach it, both tools as well as process wise. Change-Id: I7d427d56e64d57af675faeff221fe4148867e6e9
This commit is contained in:
Ihar Hrachyshka
parent
e27bfc3ce5
commit
c5da030c60
@ -324,6 +324,183 @@ crop up, the bot will email the `openstack-stable-maint mailing list`_. It is
|
|||||||
best to react quickly to these and get them resolved ASAP to prevent them from
|
best to react quickly to these and get them resolved ASAP to prevent them from
|
||||||
piling up. Please subscribe if you're interested in helping out.
|
piling up. Please subscribe if you're interested in helping out.
|
||||||
|
|
||||||
|
Proactive backports
|
||||||
|
===================
|
||||||
|
|
||||||
|
To make sure suitable bug fixes that land in *master* branches are delivered to
|
||||||
|
stable branch consumers in timely manner, and to avoid situations when a high
|
||||||
|
impact bug fix falls through the cracks and does not get quickly provisioned to
|
||||||
|
users, projects may adopt a 'proactive' approach towards tracking patches that
|
||||||
|
are candidates for backports, as described below.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
The first project that adopted the described approach is Neutron. Other
|
||||||
|
projects are welcome to experiment with similar practices and provide
|
||||||
|
feedback and improvements.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Tools mentioned in the guidelines below are currently maintained in
|
||||||
|
`openstack-infra/release-tools repository`_. Most of them are implemented as
|
||||||
|
Unix filters that can be interconnected into a pipeline to accommodate for
|
||||||
|
specific project needs and practices.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Guidelines below assume that there is a group of people behind the effort
|
||||||
|
that are willing to help. Tips on how to build the subteam are out of scope
|
||||||
|
for the document.
|
||||||
|
|
||||||
|
From high level perspective, proactive backporting process consists of the following steps:
|
||||||
|
|
||||||
|
#. identify bugs fixed since the previous triage event;
|
||||||
|
#. of those, pick only those bugs that does not break stable policy policies;
|
||||||
|
#. distribute identified backport candidates among subteam members;
|
||||||
|
#. subteam members consider each candidate bug for inclusion into *stable* and
|
||||||
|
*oldstable* branches; if applicable, backports are proposed for review and
|
||||||
|
tracked until inclusion into appropriate branches;
|
||||||
|
#. new stable releases are created in due time.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
Most of those steps require human intervension (with the prominent
|
||||||
|
exception of the first step) because triaging requires specific judgement.
|
||||||
|
New release proposals can be automated, but at the moment, this is left out
|
||||||
|
of scope for this document.
|
||||||
|
|
||||||
|
This is ongoing process, and it's usually executed on weekly basis, or with
|
||||||
|
other frequency that fits better the subteam and the project in question.
|
||||||
|
|
||||||
|
Now, let's cover each step with more details.
|
||||||
|
|
||||||
|
Identify new bug fixes
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
The process assumes that the subteam keeps track of the last git hash that was
|
||||||
|
validated somewhere. For the initial candidate list generation, it's advised to
|
||||||
|
start on a branch boundary (the latest common git commit between *stable* and
|
||||||
|
*master* branches).
|
||||||
|
|
||||||
|
For every new git commit found in *master* branch, commit message is checked
|
||||||
|
for bug tags (Closes-Bug, Partial-Bug, Related-Bug, ...) All bugs mentioned are
|
||||||
|
considered for initial filtering.
|
||||||
|
|
||||||
|
For this exact need, use the following release tool:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ ./bugs-fixed-since.py --repo ../neutron --start 1ce8ce9546479c0ce6055c0c205a8885699e3051
|
||||||
|
1514424
|
||||||
|
1560464
|
||||||
|
1546110
|
||||||
|
...
|
||||||
|
|
||||||
|
Filter out features and enhancements
|
||||||
|
------------------------------------
|
||||||
|
|
||||||
|
Due to stable policy described above, new features and enhancements are
|
||||||
|
generally not allowed in stable branches. For example, to filter out bugs that
|
||||||
|
have importance set to Wishlist in Launchpad, you can use the following tool:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ ./bugs-fixed-since.py [...] | lp-filter-bugs-by-importance.py neutron --importance Wishlist
|
||||||
|
1514424
|
||||||
|
1560464
|
||||||
|
1546110
|
||||||
|
...
|
||||||
|
|
||||||
|
The resulting list is expected to contain only actual bug fixes.
|
||||||
|
|
||||||
|
In case you also want to filter out bugs of Low importance, append another call to the tool:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ [...] | lp-filter-bugs-by-importance.py neutron --importance Low
|
||||||
|
1514424
|
||||||
|
1560464
|
||||||
|
1546110
|
||||||
|
...
|
||||||
|
|
||||||
|
Once you are satisfied with the query result, you should remember the latest
|
||||||
|
commit checked, and also store the bug list somewhere.
|
||||||
|
|
||||||
|
To achieve the latter, multiple directions can be taken.
|
||||||
|
|
||||||
|
#. One way is to store it in some external tool like Etherpad. If this
|
||||||
|
direction is chosen, the following tool may become handy to make the list
|
||||||
|
more consumable:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ [...] | ./annotate-lp-bugs.py neutron
|
||||||
|
https://bugs.launchpad.net/bugs/1514424 "neutron metadata ns proxy does not support ssl" (Medium,Fix Released) [in-stable-mitaka,l3-ipam-dhcp] [vuntz]
|
||||||
|
https://bugs.launchpad.net/bugs/1560464 "ovsdb monitor doesn't return ofport" (High,Fix Released) [in-stable-liberty,in-stable-mitaka] [kevinbenton]
|
||||||
|
https://bugs.launchpad.net/bugs/1546110 "DB error causes router rescheduling loop to fail" (Medium,Fix Released) [in-stable-kilo,in-stable-liberty,in-stable-mitaka,l3-ipam-dhcp,liberty-backport-potential] [brian-haley]
|
||||||
|
...
|
||||||
|
|
||||||
|
#. Another alternative is to tag backport candidates in Launchpad. For that,
|
||||||
|
it's advised to avoid using *$release-backport-potential* tags, and instead
|
||||||
|
introduce a new tag per project team (f.e. *neutron-proactive-backport-potential*
|
||||||
|
for Neutron). This is to avoid conflicts in the tag usage by multiple teams
|
||||||
|
running independent backporting processes when bug fixes spanning multiple
|
||||||
|
projects are considered.
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ [...] | ./lp-tag.py neutron-proactive-backport-potential
|
||||||
|
|
||||||
|
This command will tag all identified backport candidates with the project
|
||||||
|
specific tag. For example, check the `neutron proactive-backport-potential
|
||||||
|
dashboard`_.
|
||||||
|
|
||||||
|
With that, you get access to all filtering features available in Launchpad.
|
||||||
|
|
||||||
|
Distribute the work
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Once you have a list of candidate bug fixes to consider for backporting, it's
|
||||||
|
time to distribute it among subteam members. Depending on which method is
|
||||||
|
chosen above to track candidate bug fixes, you may utilize Launchpad search
|
||||||
|
queries, or other filtering technique to identify bugs of specific topics of
|
||||||
|
interest, to distribute the work to folks who are experts in those topics.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Exact search queries and filters are project specific and largely depend on
|
||||||
|
existing bug tracking practices adopted by projects. Hence they are out of
|
||||||
|
scope for the document.
|
||||||
|
|
||||||
|
Candidate triage
|
||||||
|
----------------
|
||||||
|
|
||||||
|
Each candidate bug should be assessed on its applicability to *stable* and
|
||||||
|
*oldstable* branches, as per corresponding support phase definitions. For
|
||||||
|
example, for *stable* branch, all bug fixes of user value can be considered to
|
||||||
|
backport; while for *oldstable* branch, only critical bugs are allowed to be
|
||||||
|
backported.
|
||||||
|
|
||||||
|
For every applicable stable branch, a backport is proposed in Gerrit. The
|
||||||
|
backporter is expected to follow the progress of the backport to make sure it's
|
||||||
|
not lost in reviews.
|
||||||
|
|
||||||
|
Once all applicable backports are proposed to Gerrit and are on their path
|
||||||
|
towards stable inclusion, *<project>-proactive-backport-potential* tag can be
|
||||||
|
removed from the bug.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
If possible, consider keeping the order of backports in a way that would
|
||||||
|
reduce the number of git conflicts.
|
||||||
|
|
||||||
|
Release often
|
||||||
|
-------------
|
||||||
|
|
||||||
|
Proactive backporting process is expected to trigger higher volume of changes
|
||||||
|
in stable branches. To make releases more granular, it's advised participating
|
||||||
|
projects create new stable releases often. It may be done on a bi-weekly basis,
|
||||||
|
or any other schedule that fits better the project and its actual backports
|
||||||
|
volume.
|
||||||
|
|
||||||
.. _Nova Kilo nominations: https://bugs.launchpad.net/nova/kilo/+nominations
|
.. _Nova Kilo nominations: https://bugs.launchpad.net/nova/kilo/+nominations
|
||||||
.. _Nova Liberty potential: https://bugs.launchpad.net/nova/+bugs?field.tag=liberty-backport-potential
|
.. _Nova Liberty potential: https://bugs.launchpad.net/nova/+bugs?field.tag=liberty-backport-potential
|
||||||
.. _Oslo policies: http://specs.openstack.org/openstack/oslo-specs/specs/policy/incubator.html#stable-branches
|
.. _Oslo policies: http://specs.openstack.org/openstack/oslo-specs/specs/policy/incubator.html#stable-branches
|
||||||
@ -337,3 +514,5 @@ piling up. Please subscribe if you're interested in helping out.
|
|||||||
.. _stable maintenance core team: https://review.openstack.org/#/admin/groups/530,members
|
.. _stable maintenance core team: https://review.openstack.org/#/admin/groups/530,members
|
||||||
.. _openstack-infra/project-config repo: http://git.openstack.org/cgit/openstack-infra/project-config/
|
.. _openstack-infra/project-config repo: http://git.openstack.org/cgit/openstack-infra/project-config/
|
||||||
.. _on Designate: https://review.openstack.org/#/c/292617/
|
.. _on Designate: https://review.openstack.org/#/c/292617/
|
||||||
|
.. _openstack-infra/release-tools repository: http://git.openstack.org/cgit/openstack-infra/release-tools/
|
||||||
|
.. _neutron proactive-backport-potential dashboard: https://bugs.launchpad.net/neutron/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=FIXRELEASED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=neutron-proactive-backport-potential&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user