Make writing design docs more collaborative and iterative

When we introduced the design-driven contribution process we started
with a simple process and template for writing design docs, with the
intention to refine the process and template later. Now we gathered some
first experience with design discussions and we noticed/foresee a few
things that are not optimal yet, hence we want to improve the process
and the template.

Things that we want to improve:

1. Agreeing on a solution is difficult and design reviews can take a
   long time. At the moment a design doc can only be submitted once
   everyone agrees on the proposed solution.
2. If alternative solutions are proposed in comments they may not get
   the amount of attention that they deserve.
3. It is seen as the responsibility of the design doc author to pick up
   newly suggested alternative solutions and include them into the
   design doc, but it's hard for the author to parse proposed solutions
   from review comments and describe them accurately and with details in
   the design doc.
4. If newly suggested alternative solutions get included into the design
   doc they likely end up in the 'Alternatives Considered' section,
   which makes them look less favored than the solution that was
   initally advertised in the design doc.
5. Picking up a newly suggested solution as the new main solution
   requires rewriting large parts of the design doc (hence there may be
   some resistence against doing this).
6. There may be a tendency to favor the initially proposed solution
   since more work was going into it and alternative solutions are only
   briefly described.
7. Unrelated issues that are identified in design discussions tend to be
   ignored and forgotten.

Our goal with design discussions was to collaborate as much as possible
and to write design docs together, in an iterative manner. With the
current process and template this is not working well. To improve this
we suggest to split design docs into multiple files that can be written
and refined by several persons in parallel:

* index.md:
  Entry file that links to the files below.
* use-cases.md:
  Describes the use-cases, acceptance criteria and background.
* solution-<n>.md:
  Each possible solution (with the pros and cons, and implementation
  details) is described in a separate file.
* conclusion.md:
  Describes the conclusion of the design discussion.

It is expected that:

* An agreement on the use-cases is achieved before solutions are being
  discussed in detail.
* Anyone who has ideas for an alternative solution uploads a change
  with a solution-<n>.md that describes their solution.
* All possible solutions are fairly discussed with their pros and cons,
  and treated equally until a decision is made.
* Unrelated issues that are identified during discussions are extracted
  into new design docs (initially consisting only of an index.md and a
  use-cases.md file).
* Changes making iterative improvements can be submitted frequently
  (e.g. additional uses-cases can be added later, solutions can be
  submitted without describing implementation details, etc.).

In addition some aspects of the documentation which were outdated have
been adapted to reality:

* Design docs are written in Markdown, not Asciidoc (hence the templates
  were migrated).
* Design docs are reviewed and stored in the homepage repo, not in the
  gerrit repo.

Furthermore some minor improvements were done:

* It is now recommended to use the 'dir' query operator to watch
  design doc changes ('dir:pages/design-docs'), since it's more reliable
  than the 'hashtag' operator which was suggested before since people
  tend to forget to set this hashtag.
* Call out in the template for use-cases that Google-specific uses cases
  should be clearly marked as such (the Google setup is pretty different
  and not making this transparent can confuse readers).
* The design docs now talk about use-cases and acceptance criteria
  instead of objectives, since formulating use-cases forces the author
  to think from the users perspective and it's easier for reader to
  understand how users are affected.
* The period in which the ESC is expected to comment on newly proposed
  features was increased to 14 calendar days. This allows the ESC to
  discuss the proposal in the ESC meeting, which happens bi-weekly,
  before commenting on it.

Signed-off-by: Edwin Kempin <ekempin@google.com>
Change-Id: I401d5ec21ff2b568dab7346db212c9166537dfa9

Documentation/dev-design-doc-conclusion-template.md

@@ -0,0 +1,8 @@
+# Conclusion
+
+Describe which decision was made and what were the reasons for it.
+
+## <a id="implementation-plan"> Implementation Plan
+
+If known, say who is driving the implementation, for when the
+implementation is planned and which priority it has.

Documentation/dev-design-doc-index-template.md

@@ -0,0 +1,8 @@
+# Design Doc - ${title}
+
+* [Use Cases](use-cases.html)
+* [Solution - ${solution-name-1}](solution-1.html)
+* [Solution - ${solution-name-2}](solution-2.html)
+* ...
+* [Conclusion](conclusion.html)
+

Documentation/dev-design-doc-solution-template.md

@@ -0,0 +1,57 @@
+# Solution - ${solution-name}
+
+## <a id="overview"> Overview
+
+High-level overview; put details in the next section and background in
+the 'Background' section (see dev-design-doc-use-cases-template.txt).
+
+Should be understandable by engineers that are not working on Gerrit.
+
+If a solution is a variant of another solution, that other solution
+should be linked here.
+
+## <a id="detailed-design"> Detailed Design
+
+How does the overall design work? Details about the algorithms,
+storage format, APIs, etc., should be included here.
+
+For the initial review, it is ok for this to lack implementation
+details of minor importance.
+
+### <a id="scalability"> Scalability
+
+How does the solution scale? Consider both, data size increase (if
+applicable) and traffic increase (if applicable).
+
+## <a id="alternatives-considered"> Alternatives Considered
+
+Within the scope of this solution you may need to describe what you did
+not do or why simpler approaches don't work. Mention other things to
+watch out for (if any).
+
+Do not describe alternative solutions in this section, as each solution
+should be described in a separate file.
+
+## <a id="pros-and-cons"> Pros and Cons
+
+Objectively list all points that speak in favor/against this solution.
+
+## <a id="implementation-plan"> Implementation Plan
+
+If known, say who would be willing to drive the implementation.
+
+It is possible to contribute solutions without having resources to do
+the implementation. In this case, say so here.
+
+If mentor support is desired, say so here. Also briefly describe any
+circumstances that can help with finding a suitable mentor.
+
+## <a id="time-estimation"> Time Estimation
+
+A rough itemized estimation of how much time it takes to implement this
+feature. Break down the feature into work items and estimate each item
+separately.
+
+If a mentor is assigned, this section must define a maximum time frame
+after which the mentorship automatically ends even if the feature isn't
+fully done yet.

Documentation/dev-design-doc-template.txt

@@ -1,79 +0,0 @@
-= Gerrit Code Review - ${title}
-
-[[objective]]
-== Objective
-
-In a few sentences, describe the key system objectives. Define the
-goals and non-goals.
-
-[[background]]
-== Background
-
-Stuff one needs to know to understand this doc (e.g. motivating
-examples, previous versions and problems, links to related
-changes/design docs, etc.
-
-Note: this is background; do not write about your design or ideas to
-solve problems here.
-
-[[overview]]
-== Overview
-
-High-level overview; put details in the next section and background in
-the previous section. Should be understandable by engineers that are
-not working on Gerrit.
-
-[[detailed-design]]
-== Detailed Design
-
-How does the overall design work? Details about the algorithms,
-storage format, APIs, etc., should be included here.
-
-It is ok for this to lack in detail at first for initial review.
-
-[[alternatives-considered]]
-== Alternatives Considered
-
-You may need to describe what you did not do or why simpler approaches
-don't work. Mention other things to watch out for (if any).
-
-[[implemenation-plan]]
-== Implementation Plan
-
-If known, say who is driving the implementation, for when the
-implementation is planned and which priority it has for you.
-
-It is possible to contribute designs without having resources to do the
-implementation. In this case, say so here.
-
-If mentor support is desired, say so here. Also briefly describe any
-circumstances that can help with finding a suitable mentor.
-
-[[time-estimation]]
-=== Time Estimation
-
-A rough itemized estimation of how much time it takes to implement this
-feature. Break down the feature into work items and estimate each item
-separately.
-
-If a mentor is assigned, this section must define a maximum time frame
-after which the mentorship automatically ends even if the feature isn't
-fully done yet.
-
-[[done-criteria]]
-== Done Criteria
-
-Describe the conditions that must be satisfied to consider this feature
-as done.
-
-If a mentor is assigned, the mentorship ends when this state is reached.
-Please note that a mentorship can also end earlier if the maximum time
-frame for the mentorship has exceeded (see section 'Time Estimation'
-above).
-
-GERRIT
-------
-Part of link:index.html[Gerrit Code Review]
-
-SEARCHBOX
----------

Documentation/dev-design-doc-use-cases-template.md

@@ -0,0 +1,38 @@
+# Use Cases
+
+In a few sentences, describe the use-cases as interactions between a
+user and a system to attain particular goals.
+
+Should be understandable by anyone who is familiar with using Gerrit.
+
+Optionally, differentiate between primary and secondary use-cases.
+Secondary use-cases are related to the primary use-cases, but
+addressing them within the scope of this design is not mandatory. This
+means they may not be covered by all proposed solutions. Secondary
+use-cases that are not addressed by the concluded solution, may be
+discussed in separate design docs. In this case links to these design
+docs should be added here.
+
+Optionally, define non-goals.
+
+It is possible that use-cases are specific to custom setups (e.g. the
+multi-master setup at Google). In this case, say so here.
+
+## <a id="acceptance-criteria"> Acceptance Criteria
+
+Describe conditions that must be satisfied to consider the feature as
+done.
+
+If a mentor is assigned, the mentorship ends when this state is reached.
+Please note that a mentorship can also end earlier if the maximum time
+frame for the mentorship has exceeded (see section 'Time Estimation'
+in dev-design-doc-conclusion-template.txt).
+
+## <a id="background"> Background
+
+Stuff one needs to know to understand the use-cases (e.g. motivating
+examples, previous versions and problems, links to related
+changes/design docs, etc.).
+
+Note: this is background; do not write about your design or ideas to
+solve problems here.

Documentation/dev-design-docs.txt

@@ -4,14 +4,77 @@ For the link:dev-contributing.html#design-driven-contribution-process[
 design-driven contribution process] it is required to specify features
 upfront in a design doc.
 
+[[structure]]
+== Design Doc Structure
+
+A design doc should discuss the following aspects:
+
+* Use-Cases:
+  The interactions between a user and a system to attain particular
+  goals.
+* Acceptance Criteria
+  Conditions that must be satisfied to consider the feature as done.
+* Background:
+  Stuff one needs to know to understand the use-cases (e.g. motivating
+  examples, previous versions and problems, links to related
+  changes/design docs, etc.)
+* Possible Solutions:
+  Possible solutions with the pros and cons, and explanation of
+  implementation details.
+* Conclusion:
+  Which decision was made and what were the reasons for it.
+
+[[collaboration]]
+As community we want to collaborate on design docs as much as possible
+and write them together, in an iterative manner. To make this work well
+design docs are split into multiple files that can be written and
+refined by several persons in parallel:
+
+* `index.md`:
+  Entry file that links to the files below (also see
+  'dev-design-doc-index-template.md').
+* `use-cases.md`:
+  Describes the use-cases, acceptance criteria and background (also see
+  'dev-design-doc-use-cases-template.md').
+* `solution-<n>.md`:
+  Each possible solution (with the pros and cons, and implementation
+  details) is described in a separate file (also see
+  'dev-design-doc-solution-template.md').
+* `conclusion.md`:
+  Describes the conclusion of the design discussion (also see
+  'dev-design-doc-conclusion-template.md').
+
+[[expectation]]
+It is expected that:
+
+* An agreement on the use-cases is achieved before solutions are being
+  discussed in detail.
+* Anyone who has ideas for an alternative solution uploads a change
+  with a `solution-<n>.md` that describes their solution. In case of
+  doubt whether an idea is a refinement of an existing solution or an
+  alternative solution, it's up to the owner of the discussed solution
+  to decide if the solution should be updated, or if the proposer
+  should start a new alternative solution.
+* All possible solutions are fairly discussed with their pros and cons,
+  and treated equally until a conclusion is made.
+* Unrelated issues (judged by the design doc owner) that are identified
+  during discussions are extracted into new design docs (initially
+  consisting only of an `index.md` and a `use-cases.md` file).
+* Changes making iterative improvements can be submitted frequently
+  (e.g. additional uses-cases can be added later, solutions can be
+  submitted without describing implementation details, etc.).
+* After a conclusion has been approved contributors are expected to
+  keep the design doc updated and fill in gaps while they go forward
+  with the implementation.
+
 [[propose]]
 == How to propose a new design?
 
-To propose a new design, add a `design-${title}.txt` file to this
-folder and push it as change for review. The design doc should follow
-the structure of the link:dev-design-doc-template.html[design doc
-template] and the change should be marked with the hashtag
-`design-doc`.
+To propose a new design, upload a change to the
+link:https://gerrit-review.googlesource.com/admin/repos/homepage[
+homepage] repository that adds a new folder under `pages/design-docs/`
+which contains at least an `index.md` and a `uses-cases.md` file (see
+link:#structure[design doc structure] above).
 
 Pushing a design doc for review requires to be a
 link:dev-roles.html#contributor[contributor].
@@ -28,22 +91,24 @@ to do it (which is not guaranteed to happen).
 Everyone in the link:dev-roles.html[Gerrit community] is welcome to
 take part in the design review and comment on the design.
 
-Changes with new design docs should stay open for a minimum of 10
-calendar days so that everyone has a fair chance to see them. It is
-important that concerns regarding a feature are raised during this time
-frame since once a design is approved and submitted the implementation
-may start immediately.
+Ideas for alternative solutions should be uploaded as a change that
+describes the solution (see link:#collaboration[above]).
 
-Within the 10 calendar days time frame, the contributor should hear back
-from the link:dev-processes.html#steering-committee[engineering steering committee]
-whether the proposed feature is in scope of the project and if it can
-be accepted.
+Changes which make a conclusion on a design (changes that add/change
+the `conclusion.md` file, see link:#structure[Design Doc Structure])
+should stay open for a minimum of 10 calendar days so that everyone has
+a fair chance to see them. It is important that concerns regarding a
+feature are raised during this time frame since once a conclusion is
+approved and submitted the implementation may start immediately.
 
-In order to be accepted/submitted, it is not necessary that the design
-doc fully specifies all the details, but the idea of the feature and
-how it fits into Gerrit should be sufficiently clear (judged by the
-engineering steering committee). Contributors are expected to keep the design doc
-updated and fill in gaps while they go forward with the implementation.
+Other design doc changes can and should be submitted quickly so that
+collaboration and iterative refinements work smoothly (see
+link:#collaboration[above]).
+
+For proposed features the contributor should hear back from the
+link:dev-processes.html#steering-committee[engineering steering
+committee] within 14 calendar days whether the proposed feature is in
+scope of the project and if it can be accepted.
 
 [[watch-designs]]
 == How to get notified for new design docs?
@@ -51,8 +116,8 @@ updated and fill in gaps while they go forward with the implementation.
 . Go to the
   link:https://gerrit-review.googlesource.com/settings/#Notifications[
   notification settings]
-. Add a project watch for the `gerrit` repository with the following
-  query: `hashtag:design-doc`
+. Add a project watch for the `homepage` repository with the following
+  query: `dir:pages/design-docs`
 
 GERRIT
 ------