Merge "Add migration policy to upgrades devref"

doc/source/upgrade.rst

@@ -63,6 +63,60 @@ storage from one database location to another.
 
 :emphasis:`Note: Database downgrades are not supported.`
 
+Migration policy:
+'''''''''''''''''
+
+The following guidelines for schema and data migrations are followed in order
+to ease upgrades:
+
+* Additive schema migrations - In general, almost all schema migrations should
+  be additive.  Put simply, they should only create elements like columns,
+  indices, and tables.
+
+* Subtractive schema migrations - To remove an element like a column or table
+  during the N release cycle:
+
+  #. The element must be deprecated and retained for backward compatibility.
+     (This allows for graceful upgrade from N to N+1.)
+
+  #. Data migration, by the objects layer, must completely migrate data from
+     the old version of the schema to the new version.
+
+     * `Data migration example
+       <http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/flavor-from-sysmeta-to-blob.html>`_
+     * `Data migration enforcement example
+       <https://review.openstack.org/#/c/174480/15/nova/db/sqlalchemy/migrate_repo/versions/291_enforce_flavors_migrated.py>`_
+       (for sqlalchemy migrate/deprecated scripts):
+
+  #. The column can then be removed with a migration at the start of N+2.
+
+* All schema migrations should be idempotent.  (For example, a migration
+  should check if an element exists in the schema before attempting to add
+  it.)  This logic comes for free in the autogenerated workflow of
+  the online migrations.
+
+* Constraints - When adding a foreign or unique key constraint, the schema
+  migration code needs to handle possible problems with data before applying
+  the constraint. (Example:  A unique constraint must clean up duplicate
+  records before applying said constraint.)
+
+* Data migrations - As mentioned above, data migrations will be done in an
+  online fashion by custom code in the object layer that handles moving data
+  between the old and new portions of the schema.  In addition, for each type
+  of data migration performed, there should exist a nova-manage option for an
+  operator to manually request that rows be migrated.
+
+  * See `flavor migration spec
+    <http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/flavor-from-sysmeta-to-blob.html>`_
+    for an example of data migrations in the object layer.
+
+*Future* work -
+   #. Adding plumbing to enforce that relevant data migrations are completed
+      before running `contract` in the expand/migrate/contract schema migration
+      workflow.  A potential solution would be for `contract` to run a gating
+      test for each specific subtract operation to determine if the operation
+      can be completed.
+
 Concepts
 --------