sql: Move migrations to 'legacy_migrations'

We're going to add new alembic-based migrations shortly. These will live
in the 'keystone.common.sql.migrations' module. Prepare for this by
moving the existing migrations from ''keystone.common.sql' into a common
'keystone.common.sql.legacy_migrations' module.

Change-Id: I5ab7b010b21268977f73738e895bbd21442e9455
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This commit is contained in:
Stephen Finucane 2022-01-10 17:13:14 +00:00
parent 15847926ef
commit aebd037f50
46 changed files with 82 additions and 38 deletions

View File

@ -17,10 +17,17 @@
Database Migrations
===================
.. note::
The framework being used is currently being migrated from
SQLAlchemy-Migrate to Alembic, meaning this information will change in the
near-term.
Starting with Newton, keystone supports upgrading both with and without
downtime. In order to support this, there are three separate migration
repositories (all under ``keystone/common/sql/``) that match the three phases
of an upgrade (schema expansion, data migration, and schema contraction):
repositories (all under ``keystone/common/sql/legacy_migrations``) that match
the three phases of an upgrade (schema expansion, data migration, and schema
contraction):
``expand_repo``
For additive schema modifications and triggers to ensure data is kept in

View File

@ -53,9 +53,9 @@ Refer to the :doc:`API Change tutorial <api_change_tutorial>`. In short, you wil
steps:
#. Create a SQL migration to add the parameter to the database table
(:py:mod:`keystone.common.sql.expand_repo.versions`,
:py:mod:`keystone.common.sql.data_migration_repo.versions`,
:py:mod:`keystone.common.sql.contract_repo.versions`)
(:py:mod:`keystone.common.sql.legacy_migration.expand_repo.versions`,
:py:mod:`keystone.common.sql.legacy_migration.data_migration_repo.versions`,
:py:mod:`keystone.common.sql.legacy_migration.contract_repo.versions`)
#. Add a SQL migration unit test (`keystone/tests/unit/test_sql_upgrade.py`)

View File

@ -138,6 +138,12 @@ Identity module.
Testing Schema Migrations
-------------------------
.. note::
The framework being used is currently being migrated from
SQLAlchemy-Migrate to Alembic, meaning this information will change in the
near-term.
The application of schema migrations can be tested using SQLAlchemy Migrate's
built-in test runner, one migration at a time.
@ -151,9 +157,9 @@ version control:
.. code-block:: bash
$ python keystone/common/sql/expand_repo/manage.py test \
$ python keystone/common/sql/legacy_migrations/expand_repo/manage.py test \
--url=sqlite:///test.db \
--repository=keystone/common/sql/expand_repo/
--repository=keystone/common/sql/legacy_migrations/expand_repo/
This command references to a SQLite database (test.db) to be used. Depending on
the migration, this command alone does not make assertions as to the integrity

View File

@ -1,4 +0,0 @@
This is a database migration repository.
More information at
https://opendev.org/openstack/sqlalchemy-migrate

View File

@ -1,4 +0,0 @@
This is a database migration repository.
More information at
https://opendev.org/openstack/sqlalchemy-migrate

View File

@ -1,4 +0,0 @@
This is a database migration repository.
More information at
https://opendev.org/openstack/sqlalchemy-migrate

View File

@ -0,0 +1,13 @@
Contract repo migrations
========================
.. warning::
This repo is deprecated and will be removed in a future release. All new
migrations should be alembic-based and placed in
``keystone/common/sql/migrations``.
Contract-style or destructive migrations for the database.
This is a database migration repository. More information at
https://opendev.org/x/sqlalchemy-migrate

View File

@ -0,0 +1,13 @@
Data migration repo migrations
==============================
.. warning::
This repo is deprecated and will be removed in a future release. All new
migrations should be alembic-based and placed in
``keystone/common/sql/migrations``.
Data migrations for the database.
This is a database migration repository. More information at
https://opendev.org/x/sqlalchemy-migrate

View File

@ -0,0 +1,13 @@
Expand repo migrations
======================
.. warning::
This repo is deprecated and will be removed in a future release. All new
migrations should be alembic-based and placed in
``keystone/common/sql/migrations``.
Expand-style or additive migrations for the database.
This is a database migration repository. More information at
https://opendev.org/x/sqlalchemy-migrate

View File

@ -68,8 +68,13 @@ class Repository(object):
def find_repo(repo_name):
"""Return the absolute path to the named repository."""
path = os.path.abspath(os.path.join(
os.path.dirname(sql.__file__), repo_name))
path = os.path.abspath(
os.path.join(
os.path.dirname(sql.__file__),
'legacy_migrations',
repo_name,
)
)
if not os.path.isdir(path):
raise exception.MigrationNotProvided(sql.__name__, path)

View File

@ -12,7 +12,6 @@
# License for the specific language governing permissions and limitations
# under the License.
import os
import fixtures
@ -25,9 +24,9 @@ from oslotest import base as test_base
import sqlalchemy
import testtools
from keystone.common.sql import contract_repo
from keystone.common.sql import data_migration_repo
from keystone.common.sql import expand_repo
from keystone.common.sql.legacy_migrations import contract_repo
from keystone.common.sql.legacy_migrations import data_migration_repo
from keystone.common.sql.legacy_migrations import expand_repo
from keystone.common.sql import upgrades

View File

@ -28,7 +28,7 @@ set -e
source .tox/py38/bin/activate
INIT_VERSION=$(ls -1 keystone/common/sql/expand_repo/versions/ | head -1 | awk -F_ '{print $1}' | sed 's/^0*//')
INIT_VERSION=$(ls -1 keystone/common/sql/legacy_migrations/expand_repo/versions/ | head -1 | awk -F_ '{print $1}' | sed 's/^0*//')
INIT_VERSION=$(($INIT_VERSION-1))
echo "Detected init version of $INIT_VERSION"
@ -43,28 +43,28 @@ rm -f "/tmp/keystone-schemas/$INIT_VERSION-*.sql"
function sync () {
DB_URL=$1
python keystone/common/sql/expand_repo/manage.py version_control \
python keystone/common/sql/legacy_migrations/expand_repo/manage.py version_control \
--database "$DB_URL" \
--version "$INIT_VERSION" \
--repository keystone/common/sql/expand_repo/
python keystone/common/sql/data_migration_repo/manage.py version_control \
--repository keystone/common/sql/legacy_migrations/expand_repo/
python keystone/common/sql/legacy_migrations/data_migration_repo/manage.py version_control \
--database "$DB_URL" \
--version "$INIT_VERSION" \
--repository keystone/common/sql/data_migration_repo/
python keystone/common/sql/contract_repo/manage.py version_control \
--repository keystone/common/sql/legacy_migrations/data_migration_repo/
python keystone/common/sql/legacy_migrations/contract_repo/manage.py version_control \
--database "$DB_URL" \
--version "$INIT_VERSION" \
--repository keystone/common/sql/contract_repo/
--repository keystone/common/sql/legacy_migrations/contract_repo/
python keystone/common/sql/expand_repo/manage.py upgrade \
python keystone/common/sql/legacy_migrations/expand_repo/manage.py upgrade \
--database "$DB_URL" \
--repository keystone/common/sql/expand_repo/
python keystone/common/sql/data_migration_repo/manage.py upgrade \
--repository keystone/common/sql/legacy_migrations/expand_repo/
python keystone/common/sql/legacy_migrations/data_migration_repo/manage.py upgrade \
--database "$DB_URL" \
--repository keystone/common/sql/data_migration_repo/
python keystone/common/sql/contract_repo/manage.py upgrade \
--repository keystone/common/sql/legacy_migrations/data_migration_repo/
python keystone/common/sql/legacy_migrations/contract_repo/manage.py upgrade \
--database "$DB_URL" \
--repository keystone/common/sql/contract_repo/
--repository keystone/common/sql/legacy_migrations/contract_repo/
}
#