This reverts commit 2f743b2f20a559c5d2e05aa716b9137e928eb017.
At the Mitaka design summit we discussed removing this as it is
not complete, not under development, and not even an active
experiment for anyone. Since this being in the tree is confusing
people and known to be dangerous, this removes it from the tree.
Conflicts:
nova/cmd/manage.py
nova/db/sqlalchemy/migration.py
nova/exception.py
nova/tests/unit/db/test_migrations.py
Change-Id: Icae28ceee3ec975c907d73b95babab58dcb30c23
The migrate_version table is "locked" on DB expand to avoid switching
from the expand/migrate/contract to a sync workflow, but this is not
necessary for a dry run.
Change-Id: I5340efa170304ef4238f96e90ba5b5e7390a7b99
Closes-Bug: 1487176
Using the model as a declarative source, generate and schedule schema
changes into expand, migrate and contract phases. This allows some
(and potentially all) of the schema changes to be performed while
services are still running, reducing downtime during deploys.
Updated based on code review to move from iteritems to items based on
recent lists discussions.
Partially-Implements: blueprint online-schema-changes
Change-Id: If666d5162674aaa874ff2c40799f29e23856544d
Nova-manage recently gained the ability to manage both an 'api' and the
'main' database. It initially used get_engine() to get a sqlalchemy
engine which had no assurance of being configured for the correct
database. There is now a get_api_engine() method so it should use that
for 'nova-manage api_db' commands.
This means that it those commands will operate on the database
configured under CONF.api_database.connection
DocImpact
Partially-Implements: bp cells-v2-mapping
Change-Id: I8707baf66e329a274ed5e53aba8d25286c30603b
This creates the first new table for cells v2. This table will hold
information on how nova-api can communicate with each cell.
Change-Id: Ib224c5b22a350c6e22a95df2bc03ad96683b4c96
bp: cells-v2-mapping
This sets up the framework for adding database migrations to run on the
new api database. It splits migration commands so that they can run on
the current 'main' database or the new 'api' database.
bp: cells-v2-mapping
Change-Id: I73dee6fd51b74bd670a6dffa07e875bf86891e97
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
This adds the database migration that checks for records
where instance_uuid is NULL and fails if found until the
'nova-manage db null_instance_uuid_scan' command is run
and the records are manually deleted.
Unit tests are added for the migration to make sure the
fixed_ips table is left alone and the UniqueConstraint is
created on the instances table.
DocImpact: New 'nova-manage db null_instance_uuid_scan'
command.
UpgradeImpact: The new database migration script.
Implements blueprint enforce-unique-instance-uuid-in-db
Change-Id: I768c36b140c7e9f437907d4dfe175e1410ca44b0
The current code in migration.py suppress underlying exception message.
If Nova sql db configuration (eg. user, or permission) is incorrect, or
the query result is not as expected, nova-manage command will display
a confusing message, which may not help user to address the problem.
Need to log the exception message and stack trace, in order to improve
servicability.
Change-Id: I742f25dd643a7886a5fd5f66eee899ae5b17b439
oslo.i18n provides the i18n functions that were provided by
oslo-incubator's gettextutils module. Some tests that were
using internal details of the library were removed.
Change-Id: I44cfd5552e0dd86af21073419d31622f5fdb28e0
This sync contains the following commits from olso-incubator:
7959826 db: move all options into database group
dda24eb Introduce mysql_sql_mode option, remove old warning
0b5af67 Introduce a method to set any MySQL session SQL mode
8dccc7b Handle ibm_db_sa DBDuplicateEntry integrity errors
0f24d82 Fix migration.db_version when no tables
ac84a40 Update log translation domains
c0d357b Add model_query() to db.sqlalchemy.utils module
84254fc Fix a small typo in api.py
b8a676c Remove CONF.database.connection default value
86707cd Remove None for dict.get()
0545121 Fix duplicating of SQL queries in logs
fcf517d Update oslo log messages with translation domains
fa05b7c Restore the ability to load the DB backend lazily
630d395 Don't use cfg.CONF in oslo.db
ce69e7f Don't store engine instances in oslo.db
35dc1d7 py3kcompat: remove
b4f72b2 Don't raise MySQL 2013 'Lost connection' errors
271adfb Format sql in db.sqlalchemy.session docstring
0334cb3 Handle exception messages with six.text_type
eff69ce Drop dependency on log from oslo db code
7a11a04 Automatic retry db.api query if db connection lost
11f2add Clean up docstring in db.sqlalchemy.session
1b5147f Only enable MySQL TRADITIONAL mode if we're running against MySQL
39e1c5c Move db tests base.py to common code
986dafd Fix parsing of UC errors in sqlite 3.7.16+/3.8.2+
bcf6d5e Small edits on help strings
ae01e9a Transition from migrate to alembic
Due to API changes in oslo.db code we have to change Nova code a bit
in order to reuse oslo-incubator changes (oslo.db no longer stores
SQLAlchemy Engine and sessionmaker instances globally and it's up to
applications to create them).
Change-Id: I4aaa7151f66e0292ff66c29330f93d78c6bf78a6
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
Compacts the pre-Icehouse database migrations into a
single migration (216_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Add aggregate metadata unique constraint.
2) Fix migrations index (add deleted column).
3) Add pci_devices table.
4) fix migrations_instance_uuid_and_status_idx index.
Implements: blueprint compact-havana-db-migrations
Change-Id: Id96e13b0ed65837761bd173c9e32ebcc1ff5f376
Compacts the pre-Icehouse database migrations <= 210 into a
single migration (210_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Add more indexes for SQLite DB (per 201).
2) Add unique constraint on instance_type_extra_specs.
3) Add project_user_quotas, indexes, and constraints.
4) Add indexes to reservations.
5) Add locked by to instances.
6) Add cleaned column to instances.
7) Add host_ip/supported_instances to compute_nodes.
8) Add missing Fkeys.
9) Fix project_user_quotas_user_id_deleted index.
Implements: blueprint compact-havana-db-migrations
Change-Id: I63e00cd29eb4c26717b5ebd7a1ff11a01c0085de
Compacts the pre-Icehouse database migrations <= 200 into a
single migration (200_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Add unique constraint on quotas.
2) cleanup virtual_interfaces unique constraints
3) Remove a bunch of volume tables (cinder cleanup)
4) Indexes cleanup per 194_fix_152_migration_indexes.py
5) Add unique constraint on fixed_ips.address.
6) Add unique constraint on services columns.
7) Add unique constraint on agent_builds columns.
8) Add unique constraint on console_pools columns.
9) Add unique constraint on aggregate_hosts columns.
10) Add transport_url column to cells. (removes RPC columns)
Implements: blueprint compact-havana-db-migrations
Change-Id: Ie14e9e5de6d7d43ebabfc4680ed6434d08ecee49
Compacts the pre-Icehouse database migrations <= 190 into a
single migration (190_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Drop migrations 181 - 184 which synced up shadow table changes.
We now accomodate these changes in the base migration.
2) Rename unique contraints per 185_rename_unique_constraints.py
3) Add a new BDM table format.
4) Add instance groups tables.
5) add reason column to service
6) add cells unique constraint
7) add security groups unique constraint
Change-Id: Ibf2289f37c2376f3bd564728db5400859523c619
Implements: blueprint compact-havana-db-migrations
Compacts the pre-Icehouse database migrations <= 180 into a
single migration (180_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Add uniq_name_x_deleted, uniq_flavorid_x_deleted constraints.
2) Add key_pairs_uniq_name_and_user_id constraint.
3) Add uniq_instance_type_id_x_project_id_x_deleted constraint.
4) Add uniq_address_x_deleted constraint (floating ips)
5) Add availability_zone to volume_usage_cache.
6) Add compute node stats index.
7) Change cells deleted to int (except for shadow table
which is handled in 181_fix_179_migration_sync_shadow_table)
Change-Id: I09fdb1a2e509a3d828b83b78d38a484fe405558b
Implements: blueprint compact-havana-db-migrations
Compacts the pre-Icehouse database migrations <= 170 into a
single migration (170_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) removed 161_fix_system_metadata_none_strings.py. Nothing to
do here since this was a data conversion (no schema changes)
2) Removed some placeholder migrations for the Grizzly stable
branch.
Change-Id: I94a080ca122af009304975044f33c5e5192834ab
Implements: blueprint compact-havana-db-migrations
Compacts the pre-Icehouse database migrations <= 160 into a
single migration (160_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Make task_log period_begining, period_ending a DateTime.
2) Change type of deleted column.
3) Create shadow tables.
4) Add task log unique constraint.
5) Add security_group_default_rules table.
6) Add networks unique constraint.
7) Revert ip column length.
NOTE: This patch removes the 184_sqlite_upgrade.sql and
184_sqlite_downgrade.sql migrations since these were causing
new test failures and the standard DB migration seems to take
care of it.
Implements: blueprint compact-havana-db-migrations
Change-Id: If977e5f02fffbfb1bc257b3beddf2d7133e6c7b9
Compacts the pre-Icehouse database migrations <= 150 into a
single migration (150_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Make migrations.instance_uuid a String 36.
2) Add migrations_instance_uuid_and_status_idx index.
3) rename instance info caches seq (postgresql only, dropped from
baseline)
4) Add source_node, dest_node columns to migrations. Indexes...
5) Add new volume usage cache table.
6) Drop aggregates.availability_zone column.
7) Drop services.availability_zone column.
8) Add instance actions tables.
9) Convert PostgreSQL to INET.
10) Add host to instance faults.
Implements: blueprint compact-havana-db-migrations
Change-Id: Ic0c318967b69af32536b5ab6999affc7755ec442
Compacts the pre-Icehouse database migrations <= 140 into a single
migration (140_havana.py).
Pre-Icehouse users will now need to upgrade to Havana before
running any Icehouse migrations.
Tested on MySQL and PostgreSQL schemas on Fedora 19 to ensure the
new compacted migration generates an identical schema to the
previous migrate scripts in Havana.
Specific changes include:
1) Add counters to bw_usage_cache table.
2) Add node to instances.
3) Add indexes to instances.
4) Add indexes to migrations.
5) drop server name from instances
NOTE: for sqlite this change recreated the instances table altogether.
6) add indexes to fixed ips.
7) Drop unused postgresql volume sequences.
Change-Id: I0405c02191a47b96759131bf6b7a1379f4b5711e
Implements: blueprint compact-havana-db-migrations
This allows us to drop some unsightly
0.7.3 patching and also avoids potential
issues in using 0.8.1 which had a bad
release version string (it was called '0.7.3dev' by mistake).
This will require a requirements change here:
https://review.openstack.org/56662
Change-Id: I5f80dfc07f8256716d025002fc1a1f47e58b2142
Closes-Bug: 1251700
Updates Nova to use a new function called db_initial_version()
instead of the INIT_VERSION constant. This allows us to
move the top level INIT_VERSION = 132 into the
sqlalchemy migration.py implementation where it belongs.
Also, includes changes to do the same for the Nova
baremetal migrations.
Change-Id: I10dacdebb50d3121e3bfd094e40138807233aa64
Previous _ was monkey patched into builtins whenever
certain modules were imported. This removes that and
simply imports it when it is needed.
Change-Id: I0af2c6d8a230e94440d655d13cab9107ac20d13c
E125 continuation line does not distinguish itself from next logical
line.
E125 makes code more readable, and fixing all the existing issues allows
us to gate on this, making this one less thing reviewers have to check
for.
Fix was made using https://github.com/hhatto/autopep8
Change-Id: Ie569fd74fb84d220ceb87e37b1b1f39143bba80f
Remove a lot of getLogger lines and imports of logging in modules
which never use that functionality.
Change-Id: Icdaee2c540980412b000d02ebf1ec568dcf5b38a
Bring in the new database code from oslo.
Uses get_session() from oslo as well as changing NovaBase to derive from
a common class.
Remove test_sqlalchemy.py now that this code is test in oslo.
Implements blueprint db-common.
Change-Id: I090754981c871250dd981cbbe1a08e7181440120
Modules import nova.config for two reasons right now - firstly, to
reference nova.config.CONF and, secondly, if they use one of the
options defined in nova.config.
Often modules import nova.openstack.common.cfg and nova.config
which is a bit pointless since they could just use cfg.CONF if
they just want to nova.config in order to reference CONF.
Let's just use cfg.CONF everywhere and we can explicitly state
where we actually require options defined in nova.config.
Change-Id: Ie4184a74e3e78c99658becb18dce1c2087e450bb
Now that options have all moved from nova.flags to nova.config, we can
safely remove the nova.flags imports and replace them with nova.config
imports.
Change-Id: Ic077a72dd6419bbf1e1babe71acfa43c4e8b55c8
Use the global CONF variable instead of FLAGS. This is purely a cleanup
since FLAGS is already just another reference to CONF.
We leave the nova.flags imports until a later cleanup commit since
removing them may cause unpredictable problems due to config options not
being registered.
Change-Id: I4cd4bdb90d03d7069a419e4a7fbfde101940b5ec
Except for nova/api/, nova/tests/, nova/virt/
In preparation for automated checking being enabled (N306)
Change-Id: I5af70ea13b470ce58695ef0a5e0ff9da3ac1b135
Compacts the pre-Folsom database migrations into
a single migration (082_essex.py).
Pre-Folsom users will need to upgrade to Essex before
running any Folsom migrations.
Implements blueprint db-migration-cleanup.
Testing notes available at:
http://wiki.openstack.org/database_migration_testing
Change-Id: I64c06a3adcf83d6d723c4c11001544ba97668413
This started out as wanting cleanup_volumes to take a UUID instead
of an instance ID, and ended up as a wander through the joys of
schema updates. I am assuming that we actually want to transition
these tables across to using the instance UUID instead of just the
ID.
This is my first attempt at a schema update, so please review this
patch with skepticism. Resolves bug 977975. Partially resolves
blueprint finish-uuid-conversion.
Change-Id: Ib5a6f8a872ea0530e201c70e9ac01cd14f82c557
When a destination version is specified, check the current version and allow
downgrading the schema. Fixes bug 865357.
Change-Id: I47104158884421de92f50322a72243c170a4f1ad
