To make tags consistent with traits, change db.api.node_tag_exists()
to raise a NodeNotFound exception if the node doesn't exist.
This is a follow up to bbff319f02.
Change-Id: Id0ddf448d97b5ac22ef5dc27154a1b229514a968
Partial-Bug:#1526266
Adds a NodeTrait model to the DB and DBAPI methods for the following:
* Set/unset node traits
* Get node traits
* Add/remove a single trait
* Check for existence of a node trait
All node traits are deleted when their owner node is deleted. Traits
cannot be specified on node creation currently.
Change-Id: I8b375a33dcb94d2cc73eaa93a5cc5aed86c27178
Partial-Bug: #1722194
Now that we have rolling upgrades and the version column was
added and populated in the Pike release, we can add checks to
make sure the versions of objects in the DB are compatible
with this ironic release, before ironic-dbsync's upgrade or
online_data_migrations does its work. These ironic-dbsync
calls are made as part of upgrading to this (Queens) release.
Change-Id: I68758f8a29d483f5c0a7439fa2ea2962b2eb4124
Partial-Bug: #1708243
This modifies the backfill_version_column method that was used in Pike,
to backfill the version column for the Conductor object, since we missed
setting the Conductor's version column in Pike. This method is invoked
when the operator runs ``ironic-dbsync online_data_migrations``.
Change-Id: I42fec0ac9c8b684219f14822648e627fce529d36
Partial-Bug: 1526283
Fixes a race condition in backfill_version_column(). It was fetching
the objects whose version==None. Then it did an update to set the
value of those object versions. However, it is possible for one of
the object versionss to have been updated after the fetch but before
the update operation, in which case the update operation might set
the version to be an older version.
Change-Id: I882fdd3e83582a4d7110c68b0d84f243234ea7dd
Closes-Bug: #1715190
This adds the new command 'ironic-dbsync online_data_migrations'.
To limit downtime during upgrades, data migrations will be done online
with migration scripts that could be run during normal operation of an
ironic cluster, after the upgrade but before the next one.
Each migration script should ensure that all related DB records are
migrated to the new format. Scripts can detect the format based on
the object version which is stored in the version column.
The online data migration has one script; a function that backfills
the new version column, using versions of objects from the release
prior to this.
This includes code to check the object versions for compatibility with
an ironic release. However, the check is turned off (and will be turned on
in Queens), since we need to boot-strap the new version column before
we can turn the check on. To do this check, we need to keep a list of all
supported versions for every object; release_mapping.RELEASE_MAPPING was
modified so that the object versions is now a list instead of one value.
Change-Id: I1a9fa829951ecf98cae6896d82ba20cf89062394
Closes-Bug: #1585141
Partial-bug: #1526283
Co-Authored-By: Ruby Loo <ruby.loo@intel.com>
Switch to enginefacade.writer during migrations instead of using
legacy facade, use inspect.signature instead of inspect.getargspec
in python 3.
Change-Id: I8ce536a5ce7e9f248d75fb9de8e7c1974a3423b3
This patch does the following:
* Adds osprofiler wsgi middleware
This middleware is used for 2 things:
- It checks that person who wants to trace is trusted and knows
secret HMAC key.
- It starts tracing in case of proper trace headers
and adds first wsgi trace point, with info about HTTP request.
* Adds initialization of osprofiler at start of service
- Initialize and set an oslo.messaging based notifier instance
to osprofiler, which will be used to send notifications to Ceilometer.
* Traces HTTP/RPC/DB API calls and SQL requests
NOTE to test this patch:
1) Make the following changes in localrc to configure DevStack to enable
OSProfiler:
enable_plugin panko https://git.openstack.org/openstack/panko
enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer
enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler
# Enable the following services
CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler
ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral
ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector
ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier
ENABLED_SERVICES+=,ceilometer-api
NOTE: the order of enabling plugins matters.
2) Run stack.sh. Once DevStack environment is setup, enable profiler options
in ironic.conf and restart ironic services:
[profiler]
enabled = true
hmac_keys = SECRET_KEY
trace_sqlalchemy = true
3) Use openstackclient and run baremetal command with
--os-profile SECRET_KEY
[--profile can be used, but it is deprecated.]
For example, the following will cause the <trace-id> to be printed
after node list:
$ openstack --os-profile SECRET_KEY baremetal node list
.....
.....
Trace ID: <trace-id>
Display trace with command:
osprofiler trace show --html <trace-id>
4) The trace results can be saved using this command:
$ osprofiler trace show --html <trace-id> --out trace.html
OSprofiler spec: https://review.openstack.org/#/c/103825/
Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>
Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com>
Partial-Bug: #1560704
Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
If the system is set to use MyISAM engine as default, during upgrade
a new conductor_hardware_interfaces table will be created with MyISAM
engine. This will cause a mix of InnoDB and MyISAM tables in single
database, and obsolete a foreign key to conductors table.
This bug in particular fails test_models_sync unit test, when it is
executed on a system, where MyISAM is set as default option for local
MySQL server.
Change-Id: Ic47426b1a12eda5728e9971a27ad3767c0245d50
Closes-Bug: 1702158
This extends versioned objects and uses the object's
convert_to_version() to perform conversions between different
versions of the object. This conversion is only done at the
"boundaries" of the api and conductor services. That is, when
reading/writing to the database and when serializing/deserializing
(for RPC). Internally, the services deal with the latest versions
of the objects.
Version column is introduced to make sure reading the DB object
versions happens transparently for the developer. An exception
is raised in case of a version compatibility error.
The version column is null at first and will be filled with the
appropriate versions by a data migration script in a following patch.
Change-Id: I34629a5cbab7ff3f246ea19c0cb766badc7061db
Partial-Bug: #1526283
Co-Authored-By: Ruby Loo <ruby.loo@intel.com>
This change implements the changes to the port data model required to
support physical network awareness. This includes updating the ports
table in the database, and extending the Port object.
Change-Id: Ib22753aa6ae0fedce7fb9ecf63f135fda0185c5b
Partial-Bug: 1666009
This patch introduces methods for retrieving volume targets by
their ID. As this field does not have a unique constraint, getting
volume targets by their volume_id field will return a list. This
will allow us to simplify attaching volumes in the cinder driver
as we can now easily query the database rather than looping
through all connected volumes.
Change-Id: I05709fd2ab09bd127c0d202ddcd4863077db3d5b
Partial-Bug: #1559691
The option sqlite_db was deprecated and removed by
Id269d921e40edf95eb977b011f1753f633b79d18
As we passed it as positional argument, another max_pool_size
var was set automatically.
This patch removes dropped sqlite_db option.
Change-Id: Icdd89453ec3ebecd8375a5f2efd1a0d4bf6b4064
Closes-Bug: #1676809
Remove all calls to logging functions from the db directory of ironic.
In this instance, there were very few, and all were to _LW.
Change-Id: I474f2d51bf7cb9d6082d963de182d4a1fed142b3
Partial-bug: #1674374
Wrap all DB write operations with oslo_db decorator to retry on
deadlock. DBDeadlock exception can be raised quite often on Galera
clusters under load.
This also fixes an issue which causes conductor's periodic
tasks to stop executing.
Closes-Bug: #1637210
Closes-Bug: #1639338
Co-Authored-By: Vladyslav Drok <vdrok@mirantis.com>
Co-Authored-By: Joanna Taryma <joanna.taryma@intel.com>
Change-Id: I61db83637adfd98a5394d1f570f3de4302c93497
Before this patch if _do_update_node() sets "instance_uuid" field
it fails on second execution one by one with exception. This
prevents normal usage of DB reconnect method which proposed with
change I61db83637adfd98a5394d1f570f3de4302c93497. Check of
"instance_uuid" moved to manager method.
Change-Id: I2ae84c95cbd1895068dc4bce01bc122abc7345da
In the refactoring of some code in ironic/db/sqlalchemy/api,
there was some duplication of code in get_active_driver_dict().
This removes that duplication.
This is a follow up to 3c45f2fd1b.
Change-Id: Id32280a6c020dce8e3580ea951b5822cf6a8eabf
Partial-Bug: #1524745
This adds API version 1.30, which adds dynamic driver parameters
and response fields to `GET /v1/drivers` and `GET /v1/drivers/<name>`.
Changes RAID APIs to work for dynamic drivers.
Also changes GET /v1/drivers/<name>/properties to work for dynamic
drivers. It uses the calculated default implementation for each
interface, when calculating the properties.
Last, changes node and driver vendor passthru to work correctly
for dynamic drivers. Similar to properties, driver vendor passthru
will use the calculated default vendor implementation.
Change-Id: If13e7e7fd368273e84d9a108be93b58150432fae
Partial-Bug: #1524745
This loads hardware types into the hash ring, along with the drivers
that are currently there. This allows us to make RPC calls for nodes
that are using a dynamic driver.
Adds a dbapi method `get_active_hardware_type_dict`, similar to
`get_active_driver_dict`, to get a mapping of hardware types to active
conductors.
Change-Id: I50c1428568bd8cbe4ef252d56a6278f1a54dfcdb
Partial-Bug: #1524745
This registers the intersection of supported and enabled interfaces for
each hardware type enabled in the conductor at conductor startup, and
unregisters them at conductor shutdown. Validation is left as a todo for
now.
Change-Id: I14e88bfc304de9414de008d1cc8568dda9115ecc
Partial-Bug: #1524745
This will mark the default interface for each hardware type in use.
This is the only piece of information about hardware interfaces that is
not in the database, and so this will allow the API to fetch this
information without an RPC call.
Change-Id: I5b05853a64ebb2744f9dc87855fb31fbe5cefce6
Partial-Bug: #1524745
A helper table conductor_hardware_interfaces was created to track which
hardware types have which interfaces loaded on which conductors.
Partial-Bug: #1524745
Change-Id: Iae2f3fddb2441b6069c011cf8bf646943d34d0a4
Add storage_interface DB field, database migration,
and object entry for storage_interface for boot from
volume functionality.
Initial versions of this revision added the field to
the node object, but since then Ironic's code has
drifted such that adding an object for an interface
has to take place along with the entire driver interface,
thus the object change is in the next revision.
Co-Authored-By: Stephane Miller <stephane@alum.mit.edu>
Change-Id: Ib5d5de87163570949c12d9e95c73e17c1c12eec1
Partial-Bug: #1559691
This change adds mode and properties fields into portgroup object
and adds a new API microversion to work with them.
It also makes portgroups' address field optional for all API
microversions.
Partial-Bug: #1618754
Change-Id: Id8c62fa56908040b0df16cc54c122ce2473a4587
This patch adds a "volume_targets" DB table in order to save
the volume target information of physical nodes. With this patch,
Ironic can put/get volume target information to/from the database.
Co-Authored-By: Stephane Miller <stephane@alum.mit.edu>
Co-Authored-By: Ruby Loo <ruby.loo@intel.com>
Change-Id: I79063f9d0aafd7b740785a883732536704e43b7c
Partial-Bug: 1526231
This revision follows up on the feedback on change
I8237ab671a04ac9b01c8bdb2936cf0a661b9b1de which was
to rename one of the parameters use of "id" which is
confusing due to the "connector_id" field in the data
model for a volume connector, when id purely refers
to the "id" column in the database.
Additionally follows-up to the following object
revision I06aa1a94401e7c7faeb019926272a072d1ed1c51
and updates the variable name and some of the doc
strings in that method.
Change-Id: I6a34b195e6dd6f7898348f68646018dfa8bf77de
This patch introduces a new "volume_connectors" table in order to save
the volume connector information of physical nodes. With this patch,
Ironic can put/get volume connector information to/from the database.
Co-Authored-By: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Co-Authored-By: Stephane Miller <stephane@alum.mit.edu>
Co-Authored-By: Julia Kreger <juliaashleykreger@gmail.com>
Change-Id: I8237ab671a04ac9b01c8bdb2936cf0a661b9b1de
Partial-Bug: 1526231
For the driver composition reform we need nodes to record hardware
interfaces they're using. This change adds necessary fields.
The new fields will have a default value of None, which is the expected
value for nodes using classic drivers.
The new fields are not added to notifications yet, as they're not wired in.
Change-Id: Id7697de0276e9b2730ea62f9c562ed6e1d0f8a06
Partial-Bug: #1524745
This patch is a database change to portgroup object that adds
a new field standalone_ports_supported. The field indicates
whether interfaces that belongs to the portgroup may be used
as individual interfaces. For example when hardware doesn't
support portchannel fallback.
Related-Bug: #1618754
Change-Id: I24c2408a0769b630ce275f8688705fc35efd41a8
Nova style refactor of config options in Ironic.
- Merge cimc, cisco_ucs options into one file: conf/cisco_ucs.py
- Remove unnecessary import_opts
- Move [amt] group options to conf/amt.py
- Renamed conf/cisco_ucs.py as conf/cisco.py
- Removed netconf.py.
Change-Id: I3c3f5d21de26a9d23ce766c24674f09999f610f5
Closes-Bug: #1561100
This fills node.network_interface based on the default logic that the
node update/create calls follow, so that we know network_interface is
populated when this code is deployed.
Also adds to existing release note about how network_interface is set, to
remind people to check the logic and configs to make sure they get the
expected result.
Change-Id: I09a42c8e54d7782c591415e53fccade972ae8bdb
Closes-Bug: #1608511
This adds the "resource_class" field to the node table, object, and API,
as well as a database migration to go with it.
Change-Id: I936f2e7b2f4d26e01354e826e5595ff021c3a55c
Partial-Bug: #1604916
When an exception was caught and rethrown, it should call 'raise'
without any arguments because it shows the place where an exception
occured initially instead of place where the exception re-raised.
Change-Id: I98c11a0dd394c122cc3f3ade0ba651f99aec1530
In case of ports, it is also added to the API, as a readonly field.
It will be used for any port-specific internal information ironic
needs to store inside the port object. In this change we start using
it to store UUIDs of the cleaning ports that ironic creates, instead
of fiddling with port.extra['vif_port_id'], as extra is intended for
operator use only.
Partial-bug: #1526403
Change-Id: Ib62c3e32305619d0c55f8ec7e45b067f0f0b32d4
This patch adds a new field 'network_interface' to node object.
Its default value is None meaning 'use the default network interface'
to comply with driver composition reform. The default setting logic
will be added to the driver factory in the next patch.
Partial-bug: #1526403
Co-Authored-By: Vladyslav Drok <vdrok@mirantis.com>
Change-Id: Id9b9f4199c2640ca30b2d87c3a517c65f9cb5527
Ruby Loo