To make tags consistent with traits, change db.api.node_tag_exists()
to raise a NodeNotFound exception if the node doesn't exist.
This is a follow up to bbff319f02.
Change-Id: Id0ddf448d97b5ac22ef5dc27154a1b229514a968
Partial-Bug:#1526266
Adds a NodeTrait model to the DB and DBAPI methods for the following:
* Set/unset node traits
* Get node traits
* Add/remove a single trait
* Check for existence of a node trait
All node traits are deleted when their owner node is deleted. Traits
cannot be specified on node creation currently.
Change-Id: I8b375a33dcb94d2cc73eaa93a5cc5aed86c27178
Partial-Bug: #1722194
Now that we have rolling upgrades and the version column was
added and populated in the Pike release, we can add checks to
make sure the versions of objects in the DB are compatible
with this ironic release, before ironic-dbsync's upgrade or
online_data_migrations does its work. These ironic-dbsync
calls are made as part of upgrading to this (Queens) release.
Change-Id: I68758f8a29d483f5c0a7439fa2ea2962b2eb4124
Partial-Bug: #1708243
This modifies the backfill_version_column method that was used in Pike,
to backfill the version column for the Conductor object, since we missed
setting the Conductor's version column in Pike. This method is invoked
when the operator runs ``ironic-dbsync online_data_migrations``.
Change-Id: I42fec0ac9c8b684219f14822648e627fce529d36
Partial-Bug: 1526283
Fixes a race condition in backfill_version_column(). It was fetching
the objects whose version==None. Then it did an update to set the
value of those object versions. However, it is possible for one of
the object versionss to have been updated after the fetch but before
the update operation, in which case the update operation might set
the version to be an older version.
Change-Id: I882fdd3e83582a4d7110c68b0d84f243234ea7dd
Closes-Bug: #1715190
This adds the new command 'ironic-dbsync online_data_migrations'.
To limit downtime during upgrades, data migrations will be done online
with migration scripts that could be run during normal operation of an
ironic cluster, after the upgrade but before the next one.
Each migration script should ensure that all related DB records are
migrated to the new format. Scripts can detect the format based on
the object version which is stored in the version column.
The online data migration has one script; a function that backfills
the new version column, using versions of objects from the release
prior to this.
This includes code to check the object versions for compatibility with
an ironic release. However, the check is turned off (and will be turned on
in Queens), since we need to boot-strap the new version column before
we can turn the check on. To do this check, we need to keep a list of all
supported versions for every object; release_mapping.RELEASE_MAPPING was
modified so that the object versions is now a list instead of one value.
Change-Id: I1a9fa829951ecf98cae6896d82ba20cf89062394
Closes-Bug: #1585141
Partial-bug: #1526283
Co-Authored-By: Ruby Loo <ruby.loo@intel.com>
This patch does the following:
* Adds osprofiler wsgi middleware
This middleware is used for 2 things:
- It checks that person who wants to trace is trusted and knows
secret HMAC key.
- It starts tracing in case of proper trace headers
and adds first wsgi trace point, with info about HTTP request.
* Adds initialization of osprofiler at start of service
- Initialize and set an oslo.messaging based notifier instance
to osprofiler, which will be used to send notifications to Ceilometer.
* Traces HTTP/RPC/DB API calls and SQL requests
NOTE to test this patch:
1) Make the following changes in localrc to configure DevStack to enable
OSProfiler:
enable_plugin panko https://git.openstack.org/openstack/panko
enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer
enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler
# Enable the following services
CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler
ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral
ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector
ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier
ENABLED_SERVICES+=,ceilometer-api
NOTE: the order of enabling plugins matters.
2) Run stack.sh. Once DevStack environment is setup, enable profiler options
in ironic.conf and restart ironic services:
[profiler]
enabled = true
hmac_keys = SECRET_KEY
trace_sqlalchemy = true
3) Use openstackclient and run baremetal command with
--os-profile SECRET_KEY
[--profile can be used, but it is deprecated.]
For example, the following will cause the <trace-id> to be printed
after node list:
$ openstack --os-profile SECRET_KEY baremetal node list
.....
.....
Trace ID: <trace-id>
Display trace with command:
osprofiler trace show --html <trace-id>
4) The trace results can be saved using this command:
$ osprofiler trace show --html <trace-id> --out trace.html
OSprofiler spec: https://review.openstack.org/#/c/103825/
Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>
Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com>
Partial-Bug: #1560704
Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
This patch introduces methods for retrieving volume targets by
their ID. As this field does not have a unique constraint, getting
volume targets by their volume_id field will return a list. This
will allow us to simplify attaching volumes in the cinder driver
as we can now easily query the database rather than looping
through all connected volumes.
Change-Id: I05709fd2ab09bd127c0d202ddcd4863077db3d5b
Partial-Bug: #1559691
Remove all calls to logging functions from the db directory of ironic.
In this instance, there were very few, and all were to _LW.
Change-Id: I474f2d51bf7cb9d6082d963de182d4a1fed142b3
Partial-bug: #1674374
Wrap all DB write operations with oslo_db decorator to retry on
deadlock. DBDeadlock exception can be raised quite often on Galera
clusters under load.
This also fixes an issue which causes conductor's periodic
tasks to stop executing.
Closes-Bug: #1637210
Closes-Bug: #1639338
Co-Authored-By: Vladyslav Drok <vdrok@mirantis.com>
Co-Authored-By: Joanna Taryma <joanna.taryma@intel.com>
Change-Id: I61db83637adfd98a5394d1f570f3de4302c93497
Before this patch if _do_update_node() sets "instance_uuid" field
it fails on second execution one by one with exception. This
prevents normal usage of DB reconnect method which proposed with
change I61db83637adfd98a5394d1f570f3de4302c93497. Check of
"instance_uuid" moved to manager method.
Change-Id: I2ae84c95cbd1895068dc4bce01bc122abc7345da
In the refactoring of some code in ironic/db/sqlalchemy/api,
there was some duplication of code in get_active_driver_dict().
This removes that duplication.
This is a follow up to 3c45f2fd1b.
Change-Id: Id32280a6c020dce8e3580ea951b5822cf6a8eabf
Partial-Bug: #1524745
This adds API version 1.30, which adds dynamic driver parameters
and response fields to `GET /v1/drivers` and `GET /v1/drivers/<name>`.
Changes RAID APIs to work for dynamic drivers.
Also changes GET /v1/drivers/<name>/properties to work for dynamic
drivers. It uses the calculated default implementation for each
interface, when calculating the properties.
Last, changes node and driver vendor passthru to work correctly
for dynamic drivers. Similar to properties, driver vendor passthru
will use the calculated default vendor implementation.
Change-Id: If13e7e7fd368273e84d9a108be93b58150432fae
Partial-Bug: #1524745
This loads hardware types into the hash ring, along with the drivers
that are currently there. This allows us to make RPC calls for nodes
that are using a dynamic driver.
Adds a dbapi method `get_active_hardware_type_dict`, similar to
`get_active_driver_dict`, to get a mapping of hardware types to active
conductors.
Change-Id: I50c1428568bd8cbe4ef252d56a6278f1a54dfcdb
Partial-Bug: #1524745
This registers the intersection of supported and enabled interfaces for
each hardware type enabled in the conductor at conductor startup, and
unregisters them at conductor shutdown. Validation is left as a todo for
now.
Change-Id: I14e88bfc304de9414de008d1cc8568dda9115ecc
Partial-Bug: #1524745
This change adds mode and properties fields into portgroup object
and adds a new API microversion to work with them.
It also makes portgroups' address field optional for all API
microversions.
Partial-Bug: #1618754
Change-Id: Id8c62fa56908040b0df16cc54c122ce2473a4587
This patch adds a "volume_targets" DB table in order to save
the volume target information of physical nodes. With this patch,
Ironic can put/get volume target information to/from the database.
Co-Authored-By: Stephane Miller <stephane@alum.mit.edu>
Co-Authored-By: Ruby Loo <ruby.loo@intel.com>
Change-Id: I79063f9d0aafd7b740785a883732536704e43b7c
Partial-Bug: 1526231
This revision follows up on the feedback on change
I8237ab671a04ac9b01c8bdb2936cf0a661b9b1de which was
to rename one of the parameters use of "id" which is
confusing due to the "connector_id" field in the data
model for a volume connector, when id purely refers
to the "id" column in the database.
Additionally follows-up to the following object
revision I06aa1a94401e7c7faeb019926272a072d1ed1c51
and updates the variable name and some of the doc
strings in that method.
Change-Id: I6a34b195e6dd6f7898348f68646018dfa8bf77de
This patch introduces a new "volume_connectors" table in order to save
the volume connector information of physical nodes. With this patch,
Ironic can put/get volume connector information to/from the database.
Co-Authored-By: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Co-Authored-By: Stephane Miller <stephane@alum.mit.edu>
Co-Authored-By: Julia Kreger <juliaashleykreger@gmail.com>
Change-Id: I8237ab671a04ac9b01c8bdb2936cf0a661b9b1de
Partial-Bug: 1526231
Nova style refactor of config options in Ironic.
- Merge cimc, cisco_ucs options into one file: conf/cisco_ucs.py
- Remove unnecessary import_opts
- Move [amt] group options to conf/amt.py
- Renamed conf/cisco_ucs.py as conf/cisco.py
- Removed netconf.py.
Change-Id: I3c3f5d21de26a9d23ce766c24674f09999f610f5
Closes-Bug: #1561100
This adds the "resource_class" field to the node table, object, and API,
as well as a database migration to go with it.
Change-Id: I936f2e7b2f4d26e01354e826e5595ff021c3a55c
Partial-Bug: #1604916
When an exception was caught and rethrown, it should call 'raise'
without any arguments because it shows the place where an exception
occured initially instead of place where the exception re-raised.
Change-Id: I98c11a0dd394c122cc3f3ade0ba651f99aec1530
Currently, creating node with tags is not supported.
If tags values were passed over, an exception should
be thrown.
A follow-up patch of 19ef56cf00
Change-Id: I83e7db6a8da014962454e2b1248fe061985d98d8
Adds a new dbapi call get_node_by_port_addresses and associated objects call
Node.get_by_port_addresses. The logic is the same as in "lookup" agent passthru.
Will be used for a new lookup endpoint.
Change-Id: Ia5549fb16cd363f3492b9ca0400177c92a1aea19
Partial-Bug: #1570841
During clearing locks, also clear target_power_state. As nodes
may locked in powering process, sync_power_state task will sync
the power_state field, but nobody handles target_power_state.
Change-Id: I2293e03c05e13c716f78533680d128ba45ccda02
Closes-Bug: #1567255
Some nodes' console_enabled may be True but the corresponding
shellinabox services stopped while starting conductors, so try
to start consoles on conductor startup.
Change-Id: Ida5fda35340d62e08c779655183ef82211cd8703
Closes-Bug: #1525790
- set/unset node tags
- get node tags
- add/delete single tag
- check whether node tag exists
- Delete all tags attached to the node when it's destroyed
This will not support creating node with tags, just ignore
the specified tags.
Change-Id: Ibe83a726d904fd33b8550c8bec134cf05644560e
Partial-bug: #1526266
Ironic should be able to provide the requisite connectivity
information to the Neutron ML2 plugin to allow drivers to
provision the top-of-rack switch for the bare metal server.
The addition of portgroups in Ironic allows the concept of
link aggregation to be handled in Ironic in order to provide
support for cases where multiple interfaces on the bare metal
server connect to switch ports of a single LAG.
This commit includes changes to:
- the DB models (extension of port model and addition of portgroup
model)
- the DB tests
Partial-bug: #1526403
DocImpact
Co-Authored-By: Laura Moore (laura.moore@sap.com)
Co-Authored-By: Jenny Moorehead (jenny.moorehead@sap.com)
Co-Authored-By: Will Stevenson (will.stevenson@sap.com)
Change-Id: Ic028c316b0670c2d18de89111e83069f3441f476
The error log using node id in some exception log is not
readable, like:
'NodeLocked: Node 16 is locked by host localhost, please
retry after the current operation is completed.'
Update the node id with node uuid to make it more readable.
Change-Id: I83a0adcc28819c44323276d65bc038da7808efac
Closes-Bug: #1537992
LOG.warn is deprecated. It still used in a few places. This updates
those one to use the non-deprecated LOG.warning instead.
Closes-Bug: 1508442
Change-Id: Id947e4c8ae9894f480192a5ab034c24c25125788
This commit adds support for querying reservation of
a node by a list of desired conductor names. It adds
a new filter 'reserved_by_any_of' which is expected to
be a list of conductors.
Change-Id: I149eae115fd3267e9b9efdb99aca445e7b18ba03
Closes-Bug: 1472212
The patch [1] sets the node's provision_state to DEPLOYWAIT while the
agent is booting and writing the the image onto the local disk. But in
the ironic-conductor we have a periodic task that checks for nodes in
DEPLOYWAIT state and see if the deployment timed out based on the node's
provision_udpated_at field and CONF.conductor.deploy_callback_timeout
configuration option.
The problem is that prior to this patch the node's provision_updated_at
field was only updated when we actually changed the provision state of
the node. So, if we are deployment a image which takes a long time to
be downloaded and written to the local disk ironic-conductor could time
out the deployment even if the agent still deploying that image.
This patch fixes this problem by touching the node's provision_updated_at
field when the node is heartbeating, so that we can indicate to the
ironic-conductor that the deployment is still running and we shouldn't
time it out.
[1] f1929f0155
Closes-Bug: #1475672
Change-Id: I9373c42168cc1ffaae212f17b067a6e4b6d862fe
The new EngineFacade system is the recommended way of working with
engine and sessions in oslo.db since 1.12.0. Most notable features
are:
- transparent thread-safe initialization of EngineFacade instance
- transaction scope defined by developer-specified context (no
need to pass Session instances to `private` DB API methods as
long as some kind of context is established: e.g. a Web-request
context or a module thread-local variable)
- declarative reader/writer transactions separation (e.g. allows to
perform read-only transactions on asynchronous DB replicas)
More details on EngineFacade:
http://specs.openstack.org/openstack/oslo-specs/specs/kilo/make-enginefacade-a-facade.html
Co-Authored-By: Victor Sergeyev <vsergeyev@mirantis.com>
Change-Id: I288e1e8e8c242520f12862f8ed6f3579687c5120
Add new microversion to hide this breaking change from innocent users.
Change-Id: I63618746b67cd30a4dacc933878f56fad533576b
Implements: blueprint enroll-node-state
Periodically checks the nodes in DEPLOYING and the state of the conductor
deploying them. If we find out that a conductor that was provisioning
the node has died we then break release the node and gracefully mark
the deployment as failed.
A new method called "get_offline_conductors" was added to the database
api which return a list of conductor hostnames that are offline.
Closes-Bug: #1461937
Change-Id: I293d3ff18e949b305e6404b8d3ceb75e29b88356
Ruby Loo