Add power state change notifications
This adds optional notifications emitted when ironic changes a node's power state or when ironic detects a change in a node's power state. These notifications can be consumed by any external service listening to the message bus to perform functions like tracking node power state changes over time or automatically responding to anomalous power states. The event_types of the new notifications are: * baremetal.node.power_set.{start,end,error} * baremetal.node.power_state_corrected.success This also adds a new NodePayload class for notification payloads related to nodes. Change-Id: I82702e7f959d666bb02b59d1fc53ab50b519cb74 Closes-Bug: 1526408
This commit is contained in:
parent
beb38b3f3f
commit
ff32b51bbf
@ -20,7 +20,7 @@ JSON object structured in the following way as defined by oslo.messaging::
|
||||
}
|
||||
|
||||
Versioned notifications in ironic
|
||||
---------------------------------
|
||||
=================================
|
||||
To make it easier for consumers of ironic's notifications to use predictably,
|
||||
ironic defines each notification and its payload as oslo versioned objects
|
||||
[2]_.
|
||||
@ -52,8 +52,8 @@ oslo (level, event_type and publisher_id). Below describes how to use these
|
||||
base classes to add a new notification to ironic.
|
||||
|
||||
Adding a new notification to ironic
|
||||
-----------------------------------
|
||||
To add a new notification to ironic, new versioned notification classes should
|
||||
===================================
|
||||
To add a new notification to ironic, a new versioned notification class should
|
||||
be created by subclassing the NotificationBase class to define the notification
|
||||
itself and the NotificationPayloadBase class to define which fields the new
|
||||
notification will contain inside its payload. You may also define a schema to
|
||||
@ -147,9 +147,10 @@ in the ironic notification base classes) and emit it::
|
||||
|
||||
notify = ExampleNotification(
|
||||
event_type=notification.EventType(object='example_obj',
|
||||
action='do_something', status='start'),
|
||||
publisher=notification.NotificationPublisher(service='conductor',
|
||||
host='cond-hostname01'),
|
||||
action='do_something', status=fields.NotificationStatus.START),
|
||||
publisher=notification.NotificationPublisher(
|
||||
service='ironic-conductor',
|
||||
host='hostname01'),
|
||||
level=fields.NotificationLevel.DEBUG,
|
||||
payload=my_notify_payload)
|
||||
notify.emit(context)
|
||||
@ -178,14 +179,137 @@ This example will send the following notification over the message bus::
|
||||
}
|
||||
},
|
||||
"event_type":"baremetal.example_obj.do_something.start",
|
||||
"publisher_id":"conductor.cond-hostname01"
|
||||
"publisher_id":"ironic-conductor.hostname01"
|
||||
}
|
||||
|
||||
Existing notifications
|
||||
----------------------
|
||||
|
||||
Descriptions of notifications emitted by ironic will be documented here when
|
||||
they are added.
|
||||
Available notifications
|
||||
=======================
|
||||
.. TODO(mariojv) Move the below to deployer documentation.
|
||||
.. TODO(mariojv) Match Nova's tabular formatting below.
|
||||
|
||||
|
||||
The notifications that ironic emits are described here. They are listed
|
||||
(alphabetically) by service first, then by event_type.
|
||||
|
||||
------------------------------
|
||||
ironic-conductor notifications
|
||||
------------------------------
|
||||
|
||||
|
||||
baremetal.node.power_set
|
||||
------------------------
|
||||
|
||||
* ``baremetal.node.power_set.start`` is emitted by the ironic-conductor service
|
||||
when it begins a power state change. It has notification level INFO.
|
||||
|
||||
* ``baremetal.node.power_set.end`` is emitted when ironic-conductor
|
||||
successfully completes a power state change task. It has notification level
|
||||
INFO.
|
||||
|
||||
* ``baremetal.node.power_set.error`` is emitted by ironic-conductor when it
|
||||
fails to set a node's power state. It has notification level ERROR. This can
|
||||
occur when ironic fails to retrieve the old power state prior to setting the
|
||||
new one on the node, or when it fails to set the power state if a change is
|
||||
requested.
|
||||
|
||||
Here is an example payload for a notification with this event type. The
|
||||
"to_power" payload field indicates the power state to which the
|
||||
ironic-conductor is attempting to change the node::
|
||||
|
||||
{
|
||||
"priority": "info",
|
||||
"payload":{
|
||||
"ironic_object.namespace":"ironic",
|
||||
"ironic_object.name":"NodeSetPowerStatePayload",
|
||||
"ironic_object.version":"1.0",
|
||||
"ironic_object.data":{
|
||||
"clean_step": None,
|
||||
"console_enabled": False,
|
||||
"created_at": "2016-01-26T20:41:03+00:00",
|
||||
"driver": "fake",
|
||||
"extra": {},
|
||||
"inspection_finished_at": None,
|
||||
"inspection_started_at": None,
|
||||
"instance_uuid": "d6ea00c1-1f94-4e95-90b3-3462d7031678",
|
||||
"last_error": None,
|
||||
"maintenance": False,
|
||||
"maintenance_reason": None,
|
||||
"network_interface": "flat",
|
||||
"name": None,
|
||||
"power_state": "power off",
|
||||
"properties": {
|
||||
"memory_mb": "4096",
|
||||
"cpu_arch": "x86_64',
|
||||
"local_gb": "10",
|
||||
"cpus": "8"},
|
||||
"provision_state": "available",
|
||||
"provision_updated_at": "2016-01-27T20:41:03+00:00",
|
||||
"resource_class": None,
|
||||
"target_power_state": None,
|
||||
"target_provision_state": None,
|
||||
"updated_at": "2016-01-27T20:41:03+00:00",
|
||||
"uuid": "1be26c0b-03f2-4d2e-ae87-c02d7f33c123",
|
||||
"to_power": "power on"
|
||||
}
|
||||
},
|
||||
"event_type":"baremetal.node.power_set.start",
|
||||
"publisher_id":"ironic-conductor.hostname01"
|
||||
}
|
||||
|
||||
|
||||
|
||||
baremetal.node.power_state_corrected
|
||||
------------------------------------
|
||||
|
||||
* ``baremetal.node.power_state_corrected.success`` is emitted by
|
||||
ironic-conductor when the power state on the baremetal hardware is different
|
||||
from the previous known power state of the node and the database is corrected
|
||||
to reflect this new power state. It has notification level INFO.
|
||||
|
||||
Here is an example payload for a notification with this event_type. The
|
||||
"from_power" payload field indicates the previous power state on the node,
|
||||
prior to the correction::
|
||||
|
||||
{
|
||||
"priority": "info",
|
||||
"payload":{
|
||||
"ironic_object.namespace":"ironic",
|
||||
"ironic_object.name":"NodeCorrectedPowerStatePayload",
|
||||
"ironic_object.version":"1.0",
|
||||
"ironic_object.data":{
|
||||
"clean_step": None,
|
||||
"console_enabled": False,
|
||||
"created_at": "2016-01-26T20:41:03+00:00",
|
||||
"driver": "fake",
|
||||
"extra": {},
|
||||
"inspection_finished_at": None,
|
||||
"inspection_started_at": None,
|
||||
"instance_uuid": "d6ea00c1-1f94-4e95-90b3-3462d7031678",
|
||||
"last_error": None,
|
||||
"maintenance": False,
|
||||
"maintenance_reason": None,
|
||||
"network_interface": "flat",
|
||||
"name": None,
|
||||
"power_state": "power off",
|
||||
"properties": {
|
||||
"memory_mb": "4096",
|
||||
"cpu_arch": "x86_64',
|
||||
"local_gb": "10",
|
||||
"cpus": "8"},
|
||||
"provision_state": "available",
|
||||
"provision_updated_at": "2016-01-27T20:41:03+00:00",
|
||||
"resource_class": None,
|
||||
"target_power_state": None,
|
||||
"target_provision_state": None,
|
||||
"updated_at": "2016-01-27T20:41:03+00:00",
|
||||
"uuid": "1be26c0b-03f2-4d2e-ae87-c02d7f33c123",
|
||||
"from_power": "power on"
|
||||
}
|
||||
},
|
||||
"event_type":"baremetal.node.power_state_corrected.success",
|
||||
"publisher_id":"ironic-conductor.cond-hostname02"
|
||||
}
|
||||
|
||||
.. [1] http://docs.openstack.org/developer/oslo.messaging/notifier.html
|
||||
.. [2] http://docs.openstack.org/developer/oslo.versionedobjects
|
||||
|
@ -62,6 +62,7 @@ from ironic.common import images
|
||||
from ironic.common import states
|
||||
from ironic.common import swift
|
||||
from ironic.conductor import base_manager
|
||||
from ironic.conductor import notification_utils as notify_utils
|
||||
from ironic.conductor import task_manager
|
||||
from ironic.conductor import utils
|
||||
from ironic.conf import CONF
|
||||
@ -953,8 +954,14 @@ class ConductorManager(base_manager.BaseConductorManager):
|
||||
{'node': node.uuid, 'msg': e})
|
||||
|
||||
if error is None:
|
||||
node.power_state = power_state
|
||||
task.process_event('done')
|
||||
if power_state != node.power_state:
|
||||
old_power_state = node.power_state
|
||||
node.power_state = power_state
|
||||
task.process_event('done')
|
||||
notify_utils.emit_power_state_corrected_notification(
|
||||
task, old_power_state)
|
||||
else:
|
||||
task.process_event('done')
|
||||
else:
|
||||
LOG.error(error)
|
||||
node.last_error = error
|
||||
@ -2434,11 +2441,15 @@ def handle_sync_power_state_max_retries_exceeded(task, actual_power_state,
|
||||
if exception is not None:
|
||||
msg += _(" Error: %s") % exception
|
||||
|
||||
old_power_state = node.power_state
|
||||
node.power_state = actual_power_state
|
||||
node.last_error = msg
|
||||
node.maintenance = True
|
||||
node.maintenance_reason = msg
|
||||
node.save()
|
||||
if old_power_state != actual_power_state:
|
||||
notify_utils.emit_power_state_corrected_notification(
|
||||
task, old_power_state)
|
||||
LOG.error(msg)
|
||||
|
||||
|
||||
@ -2457,6 +2468,7 @@ def do_sync_power_state(task, count):
|
||||
On failure, the count is incremented by one
|
||||
"""
|
||||
node = task.node
|
||||
old_power_state = node.power_state
|
||||
power_state = None
|
||||
count += 1
|
||||
|
||||
@ -2509,13 +2521,15 @@ def do_sync_power_state(task, count):
|
||||
return 0
|
||||
elif node.power_state is None:
|
||||
# If node has no prior state AND we successfully got a state,
|
||||
# simply record that.
|
||||
# simply record that and send a notification.
|
||||
LOG.info(_LI("During sync_power_state, node %(node)s has no "
|
||||
"previous known state. Recording current state "
|
||||
"'%(state)s'."),
|
||||
{'node': node.uuid, 'state': power_state})
|
||||
node.power_state = power_state
|
||||
node.save()
|
||||
notify_utils.emit_power_state_corrected_notification(
|
||||
task, None)
|
||||
return 0
|
||||
|
||||
if count > max_retries:
|
||||
@ -2548,6 +2562,8 @@ def do_sync_power_state(task, count):
|
||||
'state': node.power_state})
|
||||
node.power_state = power_state
|
||||
node.save()
|
||||
notify_utils.emit_power_state_corrected_notification(
|
||||
task, old_power_state)
|
||||
|
||||
return count
|
||||
|
||||
|
131
ironic/conductor/notification_utils.py
Normal file
131
ironic/conductor/notification_utils.py
Normal file
@ -0,0 +1,131 @@
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from oslo_config import cfg
|
||||
from oslo_log import log
|
||||
from oslo_messaging import exceptions as oslo_msg_exc
|
||||
from oslo_versionedobjects import exception as oslo_vo_exc
|
||||
|
||||
from ironic.common import exception
|
||||
from ironic.common.i18n import _
|
||||
from ironic.objects import fields
|
||||
from ironic.objects import node as node_objects
|
||||
from ironic.objects import notification
|
||||
|
||||
LOG = log.getLogger(__name__)
|
||||
CONF = cfg.CONF
|
||||
|
||||
|
||||
def _emit_conductor_node_notification(task, notification_method,
|
||||
payload_method, action,
|
||||
level, status, **kwargs):
|
||||
"""Helper for emitting a conductor notification about a node.
|
||||
|
||||
:param task: a TaskManager instance.
|
||||
:param notification_method: Constructor for the notification itself.
|
||||
:param payload_method: Constructor for the notification payload. Node
|
||||
should be first argument of the method.
|
||||
:param action: Action string to go in the EventType.
|
||||
:param level: Notification level. One of
|
||||
`ironic.objects.fields.NotificationLevel.ALL`
|
||||
:param status: Status to go in the EventType. One of
|
||||
`ironic.objects.fields.NotificationStatus.ALL`
|
||||
:param **kwargs: kwargs to use when creating the notification payload.
|
||||
Passed to the payload_method.
|
||||
"""
|
||||
try:
|
||||
# Prepare our exception message just in case
|
||||
exception_values = {"node": task.node.uuid,
|
||||
"action": action,
|
||||
"status": status,
|
||||
"level": level,
|
||||
"notification_method":
|
||||
notification_method.__name__,
|
||||
"payload_method": payload_method.__name__}
|
||||
exception_message = (_("Failed to send baremetal.node."
|
||||
"%(action)s.%(status)s notification for node "
|
||||
"%(node)s with level %(level)s, "
|
||||
"notification_method %(notification_method)s, "
|
||||
"payload_method %(payload_method)s, error "
|
||||
"%(error)s"))
|
||||
payload = payload_method(task.node, **kwargs)
|
||||
notification_method(
|
||||
publisher=notification.NotificationPublisher(
|
||||
service='ironic-conductor', host=CONF.host),
|
||||
event_type=notification.EventType(
|
||||
object='node', action=action, status=status),
|
||||
level=level,
|
||||
payload=payload).emit(task.context)
|
||||
except (exception.NotificationSchemaObjectError,
|
||||
exception.NotificationSchemaKeyError,
|
||||
exception.NotificationPayloadError,
|
||||
oslo_msg_exc.MessageDeliveryFailure,
|
||||
oslo_vo_exc.VersionedObjectsException) as e:
|
||||
exception_values['error'] = e
|
||||
LOG.warning(exception_message, exception_values)
|
||||
except Exception as e:
|
||||
# NOTE(mariojv) For unknown exceptions, also log the traceback.
|
||||
exception_values['error'] = e
|
||||
LOG.exception(exception_message, exception_values)
|
||||
|
||||
|
||||
def emit_power_set_notification(task, level, status, to_power):
|
||||
"""Helper for conductor sending a set power state notification.
|
||||
|
||||
:param task: a TaskManager instance.
|
||||
:param level: Notification level. One of
|
||||
`ironic.objects.fields.NotificationLevel.ALL`
|
||||
:param status: Status to go in the EventType. One of
|
||||
`ironic.objects.fields.NotificationStatus.SUCCESS` or ERROR.
|
||||
ERROR indicates that ironic-conductor couldn't retrieve the
|
||||
power state for this node, or that it couldn't set the power
|
||||
state of the node.
|
||||
:param to_power: the power state the conductor is
|
||||
attempting to set on the node. This is used
|
||||
instead of the node's target_power_state
|
||||
attribute since the "baremetal.node.power_set.start"
|
||||
notification is sent early, before target_power_state
|
||||
is set on the node.
|
||||
"""
|
||||
_emit_conductor_node_notification(
|
||||
task,
|
||||
node_objects.NodeSetPowerStateNotification,
|
||||
node_objects.NodeSetPowerStatePayload,
|
||||
'power_set',
|
||||
level,
|
||||
status,
|
||||
to_power=to_power
|
||||
)
|
||||
|
||||
|
||||
def emit_power_state_corrected_notification(task, from_power):
|
||||
"""Helper for conductor sending a node power state corrected notification.
|
||||
|
||||
When ironic detects that the actual power state on a bare metal hardware
|
||||
is different from the power state on an ironic node (DB), the ironic
|
||||
node's power state is corrected to be that of the bare metal hardware.
|
||||
A notification is emitted about this after the database is updated to
|
||||
reflect this correction.
|
||||
|
||||
:param task: a TaskManager instance.
|
||||
:param from_power: the power state of the node before this change was
|
||||
detected
|
||||
"""
|
||||
_emit_conductor_node_notification(
|
||||
task,
|
||||
node_objects.NodeCorrectedPowerStateNotification,
|
||||
node_objects.NodeCorrectedPowerStatePayload,
|
||||
'power_state_corrected',
|
||||
fields.NotificationLevel.INFO,
|
||||
fields.NotificationStatus.SUCCESS,
|
||||
from_power=from_power
|
||||
)
|
@ -12,15 +12,19 @@
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from oslo_config import cfg
|
||||
from oslo_log import log
|
||||
from oslo_utils import excutils
|
||||
|
||||
from ironic.common import exception
|
||||
from ironic.common.i18n import _, _LE, _LI, _LW
|
||||
from ironic.common import states
|
||||
from ironic.conductor import notification_utils as notify_utils
|
||||
from ironic.conductor import task_manager
|
||||
from ironic.objects import fields
|
||||
|
||||
LOG = log.getLogger(__name__)
|
||||
CONF = cfg.CONF
|
||||
|
||||
CLEANING_INTERFACE_PRIORITY = {
|
||||
# When two clean steps have the same priority, their order is determined
|
||||
@ -78,6 +82,9 @@ def node_power_action(task, new_state):
|
||||
wrong occurred during the power action.
|
||||
|
||||
"""
|
||||
notify_utils.emit_power_set_notification(
|
||||
task, fields.NotificationLevel.INFO, fields.NotificationStatus.START,
|
||||
new_state)
|
||||
node = task.node
|
||||
target_state = states.POWER_ON if new_state == states.REBOOT else new_state
|
||||
|
||||
@ -91,6 +98,9 @@ def node_power_action(task, new_state):
|
||||
"Error: %(error)s") % {'target': new_state, 'error': e}
|
||||
node['target_power_state'] = states.NOSTATE
|
||||
node.save()
|
||||
notify_utils.emit_power_set_notification(
|
||||
task, fields.NotificationLevel.ERROR,
|
||||
fields.NotificationStatus.ERROR, new_state)
|
||||
|
||||
if curr_state == new_state:
|
||||
# Neither the ironic service nor the hardware has erred. The
|
||||
@ -107,6 +117,9 @@ def node_power_action(task, new_state):
|
||||
node['power_state'] = new_state
|
||||
node['target_power_state'] = states.NOSTATE
|
||||
node.save()
|
||||
notify_utils.emit_power_set_notification(
|
||||
task, fields.NotificationLevel.INFO,
|
||||
fields.NotificationStatus.END, new_state)
|
||||
LOG.warning(_LW("Not going to change node %(node)s power "
|
||||
"state because current state = requested state "
|
||||
"= '%(state)s'."),
|
||||
@ -134,18 +147,25 @@ def node_power_action(task, new_state):
|
||||
task.driver.power.reboot(task)
|
||||
except Exception as e:
|
||||
with excutils.save_and_reraise_exception():
|
||||
node['target_power_state'] = states.NOSTATE
|
||||
node['last_error'] = _(
|
||||
"Failed to change power state to '%(target)s'. "
|
||||
"Error: %(error)s") % {'target': target_state, 'error': e}
|
||||
node.save()
|
||||
notify_utils.emit_power_set_notification(
|
||||
task, fields.NotificationLevel.ERROR,
|
||||
fields.NotificationStatus.ERROR, new_state)
|
||||
else:
|
||||
# success!
|
||||
node['power_state'] = target_state
|
||||
node['target_power_state'] = states.NOSTATE
|
||||
node.save()
|
||||
notify_utils.emit_power_set_notification(
|
||||
task, fields.NotificationLevel.INFO, fields.NotificationStatus.END,
|
||||
new_state)
|
||||
LOG.info(_LI('Successfully set node %(node)s power state to '
|
||||
'%(state)s.'),
|
||||
{'node': node.uuid, 'state': target_state})
|
||||
finally:
|
||||
node['target_power_state'] = states.NOSTATE
|
||||
node.save()
|
||||
|
||||
|
||||
@task_manager.require_exclusive_lock
|
||||
@ -276,6 +296,9 @@ def power_state_error_handler(e, node, power_state):
|
||||
:param power_state: the power state to set on the node.
|
||||
|
||||
"""
|
||||
# NOTE This error will not emit a power state change notification since
|
||||
# this is related to spawning the worker thread, not the power state change
|
||||
# itself.
|
||||
if isinstance(e, exception.NoFreeConductorWorker):
|
||||
node.power_state = power_state
|
||||
node.target_power_state = states.NOSTATE
|
||||
|
@ -124,6 +124,23 @@ class NotificationLevelField(object_fields.BaseEnumField):
|
||||
AUTO_TYPE = NotificationLevel()
|
||||
|
||||
|
||||
class NotificationStatus(object_fields.Enum):
|
||||
START = 'start'
|
||||
END = 'end'
|
||||
ERROR = 'error'
|
||||
SUCCESS = 'success'
|
||||
|
||||
ALL = (START, END, ERROR, SUCCESS)
|
||||
|
||||
def __init__(self):
|
||||
super(NotificationStatus, self).__init__(
|
||||
valid_values=NotificationStatus.ALL)
|
||||
|
||||
|
||||
class NotificationStatusField(object_fields.BaseEnumField):
|
||||
AUTO_TYPE = NotificationStatus()
|
||||
|
||||
|
||||
class MACAddress(object_fields.FieldType):
|
||||
@staticmethod
|
||||
def coerce(obj, attr, value):
|
||||
|
@ -23,6 +23,7 @@ from ironic.conf import CONF
|
||||
from ironic.db import api as db_api
|
||||
from ironic.objects import base
|
||||
from ironic.objects import fields as object_fields
|
||||
from ironic.objects import notification
|
||||
|
||||
REQUIRED_INT_PROPERTIES = ['local_gb', 'cpus', 'memory_mb']
|
||||
|
||||
@ -395,3 +396,137 @@ class Node(base.IronicObject, object_base.VersionedObjectDictCompat):
|
||||
db_node = cls.dbapi.get_node_by_port_addresses(addresses)
|
||||
node = Node._from_db_object(cls(context), db_node)
|
||||
return node
|
||||
|
||||
|
||||
@base.IronicObjectRegistry.register
|
||||
class NodePayload(notification.NotificationPayloadBase):
|
||||
"""Base class used for all notification payloads about a Node object."""
|
||||
# NOTE: This payload does not include the Node fields "chassis_id",
|
||||
# "driver_info", "driver_internal_info", "instance_info", "raid_config",
|
||||
# "reservation", or "target_raid_config". These were excluded for reasons
|
||||
# including:
|
||||
# - increased complexity needed for creating the payload
|
||||
# - sensitive information in the fields that shouldn't be exposed to
|
||||
# external services
|
||||
# - being internal-only or hardware-related fields
|
||||
SCHEMA = {
|
||||
'clean_step': ('node', 'clean_step'),
|
||||
'console_enabled': ('node', 'console_enabled'),
|
||||
'created_at': ('node', 'created_at'),
|
||||
'driver': ('node', 'driver'),
|
||||
'extra': ('node', 'extra'),
|
||||
'inspection_finished_at': ('node', 'inspection_finished_at'),
|
||||
'inspection_started_at': ('node', 'inspection_started_at'),
|
||||
'instance_uuid': ('node', 'instance_uuid'),
|
||||
'last_error': ('node', 'last_error'),
|
||||
'maintenance': ('node', 'maintenance'),
|
||||
'maintenance_reason': ('node', 'maintenance_reason'),
|
||||
'name': ('node', 'name'),
|
||||
'network_interface': ('node', 'network_interface'),
|
||||
'power_state': ('node', 'power_state'),
|
||||
'properties': ('node', 'properties'),
|
||||
'provision_state': ('node', 'provision_state'),
|
||||
'provision_updated_at': ('node', 'provision_updated_at'),
|
||||
'resource_class': ('node', 'resource_class'),
|
||||
'target_power_state': ('node', 'target_power_state'),
|
||||
'target_provision_state': ('node', 'target_provision_state'),
|
||||
'updated_at': ('node', 'updated_at'),
|
||||
'uuid': ('node', 'uuid')
|
||||
}
|
||||
# Version 1.0: Initial version, based off of Node version 1.18.
|
||||
VERSION = '1.0'
|
||||
fields = {
|
||||
'clean_step': object_fields.FlexibleDictField(nullable=True),
|
||||
'console_enabled': object_fields.BooleanField(),
|
||||
'created_at': object_fields.DateTimeField(nullable=True),
|
||||
'driver': object_fields.StringField(nullable=True),
|
||||
'extra': object_fields.FlexibleDictField(nullable=True),
|
||||
'inspection_finished_at': object_fields.DateTimeField(nullable=True),
|
||||
'inspection_started_at': object_fields.DateTimeField(nullable=True),
|
||||
'instance_uuid': object_fields.UUIDField(nullable=True),
|
||||
'last_error': object_fields.StringField(nullable=True),
|
||||
'maintenance': object_fields.BooleanField(),
|
||||
'maintenance_reason': object_fields.StringField(nullable=True),
|
||||
'network_interface': object_fields.StringFieldThatAcceptsCallable(),
|
||||
'name': object_fields.StringField(nullable=True),
|
||||
'power_state': object_fields.StringField(nullable=True),
|
||||
'properties': object_fields.FlexibleDictField(nullable=True),
|
||||
'provision_state': object_fields.StringField(nullable=True),
|
||||
'provision_updated_at': object_fields.DateTimeField(nullable=True),
|
||||
'resource_class': object_fields.StringField(nullable=True),
|
||||
'target_power_state': object_fields.StringField(nullable=True),
|
||||
'target_provision_state': object_fields.StringField(nullable=True),
|
||||
'updated_at': object_fields.DateTimeField(nullable=True),
|
||||
'uuid': object_fields.UUIDField()
|
||||
}
|
||||
|
||||
def __init__(self, node, **kwargs):
|
||||
super(NodePayload, self).__init__(**kwargs)
|
||||
self.populate_schema(node=node)
|
||||
|
||||
|
||||
@base.IronicObjectRegistry.register
|
||||
class NodeSetPowerStateNotification(notification.NotificationBase):
|
||||
"""Notification emitted when ironic changes a node's power state."""
|
||||
# Version 1.0: Initial version
|
||||
VERSION = '1.0'
|
||||
|
||||
fields = {
|
||||
'payload': object_fields.ObjectField('NodeSetPowerStatePayload')
|
||||
}
|
||||
|
||||
|
||||
@base.IronicObjectRegistry.register
|
||||
class NodeSetPowerStatePayload(NodePayload):
|
||||
"""Payload schema for when ironic changes a node's power state."""
|
||||
# Version 1.0: Initial version
|
||||
VERSION = '1.0'
|
||||
|
||||
fields = {
|
||||
# "to_power" indicates the future target_power_state of the node. A
|
||||
# separate field from target_power_state is used so that the
|
||||
# baremetal.node.power_set.start notification, which is sent before
|
||||
# target_power_state is set on the node, has information about what
|
||||
# state the conductor will attempt to set on the node.
|
||||
'to_power': object_fields.StringField(nullable=True)
|
||||
}
|
||||
|
||||
def __init__(self, node, to_power):
|
||||
super(NodeSetPowerStatePayload, self).__init__(
|
||||
node, to_power=to_power)
|
||||
|
||||
|
||||
@base.IronicObjectRegistry.register
|
||||
class NodeCorrectedPowerStateNotification(notification.NotificationBase):
|
||||
"""Notification for when a node's power state is corrected in the database.
|
||||
|
||||
This notification is emitted when ironic detects that the actual power
|
||||
state on a bare metal hardware is different from the power state on an
|
||||
ironic node (DB). This notification is emitted after the database is
|
||||
updated to reflect this correction.
|
||||
"""
|
||||
# Version 1.0: Initial version
|
||||
VERSION = '1.0'
|
||||
|
||||
fields = {
|
||||
'payload': object_fields.ObjectField('NodeCorrectedPowerStatePayload')
|
||||
}
|
||||
|
||||
|
||||
@base.IronicObjectRegistry.register
|
||||
class NodeCorrectedPowerStatePayload(NodePayload):
|
||||
"""Notification payload schema for when a node's power state is corrected.
|
||||
|
||||
"from_power" indicates the previous power state on the ironic node
|
||||
before the node was updated.
|
||||
"""
|
||||
# Version 1.0: Initial version
|
||||
VERSION = '1.0'
|
||||
|
||||
fields = {
|
||||
'from_power': object_fields.StringField(nullable=True)
|
||||
}
|
||||
|
||||
def __init__(self, node, from_power):
|
||||
super(NodeCorrectedPowerStatePayload, self).__init__(
|
||||
node, from_power=from_power)
|
||||
|
@ -45,12 +45,18 @@ class EventType(base.IronicObject):
|
||||
fields = {
|
||||
'object': fields.StringField(nullable=False),
|
||||
'action': fields.StringField(nullable=False),
|
||||
'status': fields.EnumField(valid_values=['start', 'end', 'error',
|
||||
'success'],
|
||||
nullable=False)
|
||||
'status': fields.NotificationStatusField()
|
||||
}
|
||||
|
||||
def to_event_type_field(self):
|
||||
"""Constructs string for event_type to be sent on the wire.
|
||||
|
||||
The string is in the format: baremetal.<object>.<action>.<status>
|
||||
|
||||
:raises: ValueError if self.status is not one of
|
||||
:class:`fields.NotificationStatusField`
|
||||
:returns: event_type string
|
||||
"""
|
||||
parts = ['baremetal', self.object, self.action, self.status]
|
||||
return '.'.join(parts)
|
||||
|
||||
@ -91,7 +97,11 @@ class NotificationBase(base.IronicObject):
|
||||
NOTIFY_LEVELS[CONF.notification_level])
|
||||
|
||||
def emit(self, context):
|
||||
"""Send the notification."""
|
||||
"""Send the notification.
|
||||
|
||||
:raises NotificationPayloadError
|
||||
:raises oslo_versionedobjects.exceptions.MessageDeliveryFailure
|
||||
"""
|
||||
if not self._should_notify():
|
||||
return
|
||||
if not self.payload.populated:
|
||||
@ -132,6 +142,8 @@ class NotificationPayloadBase(base.IronicObject):
|
||||
|
||||
:param kwargs: A dict contains the source object and the keys defined
|
||||
in the SCHEMA
|
||||
:raises NotificationSchemaObjectError
|
||||
:raises NotificationSchemaKeyError
|
||||
"""
|
||||
for key, (obj, field) in self.SCHEMA.items():
|
||||
try:
|
||||
|
@ -168,3 +168,20 @@ class TestCase(testtools.TestCase):
|
||||
"""Asserts that 2 complex data structures are json equivalent."""
|
||||
self.assertEqual(jsonutils.dumps(expected, sort_keys=True),
|
||||
jsonutils.dumps(observed, sort_keys=True))
|
||||
|
||||
def assertNotificationEqual(self, notif_args, service, host, event_type,
|
||||
level):
|
||||
"""Asserts properties of arguments passed when creating a notification.
|
||||
|
||||
:param notif_args: dict of arguments notification instantiated with
|
||||
:param service: expected service that emits the notification
|
||||
:param host: expected host that emits the notification
|
||||
:param event_type: expected value of EventType field of notification
|
||||
as a string
|
||||
:param level: expected NotificationLevel
|
||||
"""
|
||||
self.assertEqual(service, notif_args['publisher'].service)
|
||||
self.assertEqual(host, notif_args['publisher'].host)
|
||||
self.assertEqual(event_type, notif_args['event_type'].
|
||||
to_event_type_field())
|
||||
self.assertEqual(level, notif_args['level'])
|
||||
|
@ -42,6 +42,7 @@ from ironic.drivers import base as drivers_base
|
||||
from ironic.drivers.modules import fake
|
||||
from ironic import objects
|
||||
from ironic.objects import base as obj_base
|
||||
from ironic.objects import fields as obj_fields
|
||||
from ironic.tests import base as tests_base
|
||||
from ironic.tests.unit.conductor import mgr_utils
|
||||
from ironic.tests.unit.db import base as tests_db_base
|
||||
@ -201,6 +202,182 @@ class ChangeNodePowerStateTestCase(mgr_utils.ServiceSetUpMixin,
|
||||
self.assertIsNone(node.target_power_state)
|
||||
self.assertIsNone(node.last_error)
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_set_power_state_notif_success(self, mock_notif):
|
||||
# Test that successfully changing a node's power state sends the
|
||||
# correct .start and .end notifications
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
driver='fake',
|
||||
power_state=states.POWER_OFF)
|
||||
|
||||
self._start_service()
|
||||
self.service.change_node_power_state(self.context,
|
||||
node.uuid,
|
||||
states.POWER_ON)
|
||||
# Give async worker a chance to finish
|
||||
self._stop_service()
|
||||
|
||||
# 2 notifications should be sent: 1 .start and 1 .end
|
||||
self.assertEqual(2, mock_notif.call_count)
|
||||
self.assertEqual(2, mock_notif.return_value.emit.call_count)
|
||||
|
||||
first_notif_args = mock_notif.call_args_list[0][1]
|
||||
second_notif_args = mock_notif.call_args_list[1][1]
|
||||
|
||||
self.assertNotificationEqual(first_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.start',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
self.assertNotificationEqual(second_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.end',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_set_power_state_notif_get_power_fail(self, mock_notif):
|
||||
# Test that correct notifications are sent when changing node power
|
||||
# state and retrieving the node's current power state fails
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
driver='fake',
|
||||
power_state=states.POWER_OFF)
|
||||
self._start_service()
|
||||
|
||||
with mock.patch.object(self.driver.power,
|
||||
'get_power_state') as get_power_mock:
|
||||
get_power_mock.side_effect = Exception('I have failed')
|
||||
self.service.change_node_power_state(self.context,
|
||||
node.uuid,
|
||||
states.POWER_ON)
|
||||
# Give async worker a chance to finish
|
||||
self._stop_service()
|
||||
|
||||
get_power_mock.assert_called_once_with(mock.ANY)
|
||||
|
||||
# 2 notifications should be sent: 1 .start and 1 .error
|
||||
self.assertEqual(2, mock_notif.call_count)
|
||||
self.assertEqual(2, mock_notif.return_value.emit.call_count)
|
||||
|
||||
first_notif_args = mock_notif.call_args_list[0][1]
|
||||
second_notif_args = mock_notif.call_args_list[1][1]
|
||||
|
||||
self.assertNotificationEqual(first_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.start',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
self.assertNotificationEqual(second_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.error',
|
||||
obj_fields.NotificationLevel.ERROR)
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_set_power_state_notif_set_power_fail(self, mock_notif):
|
||||
# Test that correct notifications are sent when changing node power
|
||||
# state and setting the node's power state fails
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
driver='fake',
|
||||
power_state=states.POWER_OFF)
|
||||
self._start_service()
|
||||
|
||||
with mock.patch.object(self.driver.power,
|
||||
'set_power_state') as set_power_mock:
|
||||
set_power_mock.side_effect = Exception('I have failed')
|
||||
self.service.change_node_power_state(self.context,
|
||||
node.uuid,
|
||||
states.POWER_ON)
|
||||
# Give async worker a chance to finish
|
||||
self._stop_service()
|
||||
|
||||
set_power_mock.assert_called_once_with(mock.ANY, states.POWER_ON)
|
||||
|
||||
# 2 notifications should be sent: 1 .start and 1 .error
|
||||
self.assertEqual(2, mock_notif.call_count)
|
||||
self.assertEqual(2, mock_notif.return_value.emit.call_count)
|
||||
|
||||
first_notif_args = mock_notif.call_args_list[0][1]
|
||||
second_notif_args = mock_notif.call_args_list[1][1]
|
||||
|
||||
self.assertNotificationEqual(first_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.start',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
self.assertNotificationEqual(second_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.error',
|
||||
obj_fields.NotificationLevel.ERROR)
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_set_power_state_notif_spawn_fail(self, mock_notif):
|
||||
# Test that failure notification is not sent when spawning the
|
||||
# background conductor worker fails
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
driver='fake',
|
||||
power_state=states.POWER_OFF)
|
||||
|
||||
self._start_service()
|
||||
with mock.patch.object(self.service,
|
||||
'_spawn_worker') as spawn_mock:
|
||||
spawn_mock.side_effect = exception.NoFreeConductorWorker()
|
||||
self.assertRaises(messaging.rpc.ExpectedException,
|
||||
self.service.change_node_power_state,
|
||||
self.context,
|
||||
node.uuid,
|
||||
states.POWER_ON)
|
||||
|
||||
spawn_mock.assert_called_once_with(
|
||||
conductor_utils.node_power_action, mock.ANY, states.POWER_ON)
|
||||
self.assertFalse(mock_notif.called)
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_set_power_state_notif_no_state_change(self, mock_notif):
|
||||
# Test that correct notifications are sent when changing node power
|
||||
# state and no state change is necessary
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
driver='fake',
|
||||
power_state=states.POWER_OFF)
|
||||
|
||||
self._start_service()
|
||||
self.service.change_node_power_state(self.context,
|
||||
node.uuid,
|
||||
states.POWER_OFF)
|
||||
# Give async worker a chance to finish
|
||||
self._stop_service()
|
||||
|
||||
# 2 notifications should be sent: 1 .start and 1 .end
|
||||
self.assertEqual(2, mock_notif.call_count)
|
||||
self.assertEqual(2, mock_notif.return_value.emit.call_count)
|
||||
|
||||
first_notif_args = mock_notif.call_args_list[0][1]
|
||||
second_notif_args = mock_notif.call_args_list[1][1]
|
||||
|
||||
self.assertNotificationEqual(first_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.start',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
self.assertNotificationEqual(second_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.end',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
|
||||
|
||||
@mgr_utils.mock_record_keepalive
|
||||
class UpdateNodeTestCase(mgr_utils.ServiceSetUpMixin,
|
||||
@ -2255,10 +2432,14 @@ class DoNodeCleanTestCase(mgr_utils.ServiceSetUpMixin,
|
||||
@mgr_utils.mock_record_keepalive
|
||||
class DoNodeVerifyTestCase(mgr_utils.ServiceSetUpMixin,
|
||||
tests_db_base.DbTestCase):
|
||||
@mock.patch('ironic.objects.node.NodeCorrectedPowerStateNotification')
|
||||
@mock.patch('ironic.drivers.modules.fake.FakePower.get_power_state')
|
||||
@mock.patch('ironic.drivers.modules.fake.FakePower.validate')
|
||||
def test__do_node_verify(self, mock_validate, mock_get_power_state):
|
||||
def test__do_node_verify(self, mock_validate, mock_get_power_state,
|
||||
mock_notif):
|
||||
mock_get_power_state.return_value = states.POWER_OFF
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeCorrectedPowerStateNotification'
|
||||
node = obj_utils.create_test_node(
|
||||
self.context, driver='fake',
|
||||
provision_state=states.VERIFYING,
|
||||
@ -2272,6 +2453,15 @@ class DoNodeVerifyTestCase(mgr_utils.ServiceSetUpMixin,
|
||||
self.service._do_node_verify(task)
|
||||
|
||||
self._stop_service()
|
||||
|
||||
# 1 notification should be sent -
|
||||
# baremetal.node.power_state_corrected.success
|
||||
mock_notif.assert_called_once_with(publisher=mock.ANY,
|
||||
event_type=mock.ANY,
|
||||
level=mock.ANY,
|
||||
payload=mock.ANY)
|
||||
mock_notif.return_value.emit.assert_called_once_with(mock.ANY)
|
||||
|
||||
node.refresh()
|
||||
|
||||
mock_validate.assert_called_once_with(task)
|
||||
@ -3438,6 +3628,34 @@ class ManagerDoSyncPowerStateTestCase(tests_db_base.DbTestCase):
|
||||
self.assertEqual(states.POWER_OFF, self.node.power_state)
|
||||
self.task.upgrade_lock.assert_called_once_with()
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeCorrectedPowerStateNotification')
|
||||
def test_state_changed_no_sync_notify(self, mock_notif, node_power_action):
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeCorrectedPowerStateNotification'
|
||||
|
||||
self._do_sync_power_state(states.POWER_ON, states.POWER_OFF)
|
||||
|
||||
self.assertFalse(self.power.validate.called)
|
||||
self.power.get_power_state.assert_called_once_with(self.task)
|
||||
self.assertFalse(node_power_action.called)
|
||||
self.assertEqual(states.POWER_OFF, self.node.power_state)
|
||||
self.task.upgrade_lock.assert_called_once_with()
|
||||
|
||||
# 1 notification should be sent:
|
||||
# baremetal.node.power_state_updated.success, indicating the DB was
|
||||
# updated to reflect the actual node power state
|
||||
mock_notif.assert_called_once_with(publisher=mock.ANY,
|
||||
event_type=mock.ANY,
|
||||
level=mock.ANY,
|
||||
payload=mock.ANY)
|
||||
mock_notif.return_value.emit.assert_called_once_with(mock.ANY)
|
||||
|
||||
notif_args = mock_notif.call_args[1]
|
||||
self.assertNotificationEqual(
|
||||
notif_args, 'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_state_corrected.success',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
|
||||
def test_state_changed_sync(self, node_power_action):
|
||||
self.config(force_power_state_during_sync=True, group='conductor')
|
||||
self.config(power_state_sync_max_retries=1, group='conductor')
|
||||
@ -3501,6 +3719,30 @@ class ManagerDoSyncPowerStateTestCase(tests_db_base.DbTestCase):
|
||||
self.service.power_state_sync_count[self.node.uuid])
|
||||
self.assertTrue(self.node.maintenance)
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeCorrectedPowerStateNotification')
|
||||
def test_max_retries_exceeded_notify(self, mock_notif, node_power_action):
|
||||
self.config(force_power_state_during_sync=True, group='conductor')
|
||||
self.config(power_state_sync_max_retries=1, group='conductor')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeCorrectedPowerStateNotification'
|
||||
|
||||
self._do_sync_power_state(states.POWER_ON, [states.POWER_OFF,
|
||||
states.POWER_OFF])
|
||||
# 1 notification should be sent:
|
||||
# baremetal.node.power_state_corrected.success, indicating
|
||||
# the DB was updated to reflect the actual node power state
|
||||
mock_notif.assert_called_once_with(publisher=mock.ANY,
|
||||
event_type=mock.ANY,
|
||||
level=mock.ANY,
|
||||
payload=mock.ANY)
|
||||
mock_notif.return_value.emit.assert_called_once_with(mock.ANY)
|
||||
|
||||
notif_args = mock_notif.call_args[1]
|
||||
self.assertNotificationEqual(
|
||||
notif_args, 'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_state_corrected.success',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
|
||||
def test_retry_then_success(self, node_power_action):
|
||||
self.config(force_power_state_during_sync=True, group='conductor')
|
||||
self.config(power_state_sync_max_retries=2, group='conductor')
|
||||
|
144
ironic/tests/unit/conductor/test_notification_utils.py
Normal file
144
ironic/tests/unit/conductor/test_notification_utils.py
Normal file
@ -0,0 +1,144 @@
|
||||
# Copyright 2016 Rackspace, Inc.
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
"""Test class for ironic-conductor notification utilities."""
|
||||
|
||||
import mock
|
||||
from oslo_versionedobjects.exception import VersionedObjectsException
|
||||
|
||||
from ironic.common import exception
|
||||
from ironic.common import states
|
||||
from ironic.conductor import notification_utils as notif_utils
|
||||
from ironic.objects import fields
|
||||
from ironic.objects import node as node_objects
|
||||
from ironic.tests.unit.db import base
|
||||
from ironic.tests.unit.objects import utils as obj_utils
|
||||
|
||||
|
||||
class TestNotificationUtils(base.DbTestCase):
|
||||
def setUp(self):
|
||||
super(TestNotificationUtils, self).setUp()
|
||||
self.config(notification_level='debug')
|
||||
self.node = obj_utils.create_test_node(self.context)
|
||||
self.task = mock.Mock(spec_set=['context', 'driver', 'node',
|
||||
'upgrade_lock', 'shared'])
|
||||
self.task.node = self.node
|
||||
|
||||
@mock.patch.object(notif_utils, '_emit_conductor_node_notification')
|
||||
def test_emit_power_state_corrected_notification(self, mock_cond_emit):
|
||||
notif_utils.emit_power_state_corrected_notification(
|
||||
self.task, states.POWER_ON)
|
||||
mock_cond_emit.assert_called_once_with(
|
||||
self.task,
|
||||
node_objects.NodeCorrectedPowerStateNotification,
|
||||
node_objects.NodeCorrectedPowerStatePayload,
|
||||
'power_state_corrected',
|
||||
fields.NotificationLevel.INFO,
|
||||
fields.NotificationStatus.SUCCESS,
|
||||
from_power=states.POWER_ON
|
||||
)
|
||||
|
||||
@mock.patch.object(notif_utils, '_emit_conductor_node_notification')
|
||||
def test_emit_power_set_notification(self, mock_cond_emit):
|
||||
notif_utils.emit_power_set_notification(
|
||||
self.task,
|
||||
fields.NotificationLevel.DEBUG,
|
||||
fields.NotificationStatus.END,
|
||||
states.POWER_ON)
|
||||
mock_cond_emit.assert_called_once_with(
|
||||
self.task,
|
||||
node_objects.NodeSetPowerStateNotification,
|
||||
node_objects.NodeSetPowerStatePayload,
|
||||
'power_set',
|
||||
fields.NotificationLevel.DEBUG,
|
||||
fields.NotificationStatus.END,
|
||||
to_power=states.POWER_ON
|
||||
)
|
||||
|
||||
def test__emit_conductor_node_notification(self):
|
||||
mock_notify_method = mock.Mock()
|
||||
# Required for exception handling
|
||||
mock_notify_method.__name__ = 'MockNotificationConstructor'
|
||||
mock_payload_method = mock.Mock()
|
||||
mock_payload_method.__name__ = 'MockPayloadConstructor'
|
||||
mock_kwargs = {'mock0': mock.Mock(),
|
||||
'mock1': mock.Mock()}
|
||||
|
||||
notif_utils._emit_conductor_node_notification(
|
||||
self.task,
|
||||
mock_notify_method,
|
||||
mock_payload_method,
|
||||
'fake_action',
|
||||
fields.NotificationLevel.INFO,
|
||||
fields.NotificationStatus.SUCCESS,
|
||||
**mock_kwargs
|
||||
)
|
||||
|
||||
mock_payload_method.assert_called_once_with(
|
||||
self.task.node, **mock_kwargs)
|
||||
mock_notify_method.assert_called_once_with(
|
||||
publisher=mock.ANY,
|
||||
event_type=mock.ANY,
|
||||
level=fields.NotificationLevel.INFO,
|
||||
payload=mock_payload_method.return_value
|
||||
)
|
||||
mock_notify_method.return_value.emit.assert_called_once_with(
|
||||
self.task.context)
|
||||
|
||||
def test__emit_conductor_node_notification_known_payload_exc(self):
|
||||
"""Test exception caught for a known payload exception."""
|
||||
mock_notify_method = mock.Mock()
|
||||
# Required for exception handling
|
||||
mock_notify_method.__name__ = 'MockNotificationConstructor'
|
||||
mock_payload_method = mock.Mock()
|
||||
mock_payload_method.__name__ = 'MockPayloadConstructor'
|
||||
mock_kwargs = {'mock0': mock.Mock(),
|
||||
'mock1': mock.Mock()}
|
||||
mock_payload_method.side_effect = exception.NotificationSchemaKeyError
|
||||
|
||||
notif_utils._emit_conductor_node_notification(
|
||||
self.task,
|
||||
mock_notify_method,
|
||||
mock_payload_method,
|
||||
'fake_action',
|
||||
fields.NotificationLevel.INFO,
|
||||
fields.NotificationStatus.SUCCESS,
|
||||
**mock_kwargs
|
||||
)
|
||||
|
||||
self.assertFalse(mock_notify_method.called)
|
||||
|
||||
def test__emit_conductor_node_notification_known_notify_exc(self):
|
||||
"""Test exception caught for a known notification exception."""
|
||||
mock_notify_method = mock.Mock()
|
||||
# Required for exception handling
|
||||
mock_notify_method.__name__ = 'MockNotificationConstructor'
|
||||
mock_payload_method = mock.Mock()
|
||||
mock_payload_method.__name__ = 'MockPayloadConstructor'
|
||||
mock_kwargs = {'mock0': mock.Mock(),
|
||||
'mock1': mock.Mock()}
|
||||
mock_notify_method.side_effect = VersionedObjectsException
|
||||
|
||||
notif_utils._emit_conductor_node_notification(
|
||||
self.task,
|
||||
mock_notify_method,
|
||||
mock_payload_method,
|
||||
'fake_action',
|
||||
fields.NotificationLevel.INFO,
|
||||
fields.NotificationStatus.SUCCESS,
|
||||
**mock_kwargs
|
||||
)
|
||||
|
||||
self.assertFalse(mock_notify_method.return_value.emit.called)
|
@ -11,6 +11,7 @@
|
||||
# under the License.
|
||||
|
||||
import mock
|
||||
from oslo_config import cfg
|
||||
from oslo_utils import uuidutils
|
||||
|
||||
from ironic.common import driver_factory
|
||||
@ -19,12 +20,15 @@ from ironic.common import states
|
||||
from ironic.conductor import task_manager
|
||||
from ironic.conductor import utils as conductor_utils
|
||||
from ironic import objects
|
||||
from ironic.objects import fields as obj_fields
|
||||
from ironic.tests import base as tests_base
|
||||
from ironic.tests.unit.conductor import mgr_utils
|
||||
from ironic.tests.unit.db import base
|
||||
from ironic.tests.unit.db import utils
|
||||
from ironic.tests.unit.objects import utils as obj_utils
|
||||
|
||||
CONF = cfg.CONF
|
||||
|
||||
|
||||
class NodeSetBootDeviceTestCase(base.DbTestCase):
|
||||
|
||||
@ -79,7 +83,6 @@ class NodeSetBootDeviceTestCase(base.DbTestCase):
|
||||
|
||||
|
||||
class NodePowerActionTestCase(base.DbTestCase):
|
||||
|
||||
def setUp(self):
|
||||
super(NodePowerActionTestCase, self).setUp()
|
||||
mgr_utils.mock_the_extension_manager()
|
||||
@ -105,6 +108,47 @@ class NodePowerActionTestCase(base.DbTestCase):
|
||||
self.assertIsNone(node['target_power_state'])
|
||||
self.assertIsNone(node['last_error'])
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_power_action_power_on_notify(self, mock_notif):
|
||||
"""Test node_power_action to power on node and send notification."""
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
uuid=uuidutils.generate_uuid(),
|
||||
driver='fake',
|
||||
power_state=states.POWER_OFF)
|
||||
task = task_manager.TaskManager(self.context, node.uuid)
|
||||
|
||||
with mock.patch.object(self.driver.power,
|
||||
'get_power_state') as get_power_mock:
|
||||
get_power_mock.return_value = states.POWER_OFF
|
||||
|
||||
conductor_utils.node_power_action(task, states.POWER_ON)
|
||||
|
||||
node.refresh()
|
||||
get_power_mock.assert_called_once_with(mock.ANY)
|
||||
self.assertEqual(states.POWER_ON, node.power_state)
|
||||
self.assertIsNone(node.target_power_state)
|
||||
self.assertIsNone(node.last_error)
|
||||
|
||||
# 2 notifications should be sent: 1 .start and 1 .end
|
||||
self.assertEqual(2, mock_notif.call_count)
|
||||
self.assertEqual(2, mock_notif.return_value.emit.call_count)
|
||||
|
||||
first_notif_args = mock_notif.call_args_list[0][1]
|
||||
second_notif_args = mock_notif.call_args_list[1][1]
|
||||
|
||||
self.assertNotificationEqual(first_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.start',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
self.assertNotificationEqual(second_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.end',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
|
||||
def test_node_power_action_power_off(self):
|
||||
"""Test node_power_action to turn node power off."""
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
@ -172,6 +216,50 @@ class NodePowerActionTestCase(base.DbTestCase):
|
||||
self.assertIsNone(node['target_power_state'])
|
||||
self.assertIsNone(node['last_error'])
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_power_action_invalid_state_notify(self, mock_notif):
|
||||
"""Test for notification when changing to an invalid power state."""
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
uuid=uuidutils.generate_uuid(),
|
||||
driver='fake',
|
||||
power_state=states.POWER_ON)
|
||||
task = task_manager.TaskManager(self.context, node.uuid)
|
||||
|
||||
with mock.patch.object(self.driver.power,
|
||||
'get_power_state') as get_power_mock:
|
||||
get_power_mock.return_value = states.POWER_ON
|
||||
|
||||
self.assertRaises(exception.InvalidParameterValue,
|
||||
conductor_utils.node_power_action,
|
||||
task,
|
||||
"INVALID_POWER_STATE")
|
||||
|
||||
node.refresh()
|
||||
get_power_mock.assert_called_once_with(mock.ANY)
|
||||
self.assertEqual(states.POWER_ON, node.power_state)
|
||||
self.assertIsNone(node.target_power_state)
|
||||
self.assertIsNotNone(node.last_error)
|
||||
|
||||
# 2 notifications should be sent: 1 .start and 1 .error
|
||||
self.assertEqual(2, mock_notif.call_count)
|
||||
self.assertEqual(2, mock_notif.return_value.emit.call_count)
|
||||
|
||||
first_notif_args = mock_notif.call_args_list[0][1]
|
||||
second_notif_args = mock_notif.call_args_list[1][1]
|
||||
|
||||
self.assertNotificationEqual(first_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.start',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
self.assertNotificationEqual(second_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.error',
|
||||
obj_fields.NotificationLevel.ERROR)
|
||||
|
||||
def test_node_power_action_already_being_processed(self):
|
||||
"""Test node power action after aborted power action.
|
||||
|
||||
@ -282,6 +370,51 @@ class NodePowerActionTestCase(base.DbTestCase):
|
||||
self.assertIsNone(node['target_power_state'])
|
||||
self.assertIsNotNone(node['last_error'])
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_power_action_failed_getting_state_notify(self, mock_notif):
|
||||
"""Test for notification when we can't get the current power state."""
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
uuid=uuidutils.generate_uuid(),
|
||||
driver='fake',
|
||||
power_state=states.POWER_ON)
|
||||
task = task_manager.TaskManager(self.context, node.uuid)
|
||||
|
||||
with mock.patch.object(self.driver.power,
|
||||
'get_power_state') as get_power_state_mock:
|
||||
get_power_state_mock.side_effect = (
|
||||
exception.InvalidParameterValue('failed getting power state'))
|
||||
|
||||
self.assertRaises(exception.InvalidParameterValue,
|
||||
conductor_utils.node_power_action,
|
||||
task,
|
||||
states.POWER_ON)
|
||||
|
||||
node.refresh()
|
||||
get_power_state_mock.assert_called_once_with(mock.ANY)
|
||||
self.assertEqual(states.POWER_ON, node.power_state)
|
||||
self.assertIsNone(node.target_power_state)
|
||||
self.assertIsNotNone(node.last_error)
|
||||
|
||||
# 2 notifications should be sent: 1 .start and 1 .error
|
||||
self.assertEqual(2, mock_notif.call_count)
|
||||
self.assertEqual(2, mock_notif.return_value.emit.call_count)
|
||||
|
||||
first_notif_args = mock_notif.call_args_list[0][1]
|
||||
second_notif_args = mock_notif.call_args_list[1][1]
|
||||
|
||||
self.assertNotificationEqual(first_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.start',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
self.assertNotificationEqual(second_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.error',
|
||||
obj_fields.NotificationLevel.ERROR)
|
||||
|
||||
def test_node_power_action_set_power_failure(self):
|
||||
"""Test if an exception is thrown when the set_power call fails."""
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
@ -311,6 +444,56 @@ class NodePowerActionTestCase(base.DbTestCase):
|
||||
self.assertIsNone(node['target_power_state'])
|
||||
self.assertIsNotNone(node['last_error'])
|
||||
|
||||
@mock.patch('ironic.objects.node.NodeSetPowerStateNotification')
|
||||
def test_node_power_action_set_power_failure_notify(self, mock_notif):
|
||||
"""Test if a notification is sent when the set_power call fails."""
|
||||
self.config(notification_level='info')
|
||||
self.config(host='my-host')
|
||||
# Required for exception handling
|
||||
mock_notif.__name__ = 'NodeSetPowerStateNotification'
|
||||
node = obj_utils.create_test_node(self.context,
|
||||
uuid=uuidutils.generate_uuid(),
|
||||
driver='fake',
|
||||
power_state=states.POWER_OFF)
|
||||
task = task_manager.TaskManager(self.context, node.uuid)
|
||||
|
||||
with mock.patch.object(self.driver.power,
|
||||
'get_power_state') as get_power_mock:
|
||||
with mock.patch.object(self.driver.power,
|
||||
'set_power_state') as set_power_mock:
|
||||
get_power_mock.return_value = states.POWER_OFF
|
||||
set_power_mock.side_effect = exception.IronicException()
|
||||
|
||||
self.assertRaises(
|
||||
exception.IronicException,
|
||||
conductor_utils.node_power_action,
|
||||
task,
|
||||
states.POWER_ON)
|
||||
|
||||
node.refresh()
|
||||
get_power_mock.assert_called_once_with(mock.ANY)
|
||||
set_power_mock.assert_called_once_with(mock.ANY,
|
||||
states.POWER_ON)
|
||||
self.assertEqual(states.POWER_OFF, node.power_state)
|
||||
self.assertIsNone(node.target_power_state)
|
||||
self.assertIsNotNone(node.last_error)
|
||||
|
||||
# 2 notifications should be sent: 1 .start and 1 .error
|
||||
self.assertEqual(2, mock_notif.call_count)
|
||||
self.assertEqual(2, mock_notif.return_value.emit.call_count)
|
||||
|
||||
first_notif_args = mock_notif.call_args_list[0][1]
|
||||
second_notif_args = mock_notif.call_args_list[1][1]
|
||||
|
||||
self.assertNotificationEqual(first_notif_args,
|
||||
'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.start',
|
||||
obj_fields.NotificationLevel.INFO)
|
||||
self.assertNotificationEqual(
|
||||
second_notif_args, 'ironic-conductor', CONF.host,
|
||||
'baremetal.node.power_set.error',
|
||||
obj_fields.NotificationLevel.ERROR)
|
||||
|
||||
|
||||
class CleanupAfterTimeoutTestCase(tests_base.TestCase):
|
||||
def setUp(self):
|
||||
@ -569,6 +752,14 @@ class ErrorHandlersTestCase(tests_base.TestCase):
|
||||
self.task.driver = mock.Mock(spec_set=['deploy'])
|
||||
self.task.node = mock.Mock(spec_set=objects.Node)
|
||||
self.node = self.task.node
|
||||
# NOTE(mariojv) Some of the test cases that use the task below require
|
||||
# strict typing of the node power state fields and would fail if passed
|
||||
# a Mock object in constructors. A task context is also required for
|
||||
# notifications.
|
||||
power_attrs = {'power_state': states.POWER_OFF,
|
||||
'target_power_state': states.POWER_ON}
|
||||
self.node.configure_mock(**power_attrs)
|
||||
self.task.context = self.context
|
||||
|
||||
@mock.patch.object(conductor_utils, 'LOG')
|
||||
def test_provision_error_handler_no_worker(self, log_mock):
|
||||
|
@ -120,3 +120,18 @@ class TestNotificationLevelField(test_base.TestCase):
|
||||
def test_coerce_bad_value(self):
|
||||
self.assertRaises(ValueError, self.field.coerce, 'obj', 'attr',
|
||||
'not_a_priority')
|
||||
|
||||
|
||||
class TestNotificationStatusField(test_base.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
super(TestNotificationStatusField, self).setUp()
|
||||
self.field = fields.NotificationStatusField()
|
||||
|
||||
def test_coerce_good_value(self):
|
||||
self.assertEqual(fields.NotificationStatus.START,
|
||||
self.field.coerce('obj', 'attr', 'start'))
|
||||
|
||||
def test_coerce_bad_value(self):
|
||||
self.assertRaises(ValueError, self.field.coerce, 'obj', 'attr',
|
||||
'not_a_priority')
|
||||
|
@ -95,7 +95,8 @@ class TestNotificationBase(test_base.TestCase):
|
||||
payload.populate_schema(test_obj=self.fake_obj)
|
||||
notif = self.TestNotification(
|
||||
event_type=notification.EventType(
|
||||
object='test_object', action='test', status='start'),
|
||||
object='test_object', action='test',
|
||||
status=fields.NotificationStatus.START),
|
||||
level=fields.NotificationLevel.DEBUG,
|
||||
publisher=notification.NotificationPublisher(
|
||||
service='ironic-conductor',
|
||||
@ -132,7 +133,8 @@ class TestNotificationBase(test_base.TestCase):
|
||||
payload.populate_schema(test_obj=self.fake_obj)
|
||||
notif = self.TestNotification(
|
||||
event_type=notification.EventType(
|
||||
object='test_object', action='test', status='start'),
|
||||
object='test_object', action='test',
|
||||
status=fields.NotificationStatus.START),
|
||||
level=fields.NotificationLevel.DEBUG,
|
||||
publisher=notification.NotificationPublisher(
|
||||
service='ironic-conductor',
|
||||
@ -153,7 +155,8 @@ class TestNotificationBase(test_base.TestCase):
|
||||
payload.populate_schema(test_obj=self.fake_obj)
|
||||
notif = self.TestNotification(
|
||||
event_type=notification.EventType(
|
||||
object='test_object', action='test', status='start'),
|
||||
object='test_object', action='test',
|
||||
status=fields.NotificationStatus.START),
|
||||
level=fields.NotificationLevel.DEBUG,
|
||||
publisher=notification.NotificationPublisher(
|
||||
service='ironic-conductor',
|
||||
@ -172,7 +175,8 @@ class TestNotificationBase(test_base.TestCase):
|
||||
an_optional_field=1)
|
||||
notif = self.TestNotification(
|
||||
event_type=notification.EventType(
|
||||
object='test_object', action='test', status='start'),
|
||||
object='test_object', action='test',
|
||||
status=fields.NotificationStatus.START),
|
||||
level=fields.NotificationLevel.DEBUG,
|
||||
publisher=notification.NotificationPublisher(
|
||||
service='ironic-conductor',
|
||||
@ -190,7 +194,8 @@ class TestNotificationBase(test_base.TestCase):
|
||||
payload = self.TestNotificationPayloadEmptySchema(fake_field='123')
|
||||
notif = self.TestNotificationEmptySchema(
|
||||
event_type=notification.EventType(
|
||||
object='test_object', action='test', status='error'),
|
||||
object='test_object', action='test',
|
||||
status=fields.NotificationStatus.ERROR),
|
||||
level=fields.NotificationLevel.ERROR,
|
||||
publisher=notification.NotificationPublisher(
|
||||
service='ironic-conductor',
|
||||
|
@ -410,8 +410,15 @@ expected_object_fingerprints = {
|
||||
'Port': '1.6-609504503d68982a10f495659990084b',
|
||||
'Portgroup': '1.2-37b374b19bfd25db7e86aebc364e611e',
|
||||
'Conductor': '1.1-5091f249719d4a465062a1b3dc7f860d',
|
||||
'EventType': '1.1-5d44b591d93189b2ea91a1af9b082df6',
|
||||
'EventType': '1.1-aa2ba1afd38553e3880c267404e8d370',
|
||||
'NotificationPublisher': '1.0-51a09397d6c0687771fb5be9a999605d',
|
||||
'NodePayload': '1.0-ccb491ab5cd247e2ba3f21af4c12eb7c',
|
||||
'NodeSetPowerStateNotification': '1.0-59acc533c11d306f149846f922739c15',
|
||||
'NodeSetPowerStatePayload': '1.0-80986cc6a099cccd481fe3e288157a07',
|
||||
'NodeCorrectedPowerStateNotification': '1.0-59acc533c11d306f149846f922739'
|
||||
'c15',
|
||||
'NodeCorrectedPowerStatePayload': '1.0-2a484d7c342caa9fe488de16dc5f1f1e',
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
@ -0,0 +1,13 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
Adds notifications for:
|
||||
|
||||
* when ironic attempts to set the power state on the node (notifications
|
||||
with event type "baremetal.node.power_set.{start, end, error}")
|
||||
* when ironic detects the power state on baremetal hardware has changed
|
||||
and updates the node in the database appropriately (notifications with
|
||||
event type "baremetal.node.power_state_corrected.success")
|
||||
|
||||
These are emitted if notifications are enabled. For more details, see the
|
||||
developer documentation.
|
Loading…
Reference in New Issue
Block a user