openstack
/
ironic
Code Issues Proposed changes

Active Node Creation via adopt state 

At present the ironic API explicitly sets the new state for
nodes to the beginning step in the ironic workflow.

As part of hardware fleet lifecycle management, an operator
expects to be able to migrate inventory and control systems
for their hardware fleet utilizing their existing inventory
data and allocation records. Ultimately this means that
an imported host MAY already be allocated and unavailable
for immediate allocation.

As such, a mechanism is required to permit users to put nodes
into an ACTIVE state without performing a deployment operation.

This adds a new API provision_state verb to allow users to move
nodes from MANAGEABLE state to ACTIVE state.

Partial-Bug: #1526315
Change-Id: Ib3eadf4172e93add9a9855582f56cbb3707f3d39
Depends-On: Ie114bfaab249d73ea3ca7c0edc314ca1ed0448eb
This commit is contained in:
Julia Kreger 2016-02-02 10:44:06 -05:00
parent 8b75becccb
commit 832826f640
15 changed files with 846 additions and 223 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

183
doc/source/deploy/adoption.rst Normal file
View File

@ -0,0 +1,183 @@
.. _adoption:
=============
Node adoption
=============
Overview
========
As part of hardware inventory lifecycle management, it is not an
unreasonable need to have the capability to be able to add hardware
that should be considered "in-use" by the Bare Metal service,
that may have been deployed by another Bare Metal service
installation or deployed via other means.
As such, the node adoption feature allows a user to define a node
as ``active`` while skipping the ``available`` and ``deploying``
states, which will prevent the node from being seen by the Compute
service as ready for use.
This feature is leveraged as part of the state machine workflow,
where a node in ``manageable`` can be moved to ``active`` state
via the provision_state verb ``adopt``. To view the state
transition capabilities, please see :ref:`states`.
How it works
============
A node initially enrolled begins in the ``enroll`` state. An operator
must then move the node to ``manageable`` state, which causes the driver's
``power`` interface to be validated. Once in ``manageable`` state,
an operator can then explicitly choose to adopt a node.
Adoption of a node results in the validation of the driver ``boot`` interface,
and upon success the process leverages what is referred to as the "takeover"
logic. The takeover process is intended for conductors to take over the
management of nodes for a conductor that has failed.
The takeover process involves the driver deploy ``prepare`` and ``take_over``
methods being called. These steps take driver specific actions such as
downloading and staging the deployment kernel and ramdisk, ISO image, any
required boot image, or boot ISO image and then places any PXE or virtual
media configuration necessary for the node should it be required.
The adoption process makes no changes to the physical node, with the
exception of operator supplied configurations where virtual media is
used to boot the node under normal circumstances. An operator should
ensure that any supplied configuration defining the node is sufficient
for the continued operation of the node moving forward. Such as, if the
node is configured to network boot via instance_info/boot_option="netboot",
then appropriate driver specific node configuration should be set to
support this capability.
Possible Risk
=============
The main risk with this feature is that supplied configuration may ultimately
be incorrect or invalid which could result in potential operational issues:
* ``rebuild`` verb - Rebuild is intended to allow a user to re-deploy the node
to a fresh state. The risk with adoption is that the image defined when an
operator adopts the node may not be the valid image for the pre-existing
configuration.
If this feature is utilized for a migration from one deployment to another,
and pristine original images are loaded and provided, then ultimately the
risk is the same with any normal use of the ``rebuild`` feature, the server
is effectively wiped.
* When deleting a node, the deletion or cleaning processes may fail if the
incorrect deployment image is supplied in the configuration as the node
may NOT have been deployed with the supplied image and driver or
compatibility issues may exist as a result.
Operators will need to be cognizant of that possibility and should plan
accordingly to ensure that deployment images are known to be compatible
with the hardware in their environment.
* Networking - Adoption will assert no new networking configuration to the
newly adopted node as that would be considered modifying the node.
Operators will need to plan accordingly and have network configuration
such that the nodes will be able to network boot.
How to use
==========
.. NOTE::
The power state that the ironic-conductor observes upon the first
successful power state check, as part of the transition to the
``manageable`` state will be enforced with a node that has been adopted.
This means a node that is in ``power off`` state will, by default, have
the power state enforced as ``power off`` moving forward, unless an
administrator actively changes the power state using the Bare Metal
service.
Requirements
------------
Requirements for use are essentially the same as to deploy a node:
* Sufficient driver information to allow for a successful
power management validation.
* Sufficient instance_info to pass deploy driver validation.
Each driver may have additional requirements dependent upon the
configuration that is supplied. An example of this would be defining
a node to always boot from the network, which will cause the conductor
to attempt to retrieve the pertinent files. Inability to do so will
result in the adoption failing, and the node being placed in the
``adopt failed`` state.
agent_ipmitool example
----------------------
This is an example to create a new node, named ``testnode``, with
sufficient information to pass basic validation in order to be taken
from the ``manageable`` state to ``active`` state.::
# Explicitly set the client API version environment variable to
# 1.17, which introduces the adoption capability.
export IRONIC_API_VERSION=1.17
ironic node-create -n testnode \
-d agent_ipmitool \
-i ipmi_address=<ip_address> \
-i ipmi_username=<username> \
-i ipmi_password=<password> \
-i deploy_kernel=<deploy_kernel_id_or_url> \
-i deploy_ramdisk=<deploy_ramdisk_id_or_url>
ironic port-create --node <node_uuid> -a <node_mac_address>
ironic node-update testnode add \
instance_info/image_source="http://localhost:8080/blankimage"
ironic node-set-provision-state testnode manage
ironic node-set-provision-state testnode adopt
.. NOTE::
In the above example, the image_source setting must reference a valid
image or file, however that image or file can ultimately be empty.
.. NOTE::
The above example will naturally fail as a fake image is
defined, and no instance_info/image_checksum is defined so
any actual attempt to write the image out will fail.
.. NOTE::
A user may wish to assign an instance_uuid to a node, which could be
used to match an instance in the Compute service. Doing so is not
required for the proper operation of the Bare Metal service.
ironic node-update <node name or uuid> add instance_uuid=<uuid>
Troubleshooting
===============
Should an adoption operation fail for a node, the error that caused the
failure will be logged in the node's ``last_error`` field when viewing the
node. This error, in the case of node adoption, will largely be due to
failure of a validation step. Validation steps are dependent
upon what driver is selected for the node.
Any node that is in the ``adopt failed`` state can have the ``adopt`` verb
re-attempted. Example::
ironic node-set-provision-state <node name or uuid> adopt
If a user wishes to abort their attempt at adopting, they can then move
the node back to ``manageable`` from ``adopt failed`` state by issuing the
``manage`` verb. Example::
ironic node-set-provision-state <node name or uuid> manage
If all else fails the hardware node can be removed from the Bare Metal
service. The ``node-delete`` command, which is **not** the same as setting
the provision state to ``delete``, can be used while the node is in
``adopt failed`` state. This will delete the node without cleaning
occurring to preserve the node's current state. Example::
ironic node-delete <node name or uuid>

470
doc/source/images/states.svg
View File

@ -1,298 +1,338 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 2.30.1 (20140821.0912)
<!-- Generated by graphviz version 2.36.0 (20140111.2315)
-->
<!-- Title: Ironic states Pages: 1 -->
<svg width="1724pt" height="486pt"
viewBox="0.00 0.00 1724.00 486.31" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 482.309)">
<svg width="1724pt" height="503pt"
viewBox="0.00 0.00 1724.00 502.55" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 498.55)">
<title>Ironic states</title>
<polygon fill="white" stroke="white" points="-4,5 -4,-482.309 1721,-482.309 1721,5 -4,5"/>
<polygon fill="white" stroke="none" points="-4,4 -4,-498.55 1720,-498.55 1720,4 -4,4"/>
<!-- enroll -->
<g id="node1" class="node"><title>enroll</title>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="27" cy="-136.309" rx="27" ry="18"/>
<text text-anchor="middle" x="27" y="-133.509" font-family="Times,serif" font-size="11.00">enroll</text>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="27" cy="-187.55" rx="27" ry="18"/>
<text text-anchor="middle" x="27" y="-184.75" font-family="Times,serif" font-size="11.00">enroll</text>
</g>
<!-- verifying -->
<g id="node2" class="node"><title>verifying</title>
<ellipse fill="none" stroke="black" cx="210" cy="-136.309" rx="33.8507" ry="18"/>
<text text-anchor="middle" x="210" y="-133.509" font-family="Times,serif" font-size="11.00" fill="gray">verifying</text>
<ellipse fill="none" stroke="black" cx="210" cy="-187.55" rx="33.8507" ry="18"/>
<text text-anchor="middle" x="210" y="-184.75" font-family="Times,serif" font-size="11.00" fill="gray">verifying</text>
</g>
<!-- enroll&#45;&gt;verifying -->
<g id="edge1" class="edge"><title>enroll&#45;&gt;verifying</title>
<path fill="none" stroke="black" d="M54.319,-136.309C83.5522,-136.309 131.193,-136.309 165.889,-136.309"/>
<polygon fill="black" stroke="black" points="166.207,-139.81 176.207,-136.309 166.207,-132.81 166.207,-139.81"/>
<text text-anchor="middle" x="115" y="-139.709" font-family="Times,serif" font-size="12.00">manage (via API)</text>
<path fill="none" stroke="black" d="M54.319,-187.55C83.5522,-187.55 131.193,-187.55 165.889,-187.55"/>
<polygon fill="black" stroke="black" points="166.207,-191.05 176.207,-187.55 166.207,-184.05 166.207,-191.05"/>
<text text-anchor="middle" x="115" y="-190.95" font-family="Times,serif" font-size="12.00">manage (via API)</text>
</g>
<!-- verifying&#45;&gt;enroll -->
<g id="edge13" class="edge"><title>verifying&#45;&gt;enroll</title>
<path fill="none" stroke="black" d="M182.674,-176.984C174.862,-174.341 166.199,-171.881 158,-170.55 120.271,-164.428 109.575,-163.544 72,-170.55 67.7433,-171.344 63.368,-172.535 59.1046,-173.925"/>
<polygon fill="black" stroke="black" points="57.6647,-170.725 49.4683,-177.438 60.0622,-177.301 57.6647,-170.725"/>
<text text-anchor="middle" x="115" y="-173.95" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- manageable -->
<g id="node3" class="node"><title>manageable</title>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="346" cy="-136.309" rx="42.1875" ry="18"/>
<text text-anchor="middle" x="346" y="-133.509" font-family="Times,serif" font-size="11.00">manageable</text>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="346" cy="-187.55" rx="42.1875" ry="18"/>
<text text-anchor="middle" x="346" y="-184.75" font-family="Times,serif" font-size="11.00">manageable</text>
</g>
<!-- verifying&#45;&gt;manageable -->
<g id="edge11" class="edge"><title>verifying&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M243.78,-136.309C258.666,-136.309 276.633,-136.309 293.273,-136.309"/>
<polygon fill="black" stroke="black" points="293.422,-139.81 303.422,-136.309 293.422,-132.81 293.422,-139.81"/>
<text text-anchor="middle" x="274" y="-139.709" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
</g>
<!-- verifying&#45;&gt;enroll -->
<g id="edge12" class="edge"><title>verifying&#45;&gt;enroll</title>
<path fill="none" stroke="black" d="M182.674,-125.744C174.862,-123.1 166.199,-120.64 158,-119.309 120.271,-113.187 109.575,-112.303 72,-119.309 67.7433,-120.103 63.368,-121.294 59.1046,-122.685"/>
<polygon fill="black" stroke="black" points="57.6647,-119.484 49.4683,-126.198 60.0622,-126.061 57.6647,-119.484"/>
<text text-anchor="middle" x="115" y="-122.709" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
<g id="edge12" class="edge"><title>verifying&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M243.78,-187.55C258.666,-187.55 276.633,-187.55 293.273,-187.55"/>
<polygon fill="black" stroke="black" points="293.422,-191.05 303.422,-187.55 293.422,-184.05 293.422,-191.05"/>
<text text-anchor="middle" x="274" y="-190.95" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
</g>
<!-- cleaning -->
<g id="node4" class="node"><title>cleaning</title>
<ellipse fill="none" stroke="black" cx="551" cy="-210.309" rx="32.4445" ry="18"/>
<text text-anchor="middle" x="551" y="-207.509" font-family="Times,serif" font-size="11.00" fill="gray">cleaning</text>
<ellipse fill="none" stroke="black" cx="551" cy="-212.55" rx="32.4445" ry="18"/>
<text text-anchor="middle" x="551" y="-209.75" font-family="Times,serif" font-size="11.00" fill="gray">cleaning</text>
</g>
<!-- manageable&#45;&gt;cleaning -->
<g id="edge2" class="edge"><title>manageable&#45;&gt;cleaning</title>
<path fill="none" stroke="black" d="M356.809,-153.884C366.852,-169.946 384.075,-192.754 406,-203.309 438.278,-218.849 479.474,-219.201 509.527,-216.49"/>
<polygon fill="black" stroke="black" points="510.082,-219.951 519.66,-215.423 509.349,-212.99 510.082,-219.951"/>
<text text-anchor="middle" x="448" y="-220.709" font-family="Times,serif" font-size="12.00">provide (via API)</text>
<path fill="none" stroke="black" d="M368.881,-202.811C379.556,-209.344 392.898,-216.226 406,-219.55 440.464,-228.294 481.019,-225.179 510.294,-220.673"/>
<polygon fill="black" stroke="black" points="511.247,-224.063 520.537,-218.969 510.098,-217.158 511.247,-224.063"/>
<text text-anchor="middle" x="448" y="-227.95" font-family="Times,serif" font-size="12.00">provide (via API)</text>
</g>
<!-- manageable&#45;&gt;cleaning -->
<g id="edge3" class="edge"><title>manageable&#45;&gt;cleaning</title>
<path fill="none" stroke="black" d="M371.314,-150.921C381.713,-156.68 394.166,-162.948 406,-167.309 441.915,-180.546 453.45,-174.944 490,-186.309 498.393,-188.919 507.268,-192.179 515.545,-195.447"/>
<polygon fill="black" stroke="black" points="514.338,-198.735 524.92,-199.249 516.968,-192.248 514.338,-198.735"/>
<text text-anchor="middle" x="448" y="-189.709" font-family="Times,serif" font-size="12.00">clean (via API)</text>
<path fill="none" stroke="black" d="M387.452,-191.134C416.131,-193.836 455.493,-197.869 490,-202.55 496.387,-203.416 503.138,-204.438 509.718,-205.492"/>
<polygon fill="black" stroke="black" points="509.403,-208.987 519.838,-207.156 510.539,-202.08 509.403,-208.987"/>
<text text-anchor="middle" x="448" y="-205.95" font-family="Times,serif" font-size="12.00">clean (via API)</text>
</g>
<!-- inspecting -->
<g id="node5" class="node"><title>inspecting</title>
<ellipse fill="none" stroke="black" cx="551" cy="-23.3094" rx="37.0671" ry="18"/>
<text text-anchor="middle" x="551" y="-20.5094" font-family="Times,serif" font-size="11.00" fill="gray">inspecting</text>
<ellipse fill="none" stroke="black" cx="551" cy="-25.55" rx="37.0671" ry="18"/>
<text text-anchor="middle" x="551" y="-22.75" font-family="Times,serif" font-size="11.00" fill="gray">inspecting</text>
</g>
<!-- manageable&#45;&gt;inspecting -->
<g id="edge4" class="edge"><title>manageable&#45;&gt;inspecting</title>
<path fill="none" stroke="black" d="M354.248,-118.338C363.276,-98.3835 380.665,-66.8151 406,-50.3094 421.906,-39.947 468.219,-32.4895 504.072,-28.0722"/>
<polygon fill="black" stroke="black" points="504.827,-31.5072 514.345,-26.853 504.002,-24.556 504.827,-31.5072"/>
<text text-anchor="middle" x="448" y="-53.7094" font-family="Times,serif" font-size="12.00">inspect (via API)</text>
<path fill="none" stroke="black" d="M349.471,-169.433C354.494,-140.014 368.771,-82.1703 406,-52.55 420.855,-40.731 467.343,-33.5154 503.525,-29.5531"/>
<polygon fill="black" stroke="black" points="504.314,-32.99 513.898,-28.4735 503.589,-26.0276 504.314,-32.99"/>
<text text-anchor="middle" x="448" y="-55.95" font-family="Times,serif" font-size="12.00">inspect (via API)</text>
</g>
<!-- available -->
<g id="node6" class="node"><title>available</title>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="763" cy="-319.309" rx="34.054" ry="18"/>
<text text-anchor="middle" x="763" y="-316.509" font-family="Times,serif" font-size="11.00">available</text>
<!-- adopting -->
<g id="node6" class="node"><title>adopting</title>
<ellipse fill="none" stroke="black" cx="551" cy="-412.55" rx="32.4445" ry="18"/>
<text text-anchor="middle" x="551" y="-409.75" font-family="Times,serif" font-size="11.00" fill="gray">adopting</text>
</g>
<!-- cleaning&#45;&gt;available -->
<g id="edge22" class="edge"><title>cleaning&#45;&gt;available</title>
<path fill="none" stroke="black" d="M564.556,-226.687C575.644,-240.05 592.974,-258.437 612,-269.309 646.352,-288.94 660.573,-279.478 698,-292.309 707.652,-295.618 717.915,-299.689 727.33,-303.657"/>
<polygon fill="black" stroke="black" points="726.205,-306.983 736.775,-307.719 728.971,-300.553 726.205,-306.983"/>
<text text-anchor="middle" x="655" y="-295.709" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
</g>
<!-- clean failed -->
<g id="node13" class="node"><title>clean failed</title>
<ellipse fill="none" stroke="black" cx="964" cy="-216.309" rx="41.4846" ry="18"/>
<text text-anchor="middle" x="964" y="-213.509" font-family="Times,serif" font-size="11.00" fill="red">clean failed</text>
</g>
<!-- cleaning&#45;&gt;clean failed -->
<g id="edge23" class="edge"><title>cleaning&#45;&gt;clean failed</title>
<path fill="none" stroke="black" d="M579.519,-218.962C589.632,-221.792 601.241,-224.636 612,-226.309 740.242,-246.259 775.134,-244.727 904,-229.309 908.516,-228.769 913.194,-228.039 917.851,-227.199"/>
<polygon fill="black" stroke="black" points="918.802,-230.579 927.939,-225.214 917.451,-223.71 918.802,-230.579"/>
<text text-anchor="middle" x="763" y="-243.709" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- clean wait -->
<g id="node14" class="node"><title>clean wait</title>
<ellipse fill="none" stroke="black" cx="763" cy="-204.309" rx="37.7689" ry="18"/>
<text text-anchor="middle" x="763" y="-201.509" font-family="Times,serif" font-size="11.00" fill="gray">clean wait</text>
</g>
<!-- cleaning&#45;&gt;clean wait -->
<g id="edge24" class="edge"><title>cleaning&#45;&gt;clean wait</title>
<path fill="none" stroke="black" d="M583.619,-209.899C613.196,-209.459 658.599,-208.637 698,-207.309 703.489,-207.124 709.241,-206.899 714.951,-206.656"/>
<polygon fill="black" stroke="black" points="715.283,-210.145 725.118,-206.204 714.972,-203.152 715.283,-210.145"/>
<text text-anchor="middle" x="655" y="-212.709" font-family="Times,serif" font-size="12.00" fill="gray">wait</text>
<!-- manageable&#45;&gt;adopting -->
<g id="edge5" class="edge"><title>manageable&#45;&gt;adopting</title>
<path fill="none" stroke="black" d="M347.291,-205.559C348.896,-242.554 357.935,-327.494 406,-373.55 433.219,-399.631 476.034,-408.662 507.913,-411.6"/>
<polygon fill="black" stroke="black" points="508.033,-415.119 518.267,-412.379 508.557,-408.139 508.033,-415.119"/>
<text text-anchor="middle" x="448" y="-411.95" font-family="Times,serif" font-size="12.00">adopt (via API)</text>
</g>
<!-- cleaning&#45;&gt;manageable -->
<g id="edge25" class="edge"><title>cleaning&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M538.574,-193.664C527.829,-179.491 510.378,-159.94 490,-150.309 461.377,-136.782 425.988,-133.226 397.577,-133.08"/>
<polygon fill="black" stroke="black" points="397.42,-129.581 387.451,-133.168 397.481,-136.581 397.42,-129.581"/>
<text text-anchor="middle" x="448" y="-153.709" font-family="Times,serif" font-size="12.00" fill="gray">manage</text>
<g id="edge26" class="edge"><title>cleaning&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M531.764,-198.035C520.454,-189.983 505.219,-180.765 490,-176.55 458.156,-167.73 420.901,-171.01 392.401,-176.215"/>
<polygon fill="black" stroke="black" points="391.458,-172.833 382.324,-178.201 392.812,-179.701 391.458,-172.833"/>
<text text-anchor="middle" x="448" y="-179.95" font-family="Times,serif" font-size="12.00" fill="gray">manage</text>
</g>
<!-- available -->
<g id="node7" class="node"><title>available</title>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="763" cy="-313.55" rx="34.054" ry="18"/>
<text text-anchor="middle" x="763" y="-310.75" font-family="Times,serif" font-size="11.00">available</text>
</g>
<!-- cleaning&#45;&gt;available -->
<g id="edge23" class="edge"><title>cleaning&#45;&gt;available</title>
<path fill="none" stroke="black" d="M566.869,-228.52C578.227,-239.83 594.807,-254.535 612,-263.55 647.041,-281.923 660.573,-273.719 698,-286.55 707.652,-289.859 717.915,-293.93 727.33,-297.898"/>
<polygon fill="black" stroke="black" points="726.205,-301.224 736.775,-301.959 728.971,-294.793 726.205,-301.224"/>
<text text-anchor="middle" x="655" y="-289.95" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
</g>
<!-- clean failed -->
<g id="node14" class="node"><title>clean failed</title>
<ellipse fill="none" stroke="black" cx="964" cy="-214.55" rx="41.4846" ry="18"/>
<text text-anchor="middle" x="964" y="-211.75" font-family="Times,serif" font-size="11.00" fill="red">clean failed</text>
</g>
<!-- cleaning&#45;&gt;clean failed -->
<g id="edge24" class="edge"><title>cleaning&#45;&gt;clean failed</title>
<path fill="none" stroke="black" d="M580.467,-220.482C590.368,-222.895 601.598,-225.251 612,-226.55 740.778,-242.631 775.06,-242.281 904,-227.55 908.518,-227.034 913.199,-226.32 917.857,-225.489"/>
<polygon fill="black" stroke="black" points="918.803,-228.87 927.946,-223.517 917.46,-222 918.803,-228.87"/>
<text text-anchor="middle" x="763" y="-241.95" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- clean wait -->
<g id="node15" class="node"><title>clean wait</title>
<ellipse fill="none" stroke="black" cx="763" cy="-201.55" rx="37.7689" ry="18"/>
<text text-anchor="middle" x="763" y="-198.75" font-family="Times,serif" font-size="11.00" fill="gray">clean wait</text>
</g>
<!-- cleaning&#45;&gt;clean wait -->
<g id="edge25" class="edge"><title>cleaning&#45;&gt;clean wait</title>
<path fill="none" stroke="black" d="M583.439,-210.903C618.279,-209.078 674.834,-206.116 714.953,-204.014"/>
<polygon fill="black" stroke="black" points="715.228,-207.505 725.031,-203.486 714.862,-200.514 715.228,-207.505"/>
<text text-anchor="middle" x="655" y="-212.95" font-family="Times,serif" font-size="12.00" fill="gray">wait</text>
</g>
<!-- inspecting&#45;&gt;manageable -->
<g id="edge32" class="edge"><title>inspecting&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M521.615,-12.0674C490.699,-1.80437 440.995,8.71678 406,-13.3094 373.153,-33.9839 358.182,-78.5917 351.661,-108.02"/>
<polygon fill="black" stroke="black" points="348.174,-107.607 349.624,-118.102 355.035,-108.994 348.174,-107.607"/>
<text text-anchor="middle" x="448" y="-16.7094" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
<g id="edge33" class="edge"><title>inspecting&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M522.516,-13.7533C491.262,-2.36256 440.155,9.88802 406,-14.55 359.666,-47.703 349.303,-119.445 347.246,-159.343"/>
<polygon fill="black" stroke="black" points="343.743,-159.336 346.868,-169.459 350.739,-159.597 343.743,-159.336"/>
<text text-anchor="middle" x="448" y="-17.95" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
</g>
<!-- inspect failed -->
<g id="node15" class="node"><title>inspect failed</title>
<ellipse fill="none" stroke="black" cx="763" cy="-51.3094" rx="46.1069" ry="18"/>
<text text-anchor="middle" x="763" y="-48.5094" font-family="Times,serif" font-size="11.00" fill="red">inspect failed</text>
<g id="node16" class="node"><title>inspect failed</title>
<ellipse fill="none" stroke="black" cx="763" cy="-53.55" rx="46.1069" ry="18"/>
<text text-anchor="middle" x="763" y="-50.75" font-family="Times,serif" font-size="11.00" fill="red">inspect failed</text>
</g>
<!-- inspecting&#45;&gt;inspect failed -->
<g id="edge33" class="edge"><title>inspecting&#45;&gt;inspect failed</title>
<path fill="none" stroke="black" d="M582.246,-33.1699C591.683,-35.8963 602.191,-38.5786 612,-40.3094 643.032,-45.7849 678.14,-48.5425 706.585,-49.9279"/>
<polygon fill="black" stroke="black" points="706.598,-53.4318 716.744,-50.3787 706.909,-46.4387 706.598,-53.4318"/>
<text text-anchor="middle" x="655" y="-53.7094" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- deploying -->
<g id="node7" class="node"><title>deploying</title>
<ellipse fill="none" stroke="black" cx="964" cy="-329.309" rx="35.4579" ry="18"/>
<text text-anchor="middle" x="964" y="-326.509" font-family="Times,serif" font-size="11.00" fill="gray">deploying</text>
</g>
<!-- available&#45;&gt;deploying -->
<g id="edge5" class="edge"><title>available&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M797.308,-320.983C830.44,-322.648 881.492,-325.214 918.17,-327.057"/>
<polygon fill="black" stroke="black" points="918.418,-330.573 928.581,-327.58 918.769,-323.582 918.418,-330.573"/>
<text text-anchor="middle" x="866" y="-328.709" font-family="Times,serif" font-size="12.00">active (via API)</text>
</g>
<!-- available&#45;&gt;manageable -->
<g id="edge6" class="edge"><title>available&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M728.715,-320.4C660.462,-320.882 502.935,-312.76 406,-234.309 382.975,-215.675 366.751,-185.486 357.221,-163.534"/>
<polygon fill="black" stroke="black" points="360.373,-161.997 353.312,-154.098 353.905,-164.676 360.373,-161.997"/>
<text text-anchor="middle" x="551" y="-312.709" font-family="Times,serif" font-size="12.00">manage (via API)</text>
</g>
<!-- deploy failed -->
<g id="node11" class="node"><title>deploy failed</title>
<ellipse fill="none" stroke="black" cx="1317" cy="-286.309" rx="44.498" ry="18"/>
<text text-anchor="middle" x="1317" y="-283.509" font-family="Times,serif" font-size="11.00" fill="red">deploy failed</text>
</g>
<!-- deploying&#45;&gt;deploy failed -->
<g id="edge13" class="edge"><title>deploying&#45;&gt;deploy failed</title>
<path fill="none" stroke="black" d="M996.718,-322.089C1005.51,-320.297 1015.1,-318.539 1024,-317.309 1125.54,-303.289 1152.62,-315.417 1254,-300.309 1258.8,-299.595 1263.77,-298.713 1268.72,-297.743"/>
<polygon fill="black" stroke="black" points="1269.71,-301.112 1278.79,-295.651 1268.29,-294.258 1269.71,-301.112"/>
<text text-anchor="middle" x="1172" y="-312.709" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- wait call&#45;back -->
<g id="node12" class="node"><title>wait call&#45;back</title>
<ellipse fill="none" stroke="black" cx="1172" cy="-351.309" rx="48.2143" ry="18"/>
<text text-anchor="middle" x="1172" y="-348.509" font-family="Times,serif" font-size="11.00" fill="gray">wait call&#45;back</text>
</g>
<!-- deploying&#45;&gt;wait call&#45;back -->
<g id="edge14" class="edge"><title>deploying&#45;&gt;wait call&#45;back</title>
<path fill="none" stroke="black" d="M995.727,-337.697C1004.77,-339.867 1014.73,-341.968 1024,-343.309 1053.38,-347.56 1086.37,-349.601 1113.63,-350.561"/>
<polygon fill="black" stroke="black" points="1113.68,-354.064 1123.78,-350.879 1113.9,-347.068 1113.68,-354.064"/>
<text text-anchor="middle" x="1065" y="-352.709" font-family="Times,serif" font-size="12.00" fill="gray">wait</text>
<g id="edge34" class="edge"><title>inspecting&#45;&gt;inspect failed</title>
<path fill="none" stroke="black" d="M582.246,-35.4105C591.683,-38.1369 602.191,-40.8192 612,-42.55 643.032,-48.0255 678.14,-50.7831 706.585,-52.1685"/>
<polygon fill="black" stroke="black" points="706.598,-55.6724 716.744,-52.6193 706.909,-48.6793 706.598,-55.6724"/>
<text text-anchor="middle" x="655" y="-55.95" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- active -->
<g id="node8" class="node"><title>active</title>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="1172" cy="-405.309" rx="27" ry="18"/>
<text text-anchor="middle" x="1172" y="-402.509" font-family="Times,serif" font-size="11.00">active</text>
<g id="node9" class="node"><title>active</title>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="1172" cy="-410.55" rx="27" ry="18"/>
<text text-anchor="middle" x="1172" y="-407.75" font-family="Times,serif" font-size="11.00">active</text>
</g>
<!-- adopting&#45;&gt;active -->
<g id="edge37" class="edge"><title>adopting&#45;&gt;active</title>
<path fill="none" stroke="black" d="M582.779,-408.379C592.145,-407.267 602.478,-406.195 612,-405.55 658.125,-402.423 669.776,-404.37 716,-403.55 852.885,-401.123 887.098,-399.661 1024,-398.55 1060.44,-398.254 1069.71,-395.228 1106,-398.55 1115.82,-399.449 1126.37,-401.096 1136.05,-402.888"/>
<polygon fill="black" stroke="black" points="1135.66,-406.377 1146.15,-404.861 1137,-399.508 1135.66,-406.377"/>
<text text-anchor="middle" x="866" y="-403.95" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
</g>
<!-- adopt failed -->
<g id="node17" class="node"><title>adopt failed</title>
<ellipse fill="none" stroke="black" cx="763" cy="-430.55" rx="41.4846" ry="18"/>
<text text-anchor="middle" x="763" y="-427.75" font-family="Times,serif" font-size="11.00" fill="red">adopt failed</text>
</g>
<!-- adopting&#45;&gt;adopt failed -->
<g id="edge38" class="edge"><title>adopting&#45;&gt;adopt failed</title>
<path fill="none" stroke="black" d="M578.165,-422.754C588.531,-426.286 600.662,-429.798 612,-431.55 645.152,-436.674 682.919,-436.388 712.263,-434.839"/>
<polygon fill="black" stroke="black" points="712.495,-438.332 722.27,-434.246 712.081,-431.344 712.495,-438.332"/>
<text text-anchor="middle" x="655" y="-438.95" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- available&#45;&gt;manageable -->
<g id="edge7" class="edge"><title>available&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M728.535,-313.4C662.605,-311.739 513.031,-301.112 406,-241.55 392.136,-233.835 378.894,-222.248 368.48,-211.784"/>
<polygon fill="black" stroke="black" points="370.878,-209.227 361.43,-204.434 365.826,-214.073 370.878,-209.227"/>
<text text-anchor="middle" x="551" y="-303.95" font-family="Times,serif" font-size="12.00">manage (via API)</text>
</g>
<!-- deploying -->
<g id="node8" class="node"><title>deploying</title>
<ellipse fill="none" stroke="black" cx="964" cy="-320.55" rx="35.4579" ry="18"/>
<text text-anchor="middle" x="964" y="-317.75" font-family="Times,serif" font-size="11.00" fill="gray">deploying</text>
</g>
<!-- available&#45;&gt;deploying -->
<g id="edge6" class="edge"><title>available&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M797.308,-314.722C830.44,-315.887 881.492,-317.683 918.17,-318.973"/>
<polygon fill="black" stroke="black" points="918.464,-322.486 928.581,-319.339 918.71,-315.49 918.464,-322.486"/>
<text text-anchor="middle" x="866" y="-320.95" font-family="Times,serif" font-size="12.00">active (via API)</text>
</g>
<!-- deploying&#45;&gt;active -->
<g id="edge15" class="edge"><title>deploying&#45;&gt;active</title>
<path fill="none" stroke="black" d="M984.914,-343.913C995.976,-351.47 1010.25,-360.337 1024,-366.309 1060.5,-382.163 1104.98,-392.868 1135.39,-398.991"/>
<polygon fill="black" stroke="black" points="1135.04,-402.488 1145.52,-400.967 1136.38,-395.618 1135.04,-402.488"/>
<text text-anchor="middle" x="1065" y="-394.709" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
<g id="edge16" class="edge"><title>deploying&#45;&gt;active</title>
<path fill="none" stroke="black" d="M984.128,-335.45C995.3,-343.648 1009.93,-353.547 1024,-360.55 1060.97,-378.95 1106.33,-393.232 1136.8,-401.737"/>
<polygon fill="black" stroke="black" points="1135.96,-405.136 1146.53,-404.397 1137.81,-398.383 1135.96,-405.136"/>
<text text-anchor="middle" x="1065" y="-393.95" font-family="Times,serif" font-size="12.00" fill="gray">done</text>
</g>
<!-- deploy failed -->
<g id="node12" class="node"><title>deploy failed</title>
<ellipse fill="none" stroke="black" cx="1317" cy="-283.55" rx="44.498" ry="18"/>
<text text-anchor="middle" x="1317" y="-280.75" font-family="Times,serif" font-size="11.00" fill="red">deploy failed</text>
</g>
<!-- deploying&#45;&gt;deploy failed -->
<g id="edge14" class="edge"><title>deploying&#45;&gt;deploy failed</title>
<path fill="none" stroke="black" d="M996.054,-312.65C1005.02,-310.659 1014.85,-308.748 1024,-307.55 1125.45,-294.27 1152.58,-311.095 1254,-297.55 1258.81,-296.908 1263.79,-296.074 1268.75,-295.133"/>
<polygon fill="black" stroke="black" points="1269.72,-298.506 1278.82,-293.078 1268.32,-291.648 1269.72,-298.506"/>
<text text-anchor="middle" x="1172" y="-305.95" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- wait call&#45;back -->
<g id="node13" class="node"><title>wait call&#45;back</title>
<ellipse fill="none" stroke="black" cx="1172" cy="-348.55" rx="48.2143" ry="18"/>
<text text-anchor="middle" x="1172" y="-345.75" font-family="Times,serif" font-size="11.00" fill="gray">wait call&#45;back</text>
</g>
<!-- deploying&#45;&gt;wait call&#45;back -->
<g id="edge15" class="edge"><title>deploying&#45;&gt;wait call&#45;back</title>
<path fill="none" stroke="black" d="M995.764,-328.698C1004.81,-330.863 1014.76,-333.021 1024,-334.55 1053.6,-339.448 1086.87,-342.794 1114.25,-344.983"/>
<polygon fill="black" stroke="black" points="1114.2,-348.489 1124.44,-345.766 1114.74,-341.51 1114.2,-348.489"/>
<text text-anchor="middle" x="1065" y="-346.95" font-family="Times,serif" font-size="12.00" fill="gray">wait</text>
</g>
<!-- active&#45;&gt;deploying -->
<g id="edge7" class="edge"><title>active&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1148.06,-413.944C1117.76,-423.837 1063.61,-436.169 1024,-415.309 1000.6,-402.983 984.689,-376.717 975.345,-356.437"/>
<polygon fill="black" stroke="black" points="978.466,-354.835 971.281,-347.049 972.042,-357.616 978.466,-354.835"/>
<text text-anchor="middle" x="1065" y="-429.709" font-family="Times,serif" font-size="12.00">rebuild (via API)</text>
<g id="edge8" class="edge"><title>active&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1150.34,-421.324C1120.29,-435.36 1063.77,-455.453 1024,-431.55 994.275,-413.687 978.719,-375.231 971.196,-348.631"/>
<polygon fill="black" stroke="black" points="974.525,-347.53 968.61,-338.74 967.753,-349.3 974.525,-347.53"/>
<text text-anchor="middle" x="1065" y="-445.95" font-family="Times,serif" font-size="12.00">rebuild (via API)</text>
</g>
<!-- deleting -->
<g id="node9" class="node"><title>deleting</title>
<ellipse fill="none" stroke="black" cx="1512" cy="-351.309" rx="31.0408" ry="18"/>
<text text-anchor="middle" x="1512" y="-348.509" font-family="Times,serif" font-size="11.00" fill="gray">deleting</text>
<g id="node10" class="node"><title>deleting</title>
<ellipse fill="none" stroke="black" cx="1512" cy="-348.55" rx="31.0408" ry="18"/>
<text text-anchor="middle" x="1512" y="-345.75" font-family="Times,serif" font-size="11.00" fill="gray">deleting</text>
</g>
<!-- active&#45;&gt;deleting -->
<g id="edge8" class="edge"><title>active&#45;&gt;deleting</title>
<path fill="none" stroke="black" d="M1199.12,-402.939C1250.06,-398.041 1366.1,-385.541 1462,-365.309 1466.09,-364.446 1470.34,-363.425 1474.55,-362.333"/>
<polygon fill="black" stroke="black" points="1475.49,-365.706 1484.21,-359.699 1473.64,-358.953 1475.49,-365.706"/>
<text text-anchor="middle" x="1317" y="-397.709" font-family="Times,serif" font-size="12.00">deleted (via API)</text>
</g>
<!-- error -->
<g id="node10" class="node"><title>error</title>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="1689" cy="-387.309" rx="27" ry="18"/>
<text text-anchor="middle" x="1689" y="-384.509" font-family="Times,serif" font-size="11.00" fill="red">error</text>
</g>
<!-- deleting&#45;&gt;error -->
<g id="edge30" class="edge"><title>deleting&#45;&gt;error</title>
<path fill="none" stroke="black" d="M1541.12,-344.722C1568.37,-339.691 1610.39,-335.56 1644,-348.309 1652.92,-351.693 1661.19,-357.802 1668.09,-364.196"/>
<polygon fill="black" stroke="black" points="1666,-367.058 1675.53,-371.687 1670.97,-362.125 1666,-367.058"/>
<text text-anchor="middle" x="1603" y="-351.709" font-family="Times,serif" font-size="12.00" fill="gray">error</text>
<g id="edge9" class="edge"><title>active&#45;&gt;deleting</title>
<path fill="none" stroke="black" d="M1198.7,-407.281C1249.41,-400.613 1365.74,-384.222 1462,-362.55 1466.08,-361.632 1470.32,-360.576 1474.53,-359.464"/>
<polygon fill="black" stroke="black" points="1475.47,-362.834 1484.18,-356.807 1473.61,-356.085 1475.47,-362.834"/>
<text text-anchor="middle" x="1317" y="-399.95" font-family="Times,serif" font-size="12.00">deleted (via API)</text>
</g>
<!-- deleting&#45;&gt;cleaning -->
<g id="edge31" class="edge"><title>deleting&#45;&gt;cleaning</title>
<path fill="none" stroke="black" d="M1503.74,-333.842C1482.69,-286.333 1416.8,-158.309 1318,-158.309 762,-158.309 762,-158.309 762,-158.309 694.485,-158.309 676.546,-162.51 612,-182.309 602.638,-185.181 592.863,-189.242 583.986,-193.383"/>
<polygon fill="black" stroke="black" points="582.238,-190.341 574.769,-197.855 585.293,-196.639 582.238,-190.341"/>
<text text-anchor="middle" x="1065" y="-161.709" font-family="Times,serif" font-size="12.00" fill="gray">clean</text>
<g id="edge32" class="edge"><title>deleting&#45;&gt;cleaning</title>
<path fill="none" stroke="black" d="M1503.74,-331.082C1482.69,-283.573 1416.8,-155.55 1318,-155.55 762,-155.55 762,-155.55 762,-155.55 694.339,-155.55 676.135,-159.993 612,-181.55 601.998,-184.912 591.612,-189.742 582.35,-194.591"/>
<polygon fill="black" stroke="black" points="580.63,-191.542 573.517,-199.394 583.974,-197.692 580.63,-191.542"/>
<text text-anchor="middle" x="1065" y="-158.95" font-family="Times,serif" font-size="12.00" fill="gray">clean</text>
</g>
<!-- error -->
<g id="node11" class="node"><title>error</title>
<ellipse fill="none" stroke="black" stroke-width="1.7" cx="1689" cy="-384.55" rx="27" ry="18"/>
<text text-anchor="middle" x="1689" y="-381.75" font-family="Times,serif" font-size="11.00" fill="red">error</text>
</g>
<!-- deleting&#45;&gt;error -->
<g id="edge31" class="edge"><title>deleting&#45;&gt;error</title>
<path fill="none" stroke="black" d="M1541.12,-341.962C1568.37,-336.931 1610.39,-332.8 1644,-345.55 1652.92,-348.933 1661.19,-355.042 1668.09,-361.437"/>
<polygon fill="black" stroke="black" points="1666,-364.298 1675.53,-368.928 1670.97,-359.366 1666,-364.298"/>
<text text-anchor="middle" x="1603" y="-348.95" font-family="Times,serif" font-size="12.00" fill="gray">error</text>
</g>
<!-- error&#45;&gt;deploying -->
<g id="edge9" class="edge"><title>error&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1670.75,-400.896C1640.6,-423.243 1575.53,-465.309 1513,-465.309 1171,-465.309 1171,-465.309 1171,-465.309 1104.94,-465.309 1077.73,-481.741 1024,-443.309 995.197,-422.708 979.419,-383.758 971.593,-357.176"/>
<polygon fill="black" stroke="black" points="974.909,-356.032 968.886,-347.316 968.159,-357.886 974.909,-356.032"/>
<text text-anchor="middle" x="1317" y="-468.709" font-family="Times,serif" font-size="12.00">rebuild (via API)</text>
<g id="edge10" class="edge"><title>error&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1673.16,-399.706C1644.78,-426.988 1579.69,-481.55 1513,-481.55 1171,-481.55 1171,-481.55 1171,-481.55 1104.94,-481.55 1076.23,-499.999 1024,-459.55 989.177,-432.581 974.65,-381.143 968.777,-348.968"/>
<polygon fill="black" stroke="black" points="972.19,-348.151 967.101,-338.858 965.284,-349.296 972.19,-348.151"/>
<text text-anchor="middle" x="1317" y="-484.95" font-family="Times,serif" font-size="12.00">rebuild (via API)</text>
</g>
<!-- error&#45;&gt;deleting -->
<g id="edge10" class="edge"><title>error&#45;&gt;deleting</title>
<path fill="none" stroke="black" d="M1662.26,-383.629C1636.68,-379.781 1596.41,-373.207 1562,-365.309 1557.93,-364.374 1553.69,-363.308 1549.48,-362.189"/>
<polygon fill="black" stroke="black" points="1550.4,-358.811 1539.83,-359.525 1548.53,-365.559 1550.4,-358.811"/>
<text text-anchor="middle" x="1603" y="-383.709" font-family="Times,serif" font-size="12.00">deleted (via API)</text>
</g>
<!-- deploy failed&#45;&gt;deploying -->
<g id="edge19" class="edge"><title>deploy failed&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1272.94,-283.399C1215.09,-280.765 1109.63,-280.481 1024,-305.309 1016.23,-307.561 1008.09,-310.485 1000.42,-313.513"/>
<polygon fill="black" stroke="black" points="998.946,-310.334 991.018,-317.361 1001.6,-316.812 998.946,-310.334"/>
<text text-anchor="middle" x="1172" y="-289.709" font-family="Times,serif" font-size="12.00">rebuild (via API)</text>
<g id="edge11" class="edge"><title>error&#45;&gt;deleting</title>
<path fill="none" stroke="black" d="M1662.26,-380.869C1636.68,-377.021 1596.41,-370.447 1562,-362.55 1557.93,-361.615 1553.69,-360.548 1549.48,-359.43"/>
<polygon fill="black" stroke="black" points="1550.4,-356.052 1539.83,-356.766 1548.53,-362.8 1550.4,-356.052"/>
<text text-anchor="middle" x="1603" y="-380.95" font-family="Times,serif" font-size="12.00">deleted (via API)</text>
</g>
<!-- deploy failed&#45;&gt;deploying -->
<g id="edge20" class="edge"><title>deploy failed&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1280.32,-276.116C1262.38,-271.511 1240.22,-266.596 1220,-264.309 1132.95,-254.462 1103.86,-249.275 1024,-285.309 1011.81,-290.811 999.72,-299.171 989.691,-307.132"/>
<polygon fill="black" stroke="black" points="987.276,-304.587 981.794,-313.653 991.733,-309.985 987.276,-304.587"/>
<text text-anchor="middle" x="1172" y="-267.709" font-family="Times,serif" font-size="12.00">active (via API)</text>
<path fill="none" stroke="black" d="M1273.1,-280.157C1215.43,-276.818 1110.18,-275.048 1024,-297.55 1016.3,-299.561 1008.23,-302.276 1000.62,-305.134"/>
<polygon fill="black" stroke="black" points="999.326,-301.882 991.292,-308.789 1001.88,-308.4 999.326,-301.882"/>
<text text-anchor="middle" x="1172" y="-283.95" font-family="Times,serif" font-size="12.00">rebuild (via API)</text>
</g>
<!-- deploy failed&#45;&gt;deploying -->
<g id="edge21" class="edge"><title>deploy failed&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1280.8,-273.016C1262.79,-268.167 1240.42,-262.963 1220,-260.55 1133.13,-250.288 1104.63,-244.616 1024,-278.55 1012.17,-283.53 1000.39,-291.217 990.505,-298.661"/>
<polygon fill="black" stroke="black" points="988.053,-296.136 982.351,-305.066 992.377,-301.641 988.053,-296.136"/>
<text text-anchor="middle" x="1172" y="-263.95" font-family="Times,serif" font-size="12.00">active (via API)</text>
</g>
<!-- deploy failed&#45;&gt;deleting -->
<g id="edge21" class="edge"><title>deploy failed&#45;&gt;deleting</title>
<path fill="none" stroke="black" d="M1357.34,-293.952C1386.77,-300.32 1427.67,-310.567 1462,-324.309 1468.42,-326.88 1475.06,-330.111 1481.3,-333.434"/>
<polygon fill="black" stroke="black" points="1479.65,-336.517 1490.09,-338.305 1483.04,-330.395 1479.65,-336.517"/>
<text text-anchor="middle" x="1421" y="-327.709" font-family="Times,serif" font-size="12.00">deleted (via API)</text>
<g id="edge22" class="edge"><title>deploy failed&#45;&gt;deleting</title>
<path fill="none" stroke="black" d="M1357.34,-291.193C1386.77,-297.561 1427.67,-307.808 1462,-321.55 1468.42,-324.121 1475.06,-327.351 1481.3,-330.675"/>
<polygon fill="black" stroke="black" points="1479.65,-333.758 1490.09,-335.545 1483.04,-327.636 1479.65,-333.758"/>
<text text-anchor="middle" x="1421" y="-324.95" font-family="Times,serif" font-size="12.00">deleted (via API)</text>
</g>
<!-- wait call&#45;back&#45;&gt;deploying -->
<g id="edge16" class="edge"><title>wait call&#45;back&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1141.57,-337.365C1130.66,-332.9 1118.01,-328.539 1106,-326.309 1073.77,-320.325 1036.68,-321.421 1008.63,-323.828"/>
<polygon fill="black" stroke="black" points="1007.98,-320.374 998.357,-324.803 1008.64,-327.343 1007.98,-320.374"/>
<text text-anchor="middle" x="1065" y="-329.709" font-family="Times,serif" font-size="12.00" fill="gray">resume</text>
</g>
<!-- wait call&#45;back&#45;&gt;deploy failed -->
<g id="edge17" class="edge"><title>wait call&#45;back&#45;&gt;deploy failed</title>
<path fill="none" stroke="black" d="M1203.33,-337.525C1224.94,-327.699 1254.18,-314.41 1277.45,-303.832"/>
<polygon fill="black" stroke="black" points="1279.04,-306.954 1286.7,-299.63 1276.14,-300.582 1279.04,-306.954"/>
<text text-anchor="middle" x="1246" y="-323.709" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
<g id="edge17" class="edge"><title>wait call&#45;back&#45;&gt;deploying</title>
<path fill="none" stroke="black" d="M1145.71,-333.256C1134.06,-327.073 1119.78,-320.668 1106,-317.55 1073.78,-310.26 1036.35,-311.488 1008.16,-314.28"/>
<polygon fill="black" stroke="black" points="1007.78,-310.801 998.217,-315.366 1008.54,-317.759 1007.78,-310.801"/>
<text text-anchor="middle" x="1065" y="-320.95" font-family="Times,serif" font-size="12.00" fill="gray">resume</text>
</g>
<!-- wait call&#45;back&#45;&gt;deleting -->
<g id="edge18" class="edge"><title>wait call&#45;back&#45;&gt;deleting</title>
<path fill="none" stroke="black" d="M1220.35,-351.309C1286.99,-351.309 1407.31,-351.309 1470.2,-351.309"/>
<polygon fill="black" stroke="black" points="1470.52,-354.81 1480.52,-351.309 1470.52,-347.81 1470.52,-354.81"/>
<text text-anchor="middle" x="1317" y="-354.709" font-family="Times,serif" font-size="12.00">deleted (via API)</text>
<g id="edge19" class="edge"><title>wait call&#45;back&#45;&gt;deleting</title>
<path fill="none" stroke="black" d="M1220.35,-348.55C1286.99,-348.55 1407.31,-348.55 1470.2,-348.55"/>
<polygon fill="black" stroke="black" points="1470.52,-352.05 1480.52,-348.55 1470.52,-345.05 1470.52,-352.05"/>
<text text-anchor="middle" x="1317" y="-351.95" font-family="Times,serif" font-size="12.00">deleted (via API)</text>
</g>
<!-- wait call&#45;back&#45;&gt;deploy failed -->
<g id="edge18" class="edge"><title>wait call&#45;back&#45;&gt;deploy failed</title>
<path fill="none" stroke="black" d="M1203.33,-334.765C1224.94,-324.939 1254.18,-311.651 1277.45,-301.072"/>
<polygon fill="black" stroke="black" points="1279.04,-304.195 1286.7,-296.87 1276.14,-297.822 1279.04,-304.195"/>
<text text-anchor="middle" x="1246" y="-320.95" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- clean failed&#45;&gt;manageable -->
<g id="edge29" class="edge"><title>clean failed&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M940.499,-201.349C929.799,-194.764 916.612,-187.387 904,-182.309 816.505,-147.084 791.626,-142.727 698,-131.309 591.548,-118.328 465.6,-125.555 397.254,-131.343"/>
<polygon fill="black" stroke="black" points="396.853,-127.864 387.195,-132.22 397.461,-134.838 396.853,-127.864"/>
<text text-anchor="middle" x="655" y="-134.709" font-family="Times,serif" font-size="12.00">manage (via API)</text>
<g id="edge30" class="edge"><title>clean failed&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M939.234,-199.96C928.717,-193.956 916.028,-187.308 904,-182.55 815.986,-147.736 791.872,-140.649 698,-128.55 585.203,-114.012 453.168,-150.875 387.862,-172.744"/>
<polygon fill="black" stroke="black" points="386.682,-169.448 378.343,-175.984 388.938,-176.074 386.682,-169.448"/>
<text text-anchor="middle" x="655" y="-131.95" font-family="Times,serif" font-size="12.00">manage (via API)</text>
</g>
<!-- clean wait&#45;&gt;clean failed -->
<g id="edge26" class="edge"><title>clean wait&#45;&gt;clean failed</title>
<path fill="none" stroke="black" d="M800.964,-206.538C832.338,-208.43 877.662,-211.163 912.401,-213.258"/>
<polygon fill="black" stroke="black" points="912.623,-216.778 922.815,-213.886 913.044,-209.791 912.623,-216.778"/>
<text text-anchor="middle" x="866" y="-215.709" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
<!-- clean wait&#45;&gt;cleaning -->
<g id="edge29" class="edge"><title>clean wait&#45;&gt;cleaning</title>
<path fill="none" stroke="black" d="M728.957,-193.498C698.382,-187.27 651.785,-180.888 612,-188.55 602.955,-190.292 593.541,-193.385 584.924,-196.783"/>
<polygon fill="black" stroke="black" points="583.465,-193.6 575.596,-200.694 586.172,-200.055 583.465,-193.6"/>
<text text-anchor="middle" x="655" y="-191.95" font-family="Times,serif" font-size="12.00" fill="gray">resume</text>
</g>
<!-- clean wait&#45;&gt;clean failed -->
<g id="edge27" class="edge"><title>clean wait&#45;&gt;clean failed</title>
<path fill="none" stroke="black" d="M796.273,-195.531C824.845,-189.071 867.549,-182.634 904,-190.309 912.176,-192.031 920.601,-194.959 928.434,-198.244"/>
<polygon fill="black" stroke="black" points="927.082,-201.474 937.639,-202.371 929.945,-195.086 927.082,-201.474"/>
<text text-anchor="middle" x="866" y="-193.709" font-family="Times,serif" font-size="12.00">abort (via API)</text>
<path fill="none" stroke="black" d="M800.964,-203.965C832.338,-206.014 877.662,-208.975 912.401,-211.245"/>
<polygon fill="black" stroke="black" points="912.609,-214.765 922.815,-211.925 913.065,-207.78 912.609,-214.765"/>
<text text-anchor="middle" x="866" y="-213.95" font-family="Times,serif" font-size="12.00" fill="gray">fail</text>
</g>
<!-- clean wait&#45;&gt;cleaning -->
<g id="edge28" class="edge"><title>clean wait&#45;&gt;cleaning</title>
<path fill="none" stroke="black" d="M729.336,-195.745C719.352,-193.512 708.29,-191.422 698,-190.309 659.999,-186.202 649.729,-184.187 612,-190.309 603.755,-191.647 595.112,-193.956 587.036,-196.541"/>
<polygon fill="black" stroke="black" points="585.865,-193.242 577.531,-199.783 588.125,-199.867 585.865,-193.242"/>
<text text-anchor="middle" x="655" y="-193.709" font-family="Times,serif" font-size="12.00" fill="gray">resume</text>
<!-- clean wait&#45;&gt;clean failed -->
<g id="edge28" class="edge"><title>clean wait&#45;&gt;clean failed</title>
<path fill="none" stroke="black" d="M797.117,-193.449C825.68,-187.71 867.901,-182.218 904,-189.55 911.895,-191.154 920.046,-193.851 927.674,-196.894"/>
<polygon fill="black" stroke="black" points="926.429,-200.168 937,-200.879 929.179,-193.731 926.429,-200.168"/>
<text text-anchor="middle" x="866" y="-192.95" font-family="Times,serif" font-size="12.00">abort (via API)</text>
</g>
<!-- inspect failed&#45;&gt;manageable -->
<g id="edge34" class="edge"><title>inspect failed&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M716.918,-53.1014C648.112,-56.9742 513.552,-69.0572 406,-106.309 397.571,-109.229 388.829,-113.136 380.739,-117.164"/>
<polygon fill="black" stroke="black" points="378.871,-114.19 371.599,-121.895 382.089,-120.406 378.871,-114.19"/>
<text text-anchor="middle" x="551" y="-81.7094" font-family="Times,serif" font-size="12.00">manage (via API)</text>
<g id="edge35" class="edge"><title>inspect failed&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M716.878,-56.0731C633.005,-61.4772 456.973,-76.3869 406,-108.55 386.09,-121.113 370.544,-142.979 360.419,-160.525"/>
<polygon fill="black" stroke="black" points="357.104,-159.292 355.359,-169.743 363.24,-162.661 357.104,-159.292"/>
<text text-anchor="middle" x="551" y="-81.95" font-family="Times,serif" font-size="12.00">manage (via API)</text>
</g>
<!-- inspect failed&#45;&gt;inspecting -->
<g id="edge35" class="edge"><title>inspect failed&#45;&gt;inspecting</title>
<path fill="none" stroke="black" d="M734.727,-37.0181C723.645,-31.9591 710.52,-26.887 698,-24.3094 665.054,-17.5267 627.033,-17.5072 598.051,-19.0618"/>
<polygon fill="black" stroke="black" points="597.573,-15.5846 587.809,-19.6961 598.006,-22.5712 597.573,-15.5846"/>
<text text-anchor="middle" x="655" y="-27.7094" font-family="Times,serif" font-size="12.00">inspect (via API)</text>
<g id="edge36" class="edge"><title>inspect failed&#45;&gt;inspecting</title>
<path fill="none" stroke="black" d="M734.727,-39.2587C723.645,-34.1997 710.52,-29.1276 698,-26.55 665.054,-19.7673 627.033,-19.7478 598.051,-21.3024"/>
<polygon fill="black" stroke="black" points="597.573,-17.8252 587.809,-21.9367 598.006,-24.8118 597.573,-17.8252"/>
<text text-anchor="middle" x="655" y="-29.95" font-family="Times,serif" font-size="12.00">inspect (via API)</text>
</g>
<!-- adopt failed&#45;&gt;manageable -->
<g id="edge40" class="edge"><title>adopt failed&#45;&gt;manageable</title>
<path fill="none" stroke="black" d="M732.005,-418.332C651.178,-385.257 433.099,-294.955 406,-272.55 386.665,-256.564 370.867,-232.724 360.501,-214.356"/>
<polygon fill="black" stroke="black" points="363.439,-212.432 355.586,-205.32 357.29,-215.777 363.439,-212.432"/>
<text text-anchor="middle" x="551" y="-361.95" font-family="Times,serif" font-size="12.00">manage (via API)</text>
</g>
<!-- adopt failed&#45;&gt;adopting -->
<g id="edge39" class="edge"><title>adopt failed&#45;&gt;adopting</title>
<path fill="none" stroke="black" d="M728.401,-420.602C718.68,-418.147 707.995,-415.85 698,-414.55 663.074,-410.007 623.054,-409.806 593.757,-410.573"/>
<polygon fill="black" stroke="black" points="593.364,-407.083 583.478,-410.892 593.582,-414.08 593.364,-407.083"/>
<text text-anchor="middle" x="655" y="-417.95" font-family="Times,serif" font-size="12.00">adopt (via API)</text>
</g>
</g>
</svg>

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 25 KiB

1
doc/source/index.rst
View File

@ -41,6 +41,7 @@ Administrator's Guide
deploy/raid
deploy/inspection
deploy/security
deploy/adoption
deploy/troubleshooting
Release Notes <http://docs.openstack.org/releasenotes/ironic/>

8
doc/source/webapi/v1.rst
View File

@ -32,7 +32,15 @@ always requests the newest supported API version.
API Versions History
--------------------
**1.17**
Addition of provision_state verb ``adopt`` which allows an operator
to move a node from ``manageable`` state to ``active`` state without
performing a deployment operation on the node. This is intended for
nodes that have already been deployed by external means.
**1.16**
Add ability to filter nodes by driver.
**1.15**

3
ironic/api/controllers/v1/node.py
View File

@ -94,7 +94,8 @@ _DEFAULT_RETURN_FIELDS = ('instance_uuid', 'maintenance', 'power_state',
# States where calling do_provisioning_action makes sense
PROVISION_ACTION_STATES = (ir_states.VERBS['manage'],
ir_states.VERBS['provide'],
ir_states.VERBS['abort'])
ir_states.VERBS['abort'],
ir_states.VERBS['adopt'])
_NODES_CONTROLLER_RESERVED_WORDS = None

1
ironic/api/controllers/v1/utils.py
View File

@ -51,6 +51,7 @@ MIN_VERB_VERSIONS = {
states.VERBS['inspect']: versions.MINOR_6_INSPECT_STATE,
states.VERBS['abort']: versions.MINOR_13_ABORT_VERB,
states.VERBS['clean']: versions.MINOR_15_MANUAL_CLEAN,
states.VERBS['adopt']: versions.MINOR_17_ADOPT_VERB,
}

4
ironic/api/controllers/v1/versions.py
View File

@ -46,6 +46,7 @@ BASE_VERSION = 1
# 2. '/v1/drivers/<driver-name>/properties'
# v1.15: Add ability to do manual cleaning of nodes
# v1.16: Add ability to filter nodes by driver.
# v1.17: Add 'adopt' verb for ADOPTING active nodes.
MINOR_0_JUNO = 0
MINOR_1_INITIAL_VERSION = 1
@ -64,11 +65,12 @@ MINOR_13_ABORT_VERB = 13
MINOR_14_LINKS_NODESTATES_DRIVERPROPERTIES = 14
MINOR_15_MANUAL_CLEAN = 15
MINOR_16_DRIVER_FILTER = 16
MINOR_17_ADOPT_VERB = 17
# When adding another version, update MINOR_MAX_VERSION and also update
# doc/source/webapi/v1.rst with a detailed explanation of what the version has
# changed.
MINOR_MAX_VERSION = MINOR_16_DRIVER_FILTER
MINOR_MAX_VERSION = MINOR_17_ADOPT_VERB
# String representations of the minor and maximum versions
MIN_VERSION_STRING = '{}.{}'.format(BASE_VERSION, MINOR_1_INITIAL_VERSION)

43
ironic/common/states.py
View File

@ -47,6 +47,7 @@ VERBS = {
'inspect': 'inspect',
'abort': 'abort',
'clean': 'clean',
'adopt': 'adopt',
}
""" Mapping of state-changing events that are PUT to the REST API
@ -161,19 +162,35 @@ INSPECTING = 'inspecting'
""" Node is under inspection.
This is the provision state used when inspection is started. A successfully
inspected node shall transition to MANAGEABLE status.
inspected node shall transition to MANAGEABLE state.
"""
INSPECTFAIL = 'inspect failed'
""" Node inspection failed. """
ADOPTING = 'adopting'
""" Node is being adopted.
This provision state is intended for use to move a node from MANAGEABLE to
ACTIVE state to permit designation of nodes as being "managed" by Ironic,
however "deployed" previously by external means.
"""
ADOPTFAIL = 'adopt failed'
""" Node failed to complete the adoption process.
This state is the resulting state of a node that failed to complete adoption,
potentially due to invalid or incompatible information being defined for the
node.
"""
UPDATE_ALLOWED_STATES = (DEPLOYFAIL, INSPECTING, INSPECTFAIL, CLEANFAIL, ERROR,
VERIFYING)
VERIFYING, ADOPTFAIL)
"""Transitional states in which we allow updating a node."""
DELETE_ALLOWED_STATES = (AVAILABLE, NOSTATE, MANAGEABLE, ENROLL)
DELETE_ALLOWED_STATES = (AVAILABLE, NOSTATE, MANAGEABLE, ENROLL,
ADOPTFAIL)
"""States in which node deletion is allowed."""
STABLE_STATES = (ENROLL, MANAGEABLE, AVAILABLE, ACTIVE, ERROR)
@ -243,6 +260,10 @@ machine.add_transition(AVAILABLE, DEPLOYING, 'deploy')
machine.add_state(INSPECTING, target=MANAGEABLE, **watchers)
machine.add_state(INSPECTFAIL, target=MANAGEABLE, **watchers)
# Add adopt* states
machine.add_state(ADOPTING, target=ACTIVE, **watchers)
machine.add_state(ADOPTFAIL, target=ACTIVE, **watchers)
# A deployment may fail
machine.add_transition(DEPLOYING, DEPLOYFAIL, 'fail')
@ -346,3 +367,19 @@ machine.add_transition(VERIFYING, MANAGEABLE, 'done')
# Verification can fail with setting last_error and rolling back to ENROLL
machine.add_transition(VERIFYING, ENROLL, 'fail')
# Node Adoption is being attempted
machine.add_transition(MANAGEABLE, ADOPTING, 'adopt')
# Adoption can succeed and the node should be set to ACTIVE
machine.add_transition(ADOPTING, ACTIVE, 'done')
# Node adoptions can fail and as such nodes shall be set
# into a dedicated state to hold the nodes.
machine.add_transition(ADOPTING, ADOPTFAIL, 'fail')
# Node adoption can be retried when it previously failed.
machine.add_transition(ADOPTFAIL, ADOPTING, 'adopt')
# A node that failed adoption can be moved back to manageable
machine.add_transition(ADOPTFAIL, MANAGEABLE, 'manage')

58
ironic/conductor/manager.py
View File

@ -1120,6 +1120,16 @@ class ConductorManager(base_manager.BaseConductorManager):
err_handler=utils.provisioning_error_handler)
return
if (action == states.VERBS['adopt'] and
node.provision_state in (states.MANAGEABLE,
states.ADOPTFAIL)):
task.process_event(
'adopt',
callback=self._spawn_worker,
call_args=(self._do_adoption, task),
err_handler=utils.provisioning_error_handler)
return
if (action == states.VERBS['abort'] and
node.provision_state == states.CLEANWAIT):
@ -1306,6 +1316,54 @@ class ConductorManager(base_manager.BaseConductorManager):
callback_method=utils.cleanup_after_timeout,
err_handler=utils.provisioning_error_handler)
@task_manager.require_exclusive_lock
def _do_adoption(self, task):
"""Adopt the node.
Similar to node takeover, adoption performs a driver boot
validation and then triggers node takeover in order to make the
conductor responsible for the node. Upon completion of takeover,
the node is moved to ACTIVE state.
The goal of this method is to set the conditions for the node to
be managed by Ironic as an ACTIVE node without having performed
a deployment operation.
:param task: a TaskManager instance
"""
node = task.node
LOG.debug('Conductor %(cdr)s attempting to adopt node %(node)s',
{'cdr': self.host, 'node': node.uuid})
try:
# NOTE(TheJulia): A number of drivers expect to know if a
# whole disk image was used prior to their takeover logic
# being triggered, as such we need to populate the
# internal info based on the configuration the user has
# supplied.
iwdi = images.is_whole_disk_image(task.context,
task.node.instance_info)
node.driver_internal_info['is_whole_disk_image'] = iwdi
# Calling boot validate to ensure that sufficient information
# is supplied to allow the node to be able to boot if takeover
# writes items such as kernel/ramdisk data to disk.
task.driver.boot.validate(task)
# NOTE(TheJulia): While task.driver.boot.validate() is called
# above, and task.driver.power.validate() could be called, it
# is called as part of the transition from ENROLL to MANAGEABLE
# states. As such it is redundant to call here.
self._do_takeover(task)
LOG.info(_LI("Successfully adopted node %(node)s"),
{'node': node.uuid})
task.process_event('done')
except Exception as err:
msg = (_('Error while attempting to adopt node %(node)s: '
'%(err)s.') % {'node': node.uuid, 'err': err})
LOG.error(msg)
node.last_error = msg
task.process_event('fail')
def _do_takeover(self, task):
"""Take over this node.

19
ironic/conductor/utils.py
View File

@ -41,6 +41,14 @@ CLEANING_INTERFACE_PRIORITY = {
def node_set_boot_device(task, device, persistent=False):
"""Set the boot device for a node.
Sets the boot device for a node if the node's driver interface
contains a 'management' interface.
If the node that the boot device change is being requested for
is in ADOPTING state, the boot device will not be set as that
change could potentially result in the future running state of
an adopted node being modified erroneously.
:param task: a TaskManager instance.
:param device: Boot device. Values are vendor-specific.
:param persistent: Whether to set next-boot, or make the change
@ -51,9 +59,14 @@ def node_set_boot_device(task, device, persistent=False):
"""
if getattr(task.driver, 'management', None):
task.driver.management.validate(task)
task.driver.management.set_boot_device(task,
device=device,
persistent=persistent)
# NOTE(TheJulia): When a node is in the ADOPTING state, we must
# not attempt to change the default boot device as a side effect
# of a driver's node takeover process as it is modifying
# a working machine.
if task.node.provision_state != states.ADOPTING:
task.driver.management.set_boot_device(task,
device=device,
persistent=persistent)
@task_manager.require_exclusive_lock

87
ironic/tests/unit/api/v1/test_nodes.py
View File

@ -2184,6 +2184,93 @@ class TestPut(test_api_base.BaseApiTest):
mock_rpcapi.assert_called_once_with(mock.ANY, self.node.uuid,
clean_steps, 'test-topic')
def test_adopt_raises_error_before_1_17(self):
"""Test that a lower API client cannot use the adopt verb"""
ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
{'target': states.VERBS['adopt']},
headers={api_base.Version.string: "1.16"},
expect_errors=True)
self.assertEqual(http_client.NOT_ACCEPTABLE, ret.status_code)
@mock.patch.object(rpcapi.ConductorAPI, 'do_provisioning_action')
def test_adopt_from_manage(self, mock_dpa):
"""Test that a node can be adopted from the manageable state"""
self.node.provision_state = states.MANAGEABLE
self.node.save()
ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
{'target': states.VERBS['adopt']},
headers={api_base.Version.string: "1.17"})
self.assertEqual(http_client.ACCEPTED, ret.status_code)
self.assertEqual(b'', ret.body)
mock_dpa.assert_called_once_with(mock.ANY, self.node.uuid,
states.VERBS['adopt'],
'test-topic')
@mock.patch.object(rpcapi.ConductorAPI, 'do_provisioning_action')
def test_adopt_from_adoption_failed(self, mock_dpa):
self.node.provision_state = states.ADOPTFAIL
self.node.save()
ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
{'target': states.VERBS['adopt']},
headers={api_base.Version.string: "1.17"})
self.assertEqual(http_client.ACCEPTED, ret.status_code)
self.assertEqual(b'', ret.body)
mock_dpa.assert_called_once_with(mock.ANY, self.node.uuid,
states.VERBS['adopt'],
'test-topic')
@mock.patch.object(rpcapi.ConductorAPI, 'do_provisioning_action')
def test_adopt_from_active_fails(self, mock_dpa):
self.node.provision_state = states.ACTIVE
self.node.save()
ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
{'target': states.VERBS['adopt']},
headers={api_base.Version.string: "1.17"},
expect_errors=True)
self.assertEqual(http_client.BAD_REQUEST, ret.status_code)
self.assertEqual(0, mock_dpa.call_count)
@mock.patch.object(rpcapi.ConductorAPI, 'do_provisioning_action')
def test_manage_from_adoption_failed(self, mock_dpa):
self.node.provision_state = states.ADOPTFAIL
self.node.save()
ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
{'target': states.VERBS['manage']},
headers={api_base.Version.string: "1.17"})
self.assertEqual(http_client.ACCEPTED, ret.status_code)
self.assertEqual(b'', ret.body)
mock_dpa.assert_called_once_with(mock.ANY, self.node.uuid,
states.VERBS['manage'],
'test-topic')
@mock.patch.object(rpcapi.ConductorAPI, 'do_provisioning_action')
def test_bad_requests_in_adopting_state(self, mock_dpa):
self.node.provision_state = states.ADOPTING
self.node.save()
for state in [states.ACTIVE, states.REBUILD, states.DELETED]:
ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
{'target': state},
expect_errors=True)
self.assertEqual(http_client.BAD_REQUEST, ret.status_code)
self.assertEqual(0, mock_dpa.call_count)
@mock.patch.object(rpcapi.ConductorAPI, 'do_provisioning_action')
def test_bad_requests_in_adoption_failed_state(self, mock_dpa):
self.node.provision_state = states.ADOPTFAIL
self.node.save()
for state in [states.ACTIVE, states.REBUILD, states.DELETED]:
ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
{'target': state},
expect_errors=True)
self.assertEqual(http_client.BAD_REQUEST, ret.status_code)
self.assertEqual(0, mock_dpa.call_count)
def test_set_console_mode_enabled(self):
with mock.patch.object(rpcapi.ConductorAPI,
'set_console_mode') as mock_scm:

11
ironic/tests/unit/api/v1/test_utils.py
View File

@ -182,6 +182,17 @@ class TestApiUtils(base.TestCase):
mock_request.version.minor = 10
self.assertFalse(utils.allow_links_node_states_and_driver_properties())
@mock.patch.object(pecan, 'request', spec_set=['version'])
def test_check_allow_adopt_verbs_fail(self, mock_request):
mock_request.version.minor = 16
self.assertRaises(exception.NotAcceptable,
utils.check_allow_management_verbs, 'adopt')
@mock.patch.object(pecan, 'request', spec_set=['version'])
def test_check_allow_adopt_verbs(self, mock_request):
mock_request.version.minor = 17
utils.check_allow_management_verbs('adopt')
class TestNodeIdent(base.TestCase):

150
ironic/tests/unit/conductor/test_manager.py
View File

@ -2610,6 +2610,16 @@ class DestroyNodeTestCase(mgr_utils.ServiceSetUpMixin,
self.dbapi.get_node_by_uuid,
node.uuid)
def test_destroy_node_adopt_failed_no_power_change(self):
self._start_service()
node = obj_utils.create_test_node(self.context,
driver='fake',
provision_state=states.ADOPTFAIL)
with mock.patch.object(self.driver.power,
'set_power_state') as mock_power:
self.service.destroy_node(self.context, node.uuid)
self.assertFalse(mock_power.called)
@mgr_utils.mock_record_keepalive
class UpdatePortTestCase(mgr_utils.ServiceSetUpMixin,
@ -4760,3 +4770,143 @@ class DoNodeTakeOverTestCase(mgr_utils.ServiceSetUpMixin,
mock_prepare.assert_called_once_with(mock.ANY)
mock_take_over.assert_called_once_with(mock.ANY)
mock_start_console.assert_called_once_with(mock.ANY)
@mgr_utils.mock_record_keepalive
class DoNodeAdoptionTestCase(
mgr_utils.ServiceSetUpMixin,
tests_db_base.DbTestCase):
@mock.patch('ironic.drivers.modules.fake.FakePower.validate')
@mock.patch('ironic.drivers.modules.fake.FakeBoot.validate')
@mock.patch('ironic.drivers.modules.fake.FakeConsole.start_console')
@mock.patch('ironic.drivers.modules.fake.FakeDeploy.take_over')
@mock.patch('ironic.drivers.modules.fake.FakeDeploy.prepare')
def test__do_adoption_with_takeover(self,
mock_prepare,
mock_take_over,
mock_start_console,
mock_boot_validate,
mock_power_validate):
self._start_service()
node = obj_utils.create_test_node(
self.context, driver='fake',
provision_state=states.ADOPTING)
task = task_manager.TaskManager(self.context, node.uuid)
self.service._do_adoption(task)
node.refresh()
self.assertEqual(states.ACTIVE, node.provision_state)
self.assertIsNone(node.last_error)
self.assertFalse(node.console_enabled)
mock_prepare.assert_called_once_with(mock.ANY)
mock_take_over.assert_called_once_with(mock.ANY)
self.assertFalse(mock_start_console.called)
self.assertTrue(mock_boot_validate.called)
@mock.patch('ironic.drivers.modules.fake.FakeBoot.validate')
@mock.patch('ironic.drivers.modules.fake.FakeConsole.start_console')
@mock.patch('ironic.drivers.modules.fake.FakeDeploy.take_over')
@mock.patch('ironic.drivers.modules.fake.FakeDeploy.prepare')
def test__do_adoption_take_over_failure(self,
mock_prepare,
mock_take_over,
mock_start_console,
mock_boot_validate):
# Note(TheJulia): Use of an actual possible exception that
# can be raised due to a misconfiguration.
mock_take_over.side_effect = exception.IPMIFailure(
"something went wrong")
self._start_service()
node = obj_utils.create_test_node(
self.context, driver='fake',
provision_state=states.ADOPTING)
task = task_manager.TaskManager(self.context, node.uuid)
self.service._do_adoption(task)
node.refresh()
self.assertEqual(states.ADOPTFAIL, node.provision_state)
self.assertIsNotNone(node.last_error)
self.assertFalse(node.console_enabled)
mock_prepare.assert_called_once_with(mock.ANY)
mock_take_over.assert_called_once_with(mock.ANY)
self.assertFalse(mock_start_console.called)
self.assertTrue(mock_boot_validate.called)
@mock.patch('ironic.drivers.modules.fake.FakeBoot.validate')
@mock.patch('ironic.drivers.modules.fake.FakeConsole.start_console')
@mock.patch('ironic.drivers.modules.fake.FakeDeploy.take_over')
@mock.patch('ironic.drivers.modules.fake.FakeDeploy.prepare')
def test__do_adoption_boot_validate_failure(self,
mock_prepare,
mock_take_over,
mock_start_console,
mock_boot_validate):
# Note(TheJulia): Use of an actual possible exception that
# can be raised due to a misconfiguration.
mock_boot_validate.side_effect = exception.MissingParameterValue(
"something is missing")
self._start_service()
node = obj_utils.create_test_node(
self.context, driver='fake',
provision_state=states.ADOPTING)
task = task_manager.TaskManager(self.context, node.uuid)
self.service._do_adoption(task)
node.refresh()
self.assertEqual(states.ADOPTFAIL, node.provision_state)
self.assertIsNotNone(node.last_error)
self.assertFalse(node.console_enabled)
self.assertFalse(mock_prepare.called)
self.assertFalse(mock_take_over.called)
self.assertFalse(mock_start_console.called)
self.assertTrue(mock_boot_validate.called)
@mock.patch('ironic.conductor.manager.ConductorManager._spawn_worker')
def test_do_provisioning_action_adopt_node(self, mock_spawn):
node = obj_utils.create_test_node(
self.context, driver='fake',
provision_state=states.MANAGEABLE,
target_provision_state=states.NOSTATE)
self._start_service()
self.service.do_provisioning_action(self.context, node.uuid, 'adopt')
node.refresh()
self.assertEqual(states.ADOPTING, node.provision_state)
self.assertEqual(states.ACTIVE, node.target_provision_state)
self.assertIsNone(node.last_error)
mock_spawn.assert_called_with(self.service._do_adoption, mock.ANY)
@mock.patch('ironic.conductor.manager.ConductorManager._spawn_worker')
def test_do_provisioning_action_adopt_node_retry(self, mock_spawn):
node = obj_utils.create_test_node(
self.context, driver='fake',
provision_state=states.ADOPTFAIL,
target_provision_state=states.ACTIVE)
self._start_service()
self.service.do_provisioning_action(self.context, node.uuid, 'adopt')
node.refresh()
self.assertEqual(states.ADOPTING, node.provision_state)
self.assertEqual(states.ACTIVE, node.target_provision_state)
self.assertIsNone(node.last_error)
mock_spawn.assert_called_with(self.service._do_adoption, mock.ANY)
def test_do_provisioning_action_manage_of_failed_adoption(self):
node = obj_utils.create_test_node(
self.context, driver='fake',
provision_state=states.ADOPTFAIL,
target_provision_state=states.ACTIVE)
self._start_service()
self.service.do_provisioning_action(self.context, node.uuid, 'manage')
node.refresh()
self.assertEqual(states.MANAGEABLE, node.provision_state)
self.assertEqual(states.NOSTATE, node.target_provision_state)
self.assertIsNone(node.last_error)

17
ironic/tests/unit/conductor/test_utils.py
View File

@ -60,6 +60,23 @@ class NodeSetBootDeviceTestCase(base.DbTestCase):
device='pxe',
persistent=False)
def test_node_set_boot_device_adopting(self):
mgr_utils.mock_the_extension_manager(driver="fake_ipmitool")
self.driver = driver_factory.get_driver("fake_ipmitool")
ipmi_info = utils.get_test_ipmi_info()
node = obj_utils.create_test_node(self.context,
uuid=uuidutils.generate_uuid(),
driver='fake_ipmitool',
driver_info=ipmi_info,
provision_state=states.ADOPTING)
task = task_manager.TaskManager(self.context, node.uuid)
with mock.patch.object(self.driver.management,
'set_boot_device') as mock_sbd:
conductor_utils.node_set_boot_device(task,
device='pxe')
self.assertFalse(mock_sbd.called)
class NodePowerActionTestCase(base.DbTestCase):

14
releasenotes/notes/active-node-creation-a41c9869c966c82b.yaml Normal file
View File

@ -0,0 +1,14 @@
---
features:
- Addition of the provision state target verb of ``adopt``
which allows an operator to move a node into an ``active``
state from ``manageable`` state, without performing a deployment
operation on the node. This can be used to represent nodes that have
been previously deployed by other means that will now be managed by
ironic and be later released to the available hardware pool.
other:
- When a node is enrolled into ironic, upon transition to the
``manageable`` state, the current power state of the node is
recorded. Once the node is adopted and in an ``active`` state,
that recorded power state will be enfored by ironic unless an
operator changes the power state in ironic.