diff --git a/doc/source/index.rst b/doc/source/index.rst index 90fafc472..56631be3b 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -52,6 +52,7 @@ Developers Documentation :maxdepth: 1 architecture + pluginguide API Documentation ======================== diff --git a/doc/source/pluginguide.rst b/doc/source/pluginguide.rst new file mode 100644 index 000000000..89d0ed0d2 --- /dev/null +++ b/doc/source/pluginguide.rst @@ -0,0 +1,514 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +======================================= +Heat Resource Plug-in Development Guide +======================================= +Heat allows service providers to extend the capabilities of the orchestration +service by writing their own resource plug-ins. These plug-ins are written in +Python and included in a directory configured by the service provider. This +guide describes a resource plug-in structure and life cycle in order to assist +developers in writing their own resource plug-ins. + +Resource Plug-in Life Cycle +--------------------------- +A resource plug-in is relatively simple in that it needs to extend a base +``Resource`` class and implement some relevant life cycle handler methods. +The basic life cycle methods of a resource are: + +create + The plug-in should create a new physical resource. + +update + The plug-in should update an existing resource with new + configuration or tell the engine that the resource must be destroyed + and re-created. This method is optional; the default behavior is to + create a replacement resource and then delete the old resource. + +suspend + The plug-in should suspend operation of the physical resource; this is + an optional operation. + +resume + The plug-in should resume operation of the physical resource; this is an + optional operation. + +delete + The plug-in should delete the physical resource. + +The base class ``Resource`` implements each of these life cycle methods and +defines one or more handler methods that plug-ins should implement in order +to manifest and manage the actual physical resource abstracted by the plug-in. +These handler methods will be described in detail in the following sections. + +Heat Resource Base Class +++++++++++++++++++++++++ +Plug-ins must extend the class ``heat.engine.resource.Resource``. + +This class is responsible for managing the overall life cycle of the plug-in. +It defines methods corresponding to the life cycle as well as the basic hooks +for plug-ins to handle the work of communicating with specific down-stream +services. For example, when the engine determines it is time to create a +resource, it calls the ``create`` method of the applicable plug-in. This method +is implemented in the ``Resource`` base class and handles most of the +bookkeeping and interaction with the engine. This method then calls a +``handle_create`` method defined in the plug-in class (if implemented) which is +responsible for using specific service calls or other methods needed to +instantiate the desired physical resource (server, network, volume, etc). + +Resource Status and Action +************************** + +The base class handles reporting state of the resource back to the engine. +A resource's state is the combination of the life cycle action and the status +of that action. For example, if a resource is created successfully, the status +of that resource will be ``CREATE COMPLETE``. Alternatively, if the plug-in +encounters an error when attempting to create the physical resource, the +status would be ``CREATE FAILED``. The base class handles the +reporting and persisting of resource state, so a plug-in's handler +methods only need to return data or raise exceptions as appropriate. + + +Properties and Attributes ++++++++++++++++++++++++++ +A resource's *properties* define the settings the template author can +manipulate when including that resource in a template. Some examples would be: + +* Which flavor and image to use for a Nova server +* The port to listen to on Neutron LBaaS nodes +* The size of a Cinder volume + +*Attributes* describe runtime state data of the physical resource that the +plug-in can expose to other resources in a Stack. Generally, these aren't +available until the physical resource has been created and is in a usable +state. Some examples would be: + +* The host id of a Nova server +* The status of a Neutron network +* The creation time of a Cinder volume + +Defining Resource Properties +**************************** +Each property that a resource supports must be defined in a schema that informs +the engine and validation logic what the properties are, what type each is, +and validation constraints. The schema is a dictionary whose keys define +property names and whose values describe the constraints on that property. This +dictionary must be assigned to the ``properties_schema`` attribute of the +plug-in. + +.. code-block:: python + + from heat.engine import constraints + from heat.engine import properties + from heat.openstack.common.gettextutils import _ + + nested_schema = { + "foo": properties.Schema( + properties.Schema.STRING, + _('description of foo field'), + constraints=[ + constraints.AllowedPattern('(Ba[rc]?)+'), + constraints.Length(max=10, + description="don't go crazy") + ] + ) + } + + properties_schema = { + "property_name": properties.Schema( + properties.Schema.MAP, + _('Internationalized description of property'), + required=True, + default={"Foo": "Bar"}, + schema": nested_schema + ) + } + +As shown above, some properties may themselves be complex and +reference nested schema definitions. Following are the parameters to +to the ``Schema`` constructor; all but the first have defaults. + +*data_type*: + + Defines the type of the property's value. The valid types are + the members of the list ``properties.Schema.TYPES``, currently + ``INTEGER``, ``STRING``, ``NUMBER``, ``BOOLEAN``, ``MAP``, and + ``LIST``; please use those symbolic names rather than the + literals to which they are equated. For ``LIST`` and ``MAP`` + type properties, the ``schema`` referenced constrains the + format of complex items in the list or map. + +*description*: + A description of the property and its function; also used in documentation + generation. Default is ``None`` --- but you should always provide a + description. + +*default*: + The default value to assign to this property if none was supplied in the + template. Default is ``None``. + +*schema*: + This property's value is complex and its members must conform to + this referenced schema in order to be valid. The referenced schema + dictionary has the same format as the ``properties_schema``. Default + is ``None``. + +*required*: + ``True`` if the property must have a value for the template to be valid; + ``False`` otherwise. The default is ``False`` + +*constraints*: + A list of constraints that apply to the property's value. See + `Property Constraints`_. + +*update_allowed*: + ``True`` if an existing resource can be updated, ``False`` means + update is accomplished by delete and re-create. Default is ``False``. + +Accessing property values of the plug-in at runtime is then a simple call to: + +.. code-block:: python + + self.properties['PropertyName'] + +Based on the property type, properties without a set value will return the +default "empty" value for that type: + +======= ============ +Type Empty Value +======= ============ +String '' +Number 0 +Integer 0 +List [] +Map {} +======= ============ + +Property Constraints +******************** + +Following are the available kinds of constraints. The description is +optional and, if given, states the constraint in plain language for +the end user. + +*AllowedPattern(regex, description)*: + Constrains the value to match the given regular expression; + applicable to STRING. + +*AllowedValues(allowed, description)*: + Lists the allowed values. ``allowed`` must be a + ``collections.Sequence`` or ``basestring``. Applicable to all types + of value except MAP. + +*Length(min, max, description)*: + Constrains the length of the value. Applicable to STRING, LIST, + MAP. Both ``min`` and ``max`` default to ``None``. + +*Range(min, max, description)*: + Constrains a numerical value. Applicable to INTEGER and NUMBER. + Both ``min`` and ``max`` default to ``None``. + +*CustomConstraint(name, description, environment)*: + This constructor brings in a named constraint class from an + environment. If the given environment is ``None`` (its default) + then the environment used is the global one. + +Defining Resource Attributes +**************************** +Attributes communicate runtime state of the physical resource. Note that some +plug-ins do not define any attributes and doing so is optional. If the plug-in +needs to expose attributes, it will define an ``attributes_schema`` similar to +the properties schema described above. This schema, however, is much simpler +to define as each item in the dictionary only defines the attribute name and +a description of the attribute. + +.. code-block:: python + + attributes_schema = { + "foo": _("The foo attribute"), + "bar": _("The bar attribute"), + "baz": _("The baz attribute") + } + +If attributes are defined, their values must also be resolved by the plug-in. +The simplest way to do this is to override the ``_resolve_attribute`` method +from the ``Resource`` class:: + + def _resolve_attribute(self, name): + # _example_get_physical_resource is just an example and is not defined + # in the Resource class + phys_resource = self._example_get_physical_resource() + if phys_resource: + if not hasattr(phys_resource, name): + # this is usually not needed, but this is a simple example + raise exception.InvalidTemplateAttribute(name) + return getattr(phys_resource, name) + return None + +If the plug-in needs to be more sophisticated in its attribute resolution, the +plug-in may instead choose to override ``FnGetAttr``. If this method is chosen, +however, responsibility for validating the attribute and its accessibility is +the responsibility of the plug-in. + +Property and Attribute Example +****************************** +Assume the following simple property and attribute definition: + +.. code-block:: python + + properties_schema = { + 'foo': properties.Schema( + properties.Schema.STRING, + _('foo prop description'), + default='foo', + required=True + ), + 'bar': properies.Schema( + properties.Schema.INTEGER, + _('bar prop description'), + required=True, + constraints=[ + constraints.Range(5, 10) + ] + ) + } + + attributes_schema = { + 'Attr_1': 'The first attribute', + 'Attr_2': 'The second attribute' + } + +Also assume the plug-in defining the above has been registered under the +template reference name 'Resource::Foo' (see `Registering Resource Plug-ins`_). +A template author could then use this plug-in in a stack by simply making +following declarations in a template: + +.. code-block:: yaml + + # ... other sections omitted for brevity ... + + resources: + resource-1: + type: Resource::Foo + properties: + foo: Value of the foo property + bar: 7 + + outputs: + foo-attrib-1: + value: { get_attr: [resource-1, Attr_1] } + description: The first attribute of the foo resource + foo-attrib-2: + value: { get_attr: [resource-1, Attr_2] } + description: The second attribute of the foo resource + +Life Cycle Handler Methods +++++++++++++++++++++++++++ +To do the work of managing the physical resource the plug-in supports, the +following life cycle handler methods should be implemented. Note that the +plug-in need not implement *all* of these methods; optional handlers will +be documented as such. + +Generally, the handler methods follow a basic pattern. The basic +handler method for any life cycle step follows the format +``handle_``. So for the create step, the handler +method would be ``handle_create``. Once a handler is called, an +optional ``check__complete`` may also be implemented +so that the plug-in may return immediately from the basic handler and +then take advantage of cooperative multi-threading built in to the +base class and periodically poll a down-stream service for completion; +the check method is polled until it returns ``True``. Again, for the +create step, this method would be ``check_create_complete``. + +Create +****** +.. py:function:: handle_create(self) + + Create a new physical resource. This function should make the required + calls to create the physical resource and return as soon as there is enough + information to identify the resource. The function should return this + identifying information and implement ``check_create_complete`` which will + take this information in as a parameter and then periodically be polled. + This allows for cooperative multi-threading between multiple resources that + have had their dependencies satisfied. + + *Note* once the native identifier of the physical resource is known, this + function should call ``self.resource_id_set`` passing the native identifier + of the physical resource. This will persist the identifier and make it + available to the plug-in by accessing ``self.resource_id``. + + :returns: A representation of the created physical resource + :raise: any ``Exception`` if the create failed + +.. py:function:: check_create_complete(self, token) + + If defined, will be called with the return value of ``handle_create`` + + :param token: the return value of ``handle_create``; used to poll the + physical resource's status. + :returns: ``True`` if the physical resource is active and ready for use; + ``False`` otherwise. + :raise: any ``Exception`` if the create failed. + +Update (Optional) +***************** +Note that there is a default implementation of ``handle_update`` in +``heat.engine.resource.Resource`` that simply raises an exception indicating +that updates require the engine to delete and re-create the resource +(this is the default behavior) so implementing this is optional. + +.. py:function:: handle_update(self, json_snippet, templ_diff, prop_diff) + + Update the physical resources using updated information. + + :param json_snippet: the resource definition from the updated template + :type json_snippet: collections.Mapping + :param templ_diff: changed values from the original template definition + :type templ_diff: collections.Mapping + :param prop_diff: property values that are different between the original + definition and the updated definition; keys are + property names and values are the new values. Deleted or + properties that were originally present but now absent + have values of ``None`` + :type prop_diff: collections.Mapping + +.. py:function:: check_update_complete(self, token) + + If defined, will be called with the return value of ``handle_update`` + + :param token: the return value of ``handle_update``; used to poll the + physical resource's status. + :returns: ``True`` if the update has finished; + ``False`` otherwise. + :raise: any ``Exception`` if the update failed. + +Suspend (Optional) +****************** +*These handler functions are optional and only need to be implemented if the +physical resource supports suspending* + +.. py:function:: handle_suspend(self) + + If the physical resource supports it, this function should call the native + API and suspend the resource's operation. This function should return + information sufficient for ``check_suspend_complete`` to poll the native + API to verify the operation's status. + + :return: a token containing enough information for ``check_suspend_complete`` + to verify operation status. + :raise: any ``Exception`` if the suspend operation fails. + +.. py:function:: check_suspend_complete(self, token) + + Verify the suspend operation completed successfully. + + :param token: the return value of ``handle_suspend`` + :return: ``True`` if the suspend operation completed and the physical + resource is now suspended; ``False`` otherwise. + :raise: any ``Exception`` if the suspend operation failed. + +Resume (Optional) +****************** +*These handler functions are optional and only need to be implemented if the +physical resource supports resuming from a suspended state* + +.. py:function:: handle_resume(self) + + If the physical resource supports it, this function should call the native + API and resume a suspended resource's operation. This function should return + information sufficient for ``check_resume_complete`` to poll the native + API to verify the operation's status. + + :return: a token containing enough information for ``check_resume_complete`` + to verify operation status. + :raise: any ``Exception`` if the resume operation fails. + +.. py:function:: check_resume_complete(self, token) + + Verify the resume operation completed successfully. + + :param token: the return value of ``handle_resume`` + :return: ``True`` if the resume operation completed and the physical resource + is now active; ``False`` otherwise. + :raise: any Exception if the resume operation failed. + + +Delete +****** +.. py:function:: handle_delete(self) + + Delete the physical resource. + + :return: a token containing sufficient data to verify the operations status + :raise: any ``Exception`` if the delete operation failed + +.. py:function:: handle_delete_snapshot(self, initial_state) + + Called instead of ``handle_delete`` when the deletion policy is SNAPSHOT. + + :param initial_state: the (action, status) tuple of the resource as of + the start of this method + :return: a token containing sufficient data to verify the operations status + :raise: any ``Exception`` if the delete operation failed + +.. py:function:: check_delete_complete(self, token) + + Verify the delete operation completed successfully. + + :param token: the return value of ``handle_delete`` used to verify the + status of the operation + :return: ``True`` if the delete operation completed and the physical resource + is deleted; ``False`` otherwise. + :raise: any ``Exception`` if the delete operation failed. + +Registering Resource Plug-ins ++++++++++++++++++++++++++++++ +To make your plug-in available for use in stack templates, the plug-in must +register a reference name with the engine. This is done by defining a +``resource_mapping`` function in your plug-in module that returns a map of +template resource type names and their corresponding implementation classes:: + + def resource_mapping(): + return { 'My::Custom::Plugin': MyResourceClass } + +This would allow a template author to define a resource as: + +.. code-block:: yaml + + resources: + my_resource: + type: My::Custom::Plugin + properties: + # ... your plug-in's properties ... + +Note that you can define multiple plug-ins per module by simply returning +a map containing a unique template type name for each. You may also use this to +register a single resource plug-in under multiple template type names (which +you would only want to do when constrained by backwards compatibility). + +Configuring the Engine +---------------------- +In order to use your plug-in, Heat must be configured to read your resources +from a particular directory. The ``plugin_dirs`` configuration option lists the +directories on the local file system where the engine will search for plug-ins. +Simply place the file containing your resource in one of these directories and +the engine will make them available next time the service starts. + +See one of the Installation Guides at http://docs.OpenStack.org/ for +more information on configuring the orchestration service. + +Putting It All Together +----------------------- +You can find the plugin classes in ``heat/engine/resources``. An +exceptionally simple one to start with is ``random_string.py``; it is +unusual in that it does not manipulate anything in the cloud!