openstack/deb-mistral
Code Issues Proposed changes

Adjust docs API to last changes

Browse Source 
* Added Action Executions
 * Added Environments
 * Deleted DSLv1 doc
 * Minor changes

Change-Id: I52cd6acc9f786cf2dac03cb7ab9465bc4f34f365
This commit is contained in:
Nikolay Mahotkin 2015-05-07 12:59:38 +03:00
parent ca8beb200f
commit caa2045ed1
6 changed files with 106 additions and 446 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/source/developer/webapi/index.rst
View File

@ -1,5 +1,5 @@
REST API
======================
========
.. toctree::
:maxdepth: 2

74
doc/source/developer/webapi/v2.rst
View File

@ -11,7 +11,7 @@ Basics
Media Types
^^^^^^^^^^^^
^^^^^^^^^^^
Currently this API relies on JSON to represent states of REST resources.
@ -21,15 +21,15 @@ Error States
The common HTTP Response Status Codes (https://github.com/for-GET/know-your-http-well/blob/master/status-codes.md) are used.
Application Root [/]
^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^
Application Root provides links to all possible API methods for Mistral. URLs for other resources described below are relative to Application Root.
API v2 Root [/v2/]
^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^
All API v2 urls are relative to API v2 root.
Workbooks
----------
---------
.. autotype:: mistral.api.controllers.v2.workbook.Workbook
:members:
@ -44,7 +44,7 @@ Workbooks
Workflows
----------
---------
.. autotype:: mistral.api.controllers.v2.workflow.Workflow
:members:
@ -58,7 +58,7 @@ Workflows
:webprefix: /v2/workflows
Actions
--------
-------
.. autotype:: mistral.api.controllers.v2.action.Action
:members:
@ -71,7 +71,7 @@ Actions
Executions
------------
----------
.. autotype:: mistral.api.controllers.v2.execution.Execution
:members:
@ -84,7 +84,7 @@ Executions
Tasks
------------
-----
When a workflow starts Mistral creates an execution. It in turn consists of a set of tasks. So Task is an instance of a task described in a Workflow that belongs to a particular execution.
@ -99,4 +99,60 @@ When a workflow starts Mistral creates an execution. It in turn consists of a se
:webprefix: /v2/tasks
.. rest-controller:: mistral.api.controllers.v2.task:ExecutionTasksController
:webprefix: /v2/executions
:webprefix: /v2/executions
Action Executions
-----------------
When a Task starts Mistral creates a set of Action Executions. So Action Execution is an instance of an action call described in a Workflow Task that belongs to a particular execution.
.. autotype:: mistral.api.controllers.v2.action_execution.ActionExecution
:members:
.. autotype:: mistral.api.controllers.v2.action_execution.ActionExecutions
:members:
.. rest-controller:: mistral.api.controllers.v2.action_execution:ActionExecutionsController
:webprefix: /v2/action_executions
.. rest-controller:: mistral.api.controllers.v2.action_execution:TasksActionExecutionController
:webprefix: /v2/tasks
Environments
------------
Environment contains a set of variables which can be used in specific workflow. Using Environment possible to create and map action default values - just provide '__actions' key in 'variables'. All these variables can be accessed in workflow DSL by <% $.__env %> expression.
Example of using in DSL::
workflow:
tasks:
task1:
action: std.echo output=<% $.__env.my_echo_output %>
Example of creating action defaults::
...ENV...
"variables": {
"__actions": {
"std.echo": {
"output": "my_output"
}
}
},
...ENV...
Note: using CLI, Environment can be created via JSON or YAML file.
.. autotype:: mistral.api.controllers.v2.environment.Environment
:members:
.. autotype:: mistral.api.controllers.v2.environment.Environments
:members:
.. rest-controller:: mistral.api.controllers.v2.environment:EnvironmentController
:webprefix: /v2/environments

14
doc/source/developer/writing_a_plugin_action.rst
View File

@ -1,7 +1,7 @@
How to write an Action Plugin
=============================
1. Write a class based on mistral.actions.base.Actions
1. Write a class inherited from mistral.actions.base.Action
::
from mistral.actions import base
@ -24,7 +24,16 @@ How to write an Action Plugin
mistral.actions =
example.runner = my.mistral_plugins.somefile:RunnerAction
3. Use your plugin
3. Reinstall Mistral if it was installed in system (not in virtualenv).
4. Run Db-sync tool via either::
*tools/sync_db.sh --config-file <path-to-config>*
or::
*mistral-db-manage --config-file <path-to-config> populate*
5. Use your plugin
* Now you can call the action "example.runner"
@ -36,3 +45,4 @@ How to write an Action Plugin
action: example.runner
parameters:
param: avalue_to_pass_in

404
doc/source/dsl/dsl_v1.rst
View File

@ -1,404 +0,0 @@
Mistral DSL specification
-------------------------
Version 0.1
Main objects
~~~~~~~~~~~~
- **Namespace**
- **Action**
- **Workflow**
- **Task**
- **Trigger**
Namespaces
~~~~~~~~~~
Contains a list of namespaces grouping custom (ad-hoc) actions. For
example, it's possible to create namespace "Nova" that would provide
actions "createVM", "deleteVM" and similar for VM management in
OpenStack.
Attributes
^^^^^^^^^^
All attributes are inside main keys of **Namespaces** - **namespaces
names**. Attributes of an individual namespace are:
- **class** - currently Mistral supports the following action classes
out of the box: std.http, std.mistral\_http, std.echo, std.email,
std.ssh, this is an optional attribute
- **base-parameters** - dictionary depending on type of the namespace,
this parameter is optional. Values of these parameters apply to all
actions inside the namespace. For example for std.http it can contain
url, method, body and headers.
- **actions** - list of actions provided by this namespace.
YAML example:
^^^^^^^^^^^^^
| ``Namespaces:``
| ``  Nova:``
| ``    class: std.http``
| ``    base-parameters:``
| ``      method: GET``
| ``    actions:``
| ``      create-vm:``
| ``      ......``
| ``      delete-vm:  ``
| ``      .....``
Action
~~~~~~
A function provided by specific namespace.
Attributes
^^^^^^^^^^
- **name** - action name (string without space, mandatory attribute).
- **base-parameters** - dictionary whose structure is defined by action
class (or namespace class if defined). For std.http class it contains
url, method, body and headers according to HTTP protocol
specification.
- **parameters** - list containing parameter names which should or
could be specified in task. This attribute is optional and used only
for documenting purposes.
- **output** - dictionary-transformer for action output to send this
output next to task. Keys of this dictionary will be included in task
result, values are pure YAQL or inline YAQL expressions per key which
define how the specific output should be retrieved from raw output
(from action). See more about YAQL at
https://pypi.python.org/pypi/yaql/0.3
YAML example:
^^^^^^^^^^^^^
| ``create-vm:``
| ``  base-parameters:``
| ``    url: servers``
| ``  parameters:``
| ``    - name``
| ``    - server_name  ``
| ``  output:``
| ``    vm_id: $.content.server.id``
Workflow
~~~~~~~~
Attributes
^^^^^^^^^^
- **tasks** - list of tasks in this workflow, each task represents a
computational step in the workflow.
YAML example:
^^^^^^^^^^^^^
| `` Workflow:``
| ``   tasks:``
| ``     create-vms:``
| ``     .....``
| ``     attache-volumes: ``
| ``     .....``
Task
~~~~
Represents a step in workflow, for example creating the VM
Attributes
^^^^^^^^^^
- **action** - name of action to perform
- **requires** - list of tasks which should be execute before this
tasks, or list of task names as a keys and condition as a value, this
is optional parameter
- **parameters** - actual parameters for the task, each value can be
either some number, string etc, or YAQL expression to retrieve value
from task context
- **on-success** - task which will be scheduled on execution after
current task has finished with state 'SUCCESS'
- **on-error** - task which will be scheduled on execution after
current task has finished with state 'ERROR'
- **on-finish** - task which will be scheduled on execution after
current task has finished
YAML example:
'''''''''''''
| ``create-vm:``
| ``  action: Nova.create-vm``
| ``  parameters:``
| ``    image_id: $.image_id``
| ``    flavor_id: 42``
| ``  requires:``
| ``    task2: '$.value2 = 123'``
| ``    task4: '$.value4 = 122'``
| ``  on-success: task3``
Triggers
~~~~~~~~
Using triggers it is possible to run workflows according to specific
rules: periodically setting a cron (http://en.wikipedia.org/wiki/Cron)
pattern or on external events like ceilometer alarm.
Attributes
^^^^^^^^^^
- **type** - can be PERIODIC, CEILOMETER\_ALARM
- **tasks** - list of tasks which should be execute on trigger
- **parameters** - list of task parameters
YAML example:
^^^^^^^^^^^^^
| ``triggers:``
| ``  backup-vm:``
| ``    type: periodic``
| ``    tasks: [create_backup, delete_old_backup] ``
| ``    parameters:``
| ``      cron-pattern: 1 0 * * *``
Full YAML example:
~~~~~~~~~~~~~~~~~~
This example requires the following properties provided in execution context:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#. - nova\_url ## url to Nova service, e.g. http://0.0.0.0:8774/v3
#. - server\_name ## Name you want to give to new instance
#. - image\_id ## image id from Glance service
#. - flavor\_id ## flavor id - type of instance hardware
#. - ssh\_username ## username of your VM
#. - ssh\_password ## password to your VM
#. - admin\_email ## email address to send notifications to
#. - from\_email ## email address to send notifications from
#. - smtp\_server ## SMTP server to use for sending emails (e.g.
smtp.gmail.com:587)
#. - smtp\_password ## password to connect to SMTP server
| ``Namespaces:``
| `` Nova:``
| ``   # Nova actions for creating VM, retrieving IP and VM deleting.``
| ``   class: std.http``
| ``   actions:``
| ``     createVM:``
| ``       base-parameters:``
| ``         url: '{$.nova_url}/{$.project_id}/servers'``
| ``         method: POST``
| ``         headers:``
| ``           X-Auth-Token: $.auth_token``
| ``           Content-Type: application/json``
| ``         body:``
| ``           server:``
| ``             name: $.server_name``
| ``             imageRef: $.image_id``
| ``             flavorRef: $.flavor_id``
| ``       output:``
| ``         vm_id: $.content.server.id``
| `` ``
| ``     getIP:``
| ``       base-parameters:``
| ``         url: '{$.nova_url}/{$.project_id}/servers/{$.vm_id}'``
| ``         method: GET``
| ``         headers:``
| ``           X-Auth-Token: $.auth_token``
| ``       output:``
| ``         vm_ip: "$.content.server.addresses.novanetwork.where($.'OS-EXT-IPS:type' = 'floating')[0].addr"``
| `` ``
| ``     deleteVM:``
| ``       base-parameters:``
| ``         url: '{$.nova_url}/{$.project_id}/servers/{$.vm_id}'``
| ``         method: DELETE``
| ``         headers:``
| ``           X-Auth-Token: $.auth_token``
| ``       output:``
| ``         status: $.status``
| `` ``
| `` Server:``
| ``   actions:``
| ``     # HTTP request to the server.``
| ``     calcSumm:``
| ``       class: std.http``
| ``       base-parameters:``
| ``         url: '``\ ```http://`` <http://>`__\ ``{$.vm_ip}:5000/summ'``
| ``         method: POST``
| ``         body:``
| ``           arguments: $.arguments``
| ``       output:``
| ``         summ_result: $.content.result``
| `` ``
| `` Ssh:``
| ``   class: std.ssh``
| ``   base-parameters:``
| ``     host: $.vm_ip``
| ``     username: $.username``
| ``     password: $.password``
| ``   actions:``
| ``     # Simple SSH command.``
| ``     waitSSH:``
| ``       base-parameters:``
| ``         cmd: 'ls -l'``
| `` ``
| ``     # SSH command to run the server.``
| ``     runServer:``
| ``       base-parameters:``
| ``         cmd: 'nohup python ~/web_app.py > web_app.log &'``
| `` ``
| ``Workflow:``
| `` tasks:``
| ``   # Create a VM (request to Nova).``
| ``   createVM:``
| ``     action: Nova.createVM``
| ``     parameters:``
| ``       server_name: $.server_name``
| ``       image_id: $.image_id``
| ``       flavor_id: $.flavor_id``
| ``       nova_url: $.nova_url``
| ``       project_id: $.project_id``
| ``       auth_token: $.auth_token``
| ``     publish:``
| ``       vm_id: vm_id``
| ``     on-success: waitForIP``
| ``     on-error: sendCreateVMError``
| `` ``
| ``   # Wait till the VM is assigned with IP address (request to Nova).``
| ``   waitForIP:``
| ``     action: Nova.getIP``
| ``     retry:``
| ``       count: 10``
| ``       delay: 10``
| ``     publish:``
| ``       vm_ip: vm_ip``
| ``     parameters:``
| ``       nova_url: $.nova_url``
| ``       project_id: $.project_id``
| ``       auth_token: $.auth_token``
| ``       vm_id: $.vm_id``
| ``     on-success: waitSSH``
| ``     on-error: sendCreateVMError``
| `` ``
| ``   # Wait till operating system on the VM is up (SSH command).``
| ``   waitSSH:``
| ``     action: Ssh.waitSSH``
| ``     retry:``
| ``       count: 10``
| ``       delay: 10``
| ``     parameters:``
| ``       username: $.ssh_username``
| ``       password: $.ssh_password``
| ``       vm_ip: $.vm_ip``
| ``     on-success: runServer``
| ``     on-error: sendCreateVMError``
| `` ``
| ``   # When SSH is up, we are able to run the server on VM (SSH command).``
| ``   runServer:``
| ``     action: Ssh.runServer``
| ``     parameters:``
| ``       vm_ip: $.vm_ip``
| ``       username: $.ssh_username``
| ``       password: $.ssh_password``
| ``     on-success: calcSumm``
| ``     on-error: sendCreateVMError``
| `` ``
| ``   # Send HTTP request on server and calc the result.``
| ``   calcSumm:``
| ``     action: Server.calcSumm``
| ``     retry:``
| ``       count: 10``
| ``       delay: 1``
| ``     parameters:``
| ``       arguments:``
| ``         - 32``
| ``         - 45``
| ``         - 23``
| ``       vm_ip: $.vm_ip``
| ``     publish:``
| ``       result: summ_result``
| ``     on-finish: sendResultEmail``
| `` ``
| ``   # In case of createVM error send e-mail with error message.``
| ``   sendResultEmail:``
| ``     action: std.email``
| ``     parameters:``
| ``       params:``
| ``         to: [$.admin_email]``
| ``         subject: Workflow result``
| ``         body: |``
| ``           Workflow result of execution {$.__execution.id} is {$.result}``
| `` ``
| ``           -- Thanks, Mistral Team.``
| ``       settings:``
| ``         smtp_server: $.smtp_server``
| ``         from: $.from_email``
| ``         password: $.smtp_password``
| ``     on-finish: deleteVM``
| `` ``
| ``   # In case of createVM error send e-mail with error message.``
| ``   sendCreateVMError:``
| ``     action: std.email``
| ``     parameters:``
| ``       params:``
| ``         to: [$.admin_email]``
| ``         subject: Workflow error``
| ``         body: |``
| ``           Failed to create a VM in execution {$.__execution.id}``
| `` ``
| ``           -- Thanks, Mistral Team.``
| ``       settings:``
| ``         smtp_server: $.smtp_server``
| ``         from: $.from_email``
| ``         password: $.smtp_password``
| ``     on-finish: deleteVM``
| `` ``
| ``   # Destroy the VM (request to Nova).``
| ``   deleteVM:``
| ``     action: Nova.deleteVM``
| ``     parameters:``
| ``       nova_url: $.nova_url``
| ``       project_id: $.project_id``
| ``       auth_token: $.auth_token``
| ``       vm_id: $.vm_id``
Initial execution context
^^^^^^^^^^^^^^^^^^^^^^^^^
| `` {``
| ``   "nova_url": ``\ \ ``,``
| ``   "image_id": ``\ \ ``,``
| ``   "flavor_id": ``\ \ ``,``
| ``   "server_name": ``\ \ ``,``
| ``   "ssh_username": ``\ \ ``,``
| ``   "ssh_password": ``\ \ ``,``
| ``   "admin_email": ``\ \ ``,``
| ``   "from_email": ``\ \ ``,``
| ``   "smtp_server": ``\ \ ``,``
| ``   "smtp_password": ``\ \ ``,``
| `` }``
**When a workflow starts Mistral also adds execution information into
the context so the context looks like the following:**
| `` {``
| ``   "nova_url": TBD,``
| ``   "image_id": TBD,``
| ``   "image_id": ``\ \ ``,``
| ``   "flavor_id": ``\ \ ``,``
| ``   "server_name": ``\ \ ``,``
| ``   "ssh_username": ``\ \ ``,``
| ``   "ssh_password": ``\ \ ``,``
| ``   "admin_email": ``\ \ ``,``
| ``   "from_email": ``\ \ ``,``
| ``   "smtp_server": ``\ \ ``,``
| ``   "smtp_password": ``\ \ ``,``
| ``   "__execution": {``
| ``     "id": "234234",``
| ``     "workbook_name" : "my_workbook",``
| ``     "project_id": "ghfgsdfasdfasdf"``
| ``     "auth_token": "sdfljsdfsdf-234234234",``
| ``     "trust_id": "oiretoilkjsdfglkjsdfglkjsdfg"``
| ``   }``
| `` }``

57
doc/source/dsl/dsl_v2.rst
View File

@ -62,16 +62,16 @@ YAML example
| ``    - image_ref``
| ``    - flavor_ref``
| ``  output: # Output definition``
| ``    vm_id: $.vm_id``
| ``    vm_id: <% $.vm_id %>``
| ``  tasks:``
| ``    create_server:``
| ``      action: nova.servers_create name={$.vm_name} image={$.image_ref} flavor={$.flavor_ref}``
| ``      action: nova.servers_create name=<% $.vm_name %> image=<% $.image_ref %> flavor=<% $.flavor_ref %>``
| ``      publish:``
| ``        vm_id: $.id``
| ``        vm_id: <% $.id %>``
| ``      on-success:``
| ``        - wait_for_instance``
| ``    wait_for_instance:``
| ``      action: nova.servers_find id={$.vm_id} status='ACTIVE'``
| ``      action: nova.servers_find id=<% $.vm_id %> status='ACTIVE'``
| ``      policies:``
| ``        retry:``
| ``          delay: 5``
@ -126,7 +126,7 @@ different types:
action_based_task:
 action: std.http url='openstack.org'
workflow_based_task:
  workflow: backup_vm_workflow vm_id={$.vm_id}
  workflow: backup_vm_workflow vm_id=<% $.vm_id %>
Actions will be explained below in a individual paragraph but looking
@ -151,7 +151,7 @@ attributes:
string etc, dictionary or list. It can also be a YAQL expression to
retrieve value from task context or any of the mentioned types
containing inline YAQL expressions (for example, string
"{$.movie\_name} is a cool movie!")
"<% $.movie\_name %> is a cool movie!")
- **publish** - Dictionary of variables to publish to the workflow
context. Any JSON-compatible data structure optionally containing
YAQL expression to select precisely what needs to be published.
@ -172,15 +172,14 @@ YAML example
| ``my_task:``
| ``  ...``
| ``  policies:``
| ``    wait-before: 2``
| ``    wait-after: 4``
| `` pause-before: $.my_expr``
| ``    timeout: 30``
| ``    retry:``
| ``      count: 10``
| ``      delay: 20``
| ``      break-on: $.my_var = True``
| ``  wait-before: 2``
| ``  wait-after: 4``
| `` pause-before: <% $.my_expr %>``
| ``  timeout: 30``
| ``  retry:``
| ``    count: 10``
| ``    delay: 20``
| ``    break-on: <% $.my_var = True %>``
'wait-before'
@ -287,12 +286,12 @@ YAML example
| ``    - image_id``
| ``    - flavor_id``
| ``  output:``
| ``    result: $.vm_id``
| ``    result: <% $.vm_id %>``
| ``  tasks:``
| ``    create_vm:``
| ``      action: nova.servers_create name={$.vm_name} image={$.image_id} flavor={$.flavor_id}``
| ``      action: nova.servers_create name=<% $.vm_name %> image=<% $.image_id %> flavor=<% $.flavor_id %>``
| ``      publish:``
| ``        vm_id: $.id``
| ``        vm_id: <% $.id %>``
| ``      on-error:``
| ``        - send_error_email``
| ``      on-success:``
@ -302,7 +301,7 @@ YAML example
| ``      on_complete:``
| ``        - fail``
| ``    send_success_email:``
| ``      action: send_email to='admin@mysite.org' body='Vm is successfully created and its id: {$.vm_id}'``
| ``      action: send_email to='admin@mysite.org' body='Vm is successfully created and its id: <% $.vm_id %>'``
Transitions with YAQL expressions
'''''''''''''''''''''''''''''''''
@ -316,7 +315,7 @@ access any data produced by upstream tasks. So in the example above task
| ``create_vm:``
| ``  ...``
| ``  on-success:``
| ``    - send_success_email: $.vm_id != null``
| ``    - send_success_email: <% $.vm_id != null %>``
And this would tell Mistral to run 'send\_success\_email' task only if
'vm\_id' variable published by task 'create\_vm' is not empty. YAQL
@ -364,21 +363,21 @@ YAML example
| ``    - image_id``
| ``    - flavor_id``
| ``  output:``
| ``    result: $.vm_id``
| ``    result: <% $.vm_id %>``
| ``  tasks:``
| ``    create_vm:``
| ``      action: nova.servers_create name={$.vm_name} image={$.image_id} flavor={$.flavor_id}``
| ``      action: nova.servers_create name=<% $.vm_name %> image=<% $.image_id %> flavor=<% $.flavor_id %>``
| ``      publish:``
| ``        vm_id: $.id``
| ``        vm_id: <% $.id %>``
| ``    search_for_ip:``
| ``      action: nova.floating_ips_findall instance_id=null``
| ``      publish:``
| ``        vm_ip: $[0].ip``
| ``        vm_ip: <% $[0].ip %>``
| ``    associate_ip:``
| ``      action: nova.servers_add_floating_ip server={$.vm_id} address={$.vm_ip}``
| ``      action: nova.servers_add_floating_ip server=<% $.vm_id %> address=<% $.vm_ip %>``
| ``      requires: [search_for_ip]``
| ``    send_email:``
| ``      action: send_email to='admin@mysite.org' body='Vm is created and id {$.vm_id} and ip address {$.vm_ip}'``
| ``      action: send_email to='admin@mysite.org' body='Vm is created and id <% $.vm_id %> and ip address <% $.vm_ip %>'``
| ``      requires: [create_vm, associate_ip]``
Reverse Workflow Task Attributes
@ -547,7 +546,7 @@ YAML example
| ``      subject: Something went wrong with your Mistral workflow :(``
| ``      body: |``
| ``          Please take a look at Mistral Dashboard to find out what's wrong``
| ``          with your workflow execution {$.execution_id}.``
| ``          with your workflow execution <% $.execution_id %>.``
| ``          Everything's going to be alright!``
| ``          -- Sincerely, Mistral Team.``
| ``      settings:``
@ -562,7 +561,7 @@ it as follows:
| ``  tasks:``
| ``    ...``
| ``    send_error_email``
| ``      action: error_email execution_id={$.__execution.id}``
| ``      action: error_email execution_id=<% $.__execution.id %>``
Attributes
''''''''''
@ -695,7 +694,7 @@ YAML example
| ``    input:``
| ``      - str1``
| ``      - str2``
| ``    base: std.echo output="{$.str1}{$.str2}"``
| ``    base: std.echo output="<% $.str1 %><% $.str2 %>"``
**Note**: Even though names of objects inside workbooks change upon
uploading Mistral allows referencing between those objects using local

1
doc/source/dsl/index.rst
View File

@ -5,4 +5,3 @@ DSL Specification
:maxdepth: 1
DSL v2 <dsl_v2>
DSL v1 (deprecated) <dsl_v1>