Fix docs to better reflect Jinja and YAQL usage

Closes-Bug: #1641174
Change-Id: I6e2a10932c648836a1e0632b4cd0fce3ab78f946

doc/source/index.rst

@@ -41,7 +41,7 @@ User guide
 .. toctree::
    :maxdepth: 2
 
-   user/dsl_v2
+   user/wf_lang_v2
 
 **CLI**
 

doc/source/quickstart.rst

@@ -64,7 +64,7 @@ This simple workflow iterates through a list of names in ``task1`` (using
 then stores the word "Done" as a result of the second task (`task2`).
 
 To learn more about the Mistral Workflows and what you can do, read the
-:doc:`Mistral Workflow Language specification <user/dsl_v2>`
+:doc:`Mistral Workflow Language specification <user/wf_lang_v2>`
 
 Upload the workflow
 -------------------

doc/source/terminology/workbooks.rst

@@ -80,4 +80,4 @@ the original workbook.
 * **actions** - Dictionary containing ad-hoc action definitions. *Optional*.
 
 For more details about Mistral Workflow Language itself, please see
-:doc:`Mistral Workflow Language specification </user/dsl_v2>`
+:doc:`Mistral Workflow Language specification </user/wf_lang_v2>`

doc/source/terminology/workflows.rst

@@ -137,4 +137,4 @@ YAML example
           requires: [create_vm, associate_ip]
 
 For more details about Mistral Workflow Language itself, please see
-:doc:`Mistral Workflow Language specification </user/dsl_v2>`
+:doc:`Mistral Workflow Language specification </user/wf_lang_v2>`

doc/source/user/wf_lang_v2.rst

@@ -30,12 +30,18 @@ Prerequisites
 -------------
 
 Mistral Workflow Language supports
-`YAQL <https://pypi.python.org/pypi/yaql/1.0.0>`__ and
+`YAQL <https://pypi.python.org/pypi/yaql>`__ and
 `Jinja2 <http://jinja.pocoo.org/docs/dev/>`__ expression languages to reference
 workflow context variables and thereby implements passing data between workflow
 tasks. It's also referred to as Data Flow mechanism. YAQL is a simple but
 powerful query language that allows to extract needed information from JSON
-structured data. It is allowed to use YAQL in the following sections of
+structured data. Although Jinja2 is primarily a templating technology, Mistral
+also uses it for evaluating expression so user have a choice between YAQL and
+Jinja2. It's also possible to combine both expression languages within one
+workflow definition. The only limitation is that it's impossible to use both
+types of expression within one line. As long as there are YAQL and Jinja2
+expressions on different lines of the workflow definition text, it is valid.
+It is allowed to use YAQL/Jinja2 in the following sections of
 Mistral Workflow Language:
 
 -  Workflow `'output' attribute <#common-workflow-attributes>`__
@@ -126,9 +132,9 @@ Common workflow attributes
 -  **description** - Arbitrary text containing workflow description. *Optional*.
 -  **input** - List defining required input parameter names and
    optionally their default values in a form "my_param: 123". *Optional*.
--  **output** - Any data structure arbitrarily containing
+-  **output** - Any data structure arbitrarily containing YAQL/Jinja2
    expressions that defines workflow output. May be nested. *Optional*.
--  **output-on-error** - Any data structure arbitrarily containing YAQL
+-  **output-on-error** - Any data structure arbitrarily containing YAQL/Jinja2
    expressions that defines output of a workflow to be returned if it goes into
    error state. May be nested. *Optional*.
 -  **task-defaults** - Default settings for some of task attributes
@@ -183,21 +189,21 @@ it's also easy to plug new actions into Mistral.
 Common task attributes
 ''''''''''''''''''''''
 
-All Mistral tasks regardless of workflow type have the following common
+All Mistral tasks, regardless of workflow type, have the following common
 attributes:
 
 -  **description** - Arbitrary text containing task description.
    *Optional*.
--  **action** - Name of the action associated with the task.Can be static
+-  **action** - Name of the action associated with the task. Can be a static
    value or an expression (for example,  "{{ _.action_name }}").
    *Mutually exclusive with* **workflow**. If neither action nor workflow are
-   provided then the action 'std.noop' will be used.
+   provided then the action 'std.noop' will be used that does nothing.
--  **workflow** - Name of the workflow associated with the task. Can be static
+-  **workflow** - Name of the workflow associated with the task. Can be a static
    value or an expression (for example,  "{{ _.subworkflow_name }}").
    *Mutually exclusive with* **action**.
--  **input** - Actual input parameter values of the task. *Optional*.
+-  **input** - Actual input parameter values of the task's action or workflow.
-   Value of each parameter is a JSON-compliant type such as number,
+   *Optional*. Value of each parameter is a JSON-compliant type such as number,
-   string etc, dictionary or list. It can also be an expression to
+   string etc, dictionary or list. It can also be a YAQL/Jinja2 expression to
    retrieve value from task context or any of the mentioned types
    containing inline expressions (for example, string "<%
    $.movie_name %> is a cool movie!") Can be an expression that evaluates to
@@ -229,62 +235,65 @@ attributes:
    during action execution. If set to 'true' task may be run twice.
    *Optional*. By default set to 'false'.
 
-Workflow
+workflow
 ''''''''
-Synchronously starts a sub-workflow with the given name.
+If a task has the attribute 'workflow' it synchronously starts a sub-workflow
+with the given name.
 
-Example static workflow call:
+Example of a static sub-workflow name:
 
 .. code-block:: mistral
 
     my_task:
       workflow: name_of_my_workflow
 
-Example dynamic workflow selection:
+Example of a dynamic sub-workflow name:
 
 .. code-block:: mistral
 
   ---
   version: '2.0'
 
-  name: weather_data_processing
+  framework:
+    input:
+      - magic_workflow_name: show_weather
 
-  workflows:
+    tasks:
-    framework:
+      weather_data:
-      input:
+        action: std.echo
-        - magic_workflow_name: show_weather
+        input:
+          output:
+            location: wherever
+            temperature: "22C"
+        publish:
+          weather_data: <% task().result %>
+        on-success:
+          - do_magic
 
-      tasks:
+      do_magic:
-        weather_data:
+        # Reference workflow by parameter.
-          action: std.echo
+        workflow: <% $.magic_workflow_name %>
-          input:
+        # Expand dictionary to input parameters.
-            output:
+        input: <% $.weather_data %>
-              location: wherever
-              temperature: "22C"
-          publish:
-            weather_data: <% task(weather_data).result %>
-          on-success:
-            - do_magic
 
-        do_magic:
+  show_weather:
-          # reference workflow by parameter
+    input:
-          workflow: <% $.magic_workflow_name %>
+      - location
-          # expand dictionary to input parameters
+      - temperature
-          input: <% $.weather_data %>
 
-    show_weather:
+    tasks:
-      input:
+      write_data:
-        - location
+        action: std.echo
-        - temperature
+        input:
+          output: "<% $.location %>: <% $.temperature %>"
 
-      tasks:
+In this example, we defined two workflows in one YAML snippet and the workflow
-        write_data:
+'framework' may call the workflow 'show_weather' if 'framework' receives the
-          action: std.echo
+corresponding workflow name through the input parameter 'magic_workflow_name'.
-          input:
+In this case it is set by default so a user doesn't need to pass anything
-            output: "<% $.location %>: <% $.temperature %>"
+explicitly.
 
-
+Note: Typical use for the dynamic sub-workflow selection is when parts of a
-Note: Typical use for the dynamic workflow selection is when parts of a
 workflow can be customized. E.g. collect some weather data and then execute
 some custom workflow on it.
 
@@ -333,7 +342,7 @@ has completed before starting next tasks defined in *on-success*,
 **timeout**
 
 Defines a period of time in seconds after which a task will be failed
-automatically by engine if hasn't completed.
+automatically by engine if it hasn't completed.
 
 
 **concurrency**
@@ -368,7 +377,7 @@ Retry policy can also be configured on a single line as:
       action: my_action
       retry: count=10 delay=5 break-on=<% $.foo = 'bar' %>
 
-All parameter values for any policy can be defined as expressions.
+All parameter values for any policy can be defined as YAQL/Jinja2 expressions.
 
 Input syntax
 ''''''''''''
@@ -512,6 +521,12 @@ Direct workflow task attributes
 -  **on-complete** - List of tasks which will run after the task has
    completed regardless of whether it is successful or not. *Optional*.
 
+Note: All of the above clauses cannot contain task names evaluated as
+YAQL/Jinja expressions. They have to be static values. However, task
+transitions can be conditional, based on expressions. See
+`Transitions with expressions <#transitions-with-expressions>`__ for more
+details.
+
 It is important to understand the semantics of **on-success**, **on-error**
 and **on-complete** around handling action errors.
 
@@ -525,6 +540,26 @@ stop the exception from bubbling up to an upper layer. So **on-complete**
 should only be understood as a language construction that allows to
 define some clean up actions.
 
+Transitions with expressions
+''''''''''''''''''''''''''''
+
+Task transitions can be determined by success/error/completeness of the
+previous tasks and also by additional guard expressions that can access any
+data produced by upstream tasks and as workflow input. So in the example above
+task 'create_vm' could also have a YAQL expression on transition to task
+'send_success_email' as follows:
+
+.. code-block:: mistral
+
+    create_vm:
+     ...
+     on-success:
+       - send_success_email: <% $.vm_id != null %>
+
+And this would tell Mistral to run 'send_success_email' task only if 'vm_id'
+variable published by task 'create_vm' is not empty. Expressions can also be
+applied to 'on-error' and 'on-complete'.
+
 Engine Commands
 '''''''''''''''
 
@@ -554,7 +589,7 @@ YAML example
         create_server:
           action: nova.servers_create name=<% $.vm_name %>
           publish:
-            vm_id: <% task(create_server).result.id %>
+            vm_id: <% task().result.id %>
           on-complete:
             - fail: <% not $.vm_id %>
 
@@ -578,26 +613,6 @@ never be called. ``taskB`` would not be called if ``succeed`` was used in this
 example either, but if ``pause`` was used ``taskB`` would be called after the
 workflow is resumed.
 
-Transitions with YAQL expressions
-'''''''''''''''''''''''''''''''''
-
-Task transitions can be determined by success/error/completeness of the
-previous tasks and also by additional guard expressions that can access any
-data produced by upstream tasks. So in the example above task 'create_vm' could
-also have a YAQL expression on transition to task 'send_success_email' as
-follows:
-
-.. code-block:: mistral
-
-    create_vm:
-     ...
-     on-success:
-       - send_success_email: <% $.vm_id != null %>
-
-And this would tell Mistral to run 'send_success_email' task only if 'vm_id'
-variable published by task 'create_vm' is not empty. Expressions can also be
-applied to 'on-error' and 'on-complete'.
-
 Fork
 ''''
 
@@ -792,18 +807,19 @@ provide in "vm_names" input parameter. E.g., if we specify
 vm_names=["vm1", "vm2"] then it'll create servers with these names based on
 same image and flavor. It is possible because of using "with-items" keyword
 that makes an action or a workflow associated with a task run multiple times.
-Value of "with-items" task property contains an expression in the form: in
+Value of "with-items" task property contains an expression in the form: 'my_var' in
-<% YAQL_expression %>.
+<% YAQL_expression %>. Similar for Jinja2 expression: 'my_var' in
+{{ Jinja2_expression }}.
 
 The most common form is:
 
 .. code-block:: mistral
 
     with-items:
-      - var1 in <% YAQL_expression_1 %>
+      - var1 in <% YAQL_expression_1 %> # or: var1 in <% Jinja2_expression_1 %>
-      - var2 in <% YAQL_expression_2 %>
+      - var2 in <% YAQL_expression_2 %> # or: var2 in <% Jinja2_expression_2 %>
       ...
-      - varN in <% YAQL_expression_N %>
+      - varN in <% YAQL_expression_N %> # or: varN in <% Jinja2_expression_N %>
 
 where collections expressed as YAQL_expression_1, YAQL_expression_2,
 YAQL_expression_N must have equal sizes. When a task gets started Mistral will
@@ -1120,7 +1136,7 @@ Attributes
 -  **input** - List of declared action parameters which should be specified as
    corresponding task input. This attribute is optional and used only for
    documenting purposes. Mistral now does not enforce actual input parameters to
-   exactly correspond to this list. Based parameters will be calculated based on
+   exactly correspond to this list. Base parameters will be calculated based on
    provided actual parameters with using expressions so what's used in
    expressions implicitly define real input parameters. Dictionary of actual
    input parameters (expression context) is referenced as '$.' in YAQL and as
@@ -1387,9 +1403,13 @@ returns the following fields of task executions:
 * **spec** - task execution spec dict (loaded from Mistral Workflow Language).
 * **state** - task execution state.
 * **state_info** - task execution state info.
-* **result** - task execution result.
+* **result** - task execution result. In case of a non 'with-items' task it's
+  simply a result of the task's action/sub-workflow execution. For a
+  'with-items' task it will be a list of results of corresponding
+  action/sub-workflow execution.
 * **published** - task execution published variables.
 
+
 Execution info
 ^^^^^^^^^^^^^^
 
@@ -1462,5 +1482,5 @@ Workflow definition:
 Environment
 ^^^^^^^^^^^
 
-Environment info is available by **env()**. It is passed when user submit
+Environment info is available by **env()**. It is passed when user submits
 workflow execution. It contains variables specified by user.