deb-murano/doc/source/articles/murano_pl.rst

YAML

YAML is human-readable data serialization format that is a superset of JSON. Unlike JSON YAML was designed to be read and written by humans and relies on visual indentation to denote nesting of data structures. This is similar to how Python uses indentation for block structures instead of curly brackets in most C-like languages. Also YAML can contain more data types comparing to JSON. See http://yaml.org/ for detailed description of YAML.

MuranoPL was designed to be representable in YAML so that MuranoPL code could remain readable and structured. Thus usually MuranoPL files are YAML encoded documents. But MuranoPL engine itself doesn't deal directly with YAML documents and it is up to hosting application to locate and deserialize definitions of particular classes. This gives hosting application ability to control where those definitions can be found (file system, database, remote repository etc) and possibly use some other serialization formats instead of YAML.

MuranoPL engine relies on host deserialization code to automatically detect YAQL expressions in source definition and to provide them as instances of YaqlExpression class rather than plain strings. Usually YAQL expressions can be distinguished by presence of $ (dollar sign) and operators but in YAML developer can always explicitly state the type by using YAML tags. So

Some text - a string,
$.something() - YAQL
"$.something()" - string (because of quote marks)
!!str $ - a string (because of YAML tag)
!yaql "text" - YAQL  (because of YAML tag)

YAQL

YAQL (Yet Another Query Language) is a query language that was also designed as part of Murano project. MuranoPL makes an extensive use of YAQL. YAQL description can be found here: https://github.com/ativelkov/yaql

In simple words YAQL is a language for expression evaluation. 2 + 2, foo() > bar(), true != false are all valid YAQL expressions. The interesting thing in YAQL is that it has no built in list of functions. Everything YAQL can access is customizable. YAQL cannot call any function that was not explicitly registered to be accessible by YAQL. The same is true for operators. So the result of expression 2 * foo(3, 4) is completely depended on explicitly provided implementations of "foo" and "operator*". YAQL uses dollar sign ($) to access external variables (that are also explicitly provided by host application) and function arguments. $variable is a syntax to get the value of variable "$variable", $1, $2 etc are the names for function arguments. "$" is a name for current object - data on which the expression is evaluated or a name of a single argument. Thus $ in the beginning of expression and $ in middle of it can refer to different things.

YAQL has a lot of functions out of the box that can be registered in YAQL context. For example

$.where($.myObj.myScalar > 5 and $.myObj.myArray.len() > 0 and $.myObj.myArray.any($ = 4)).select($.myObj.myArray[0]) can be executed on $ = array of objects and has a result of another array that is a filtration and projection of a source data. This is very similar to how SQL works but uses more Python-like syntax.

Note that there is no assignment operator in YAQL and '=' means comparision operator that is what '==' means in Python.

Because YAQL has no access to underlying operating system resources and 100% controllable by the host it is secure to execute YAQL expressions without establishing a trust to executed code. Also because of the functions are not predefined different functions may be accessible in different contexts. So the YAQL expressions that are used to specify property contracts are not necessarily valid in workflow definitions.

Common class structure

Here is a common template for class declarations. In sections below I'm going to explain what each section means. Note that it is in YAML format.

Name: class name
Namespaces: namespaces specification
Extends: [list of parent classes]
Properties: properties declaration
Workflow:
    methodName:
        Arguments:
            - list
            - of
            - arguments
        Body:
            - list
            - of
            - instructions

Thus MuranoPL class is a YAML dictionary with predefined key names. All keys except for Name are optional and can be omitted (but must be valid if present)

Class name

Class names are alphanumeric names of the classes. By tradition all class names begin with upper-case letter and written in PascalCasing.

In Murano all class names are globally unique. This achieved by means of namespaces. Class name may have explicit namespace specification (like ns:MyName) or implicit (just MyName which would be equal to =:MyName if = was a valid in name specification)

Namespaces

Namespaces declaration specifies prefixes that can be used in class body to make long class names shorter.

Namespaces:
    =: io.murano.services.windows
    srv: io.murano.services
    std: io.murano

In example above class name srv:Something would be automatically translated to "io.murano.services.Something".

"=" means "current namespace" so that "MyClass" would mean "io.murano.services.windows.MyClass" in example above.

If class name contains period sign (.) in its name then it is assumed to be already fully namespace-qualified and is not expanded. Thus ns.Myclass would remain as is.

To make class names globally unique it is recommended to have developer's domain name as part of namespace (as in example, similar to Java)

Extends

MuranoPL supports multiple inheritance. If present, Extends section lists base classes that are extended. If the list consists of single entry then it may be written as a scalar string instead of array. If no parents specified (or a key is omitted) then "io.murano.Object" is assumed making it the root class for all class hierarchies.

Properties

Properties are class attributes that together with methods form public class interface. Usually (but not always) properties are the values and references to other objects that are required to be entered in environment designer prior to workflow invocation.

Properties have the following declaration format:

propertyName:
    Contract: property contract
    Usage: property usage
    Default: property default

Contract

Contracts are YAQL expressions that say what type of value is expected for the property as well as additional constraints imposed on the property.

Operation	Definition
$.int()	integer value (may be null). String values that consist of digits would be converted to integer
$.int().notNull()	mandatory integer
$.string() $.string().notNull()	the same for strings. If the supplied value is not a string it will be converted to string
$.bool() $.bool().notNull()	bools are true and false. 0 is converted to false, other integers to true
$.class(ns:ClassName) $.class(ns:ClassName).notNull()	value must be a reference to an instance of specified class name
$.class(ns:ClassName, ns:DefaultClassName)	create instance of ns:DefaultClassName class if no instance provided
$.class(ns:Name).check($.p = 12)	value must be of type ns:Name and have a property 'p' equal to 12
[$.int()] [$.int().notNull()]	array of integers. Similar for other types
[$.int().check($ > 0)]	array of positive integers (thus not null)
[$.int(), $.string()]	array that has at least two elements, first is int and others are strings
[$.int(), 2] [$.int(), 2, 5]	array of ints with at least 2 items ... and maximum of 5 items
{ A: $.int(), B: [$.string()] }	dictionary with 'A' key of type int and 'B' - array of strings
$ [] {}	any scalar or data structure as is any array any dictionary
{ $.string().notNull(): $.int().notNull() }	dictionary string -> int
A: StringMap $.string().notNull(): $	dictionary with 'A' key that must be equal to 'StringMap' and other keys be any scalar or data structure

Usage

Usage states purpose of the property. This implies who and how can access it. The following usages are available:

Property	Explanation
In	Input property. Values of such properties are obtained from user and cannot be modified in MuranoPL workflows. This is default value for Usage key
Out	The value is obtained from executing MuranoPL workflow and cannot be modified by the user
InOut	Value can be edited by both user and workflow
Const	The same as In but once workflow is executed the property cannot be changed neither by user not the workflow
Runtime	Property is visible only from within workflows. It neither read from input neither serialized to workflow output

Usage attribute is optional and can be omitted (which implies In).

If the workflow tries to write to a property that is not declared with one of the types above it is considered to be private and accessible only to that class (and not serialized to output and thus would be lost upon next deployment). Attempt to read property that wasn't initialized causes exception to be thrown.

Default

Default is a value that would be used if the property value wasn't mentioned in input object model (but not when it is provided as null). Default (if specified) must conform to declared property contract. If Default is not specified then null is the default.

For properties that are references to other classes Default can modify default values for referenced value. For example

p:
  Contract: $.class(MyClass)
  Default: {a: 12}

would override default for 'a' property of MyClass for instance of MyClass that is created for this property.

Workflow

Workflows are the methods that together describe how the entities that are represented by MuranoPL classes are deployed.

In typical scenario root object in input data model is of type io.murano.Environment and has a "deploy" method. Invoking this method causes a series of infrastructure activities (typically by modifying Heat stack) and VM agents commands that cause execution of deployment scripts. Workflow role is to map data from input object model (or result of previously executed actions) to parameters of those activities and to initiate those activities in correct order. Methods have input parameters and can return value to the caller. Methods defined in Workflow section of the class using the following template:

methodName:
    Arguments:
        - list
        - of
        - arguments
    Body:
        - list
        - of
        - instructions

Arguments are optional and (if specified) declared using the same syntax as class properties except for Usage attribute that is meaningless for method parameters. E.g. arguments also have a contract and optional default.

Method body is an array of instructions that got executed sequentially. There are 3 types of instructions that can be found in workflow body: expressions, assignment and block constructs.

Expressions

Expressions are YAQL expressions that are executed for their side effect. All accessible object methods can be called in expression using $obj.methodName(arguments) syntax.

Expression	Explanation
$.methodName() $this.methodName()	invoke method 'methodName' on this (self) object
$.property.methodName() $this.property.methodName()	invocation of method on object that is in 'property' property
$.method(1, 2, 3)	methods can have arguments
$.method(1, 2, thirdParameter => 3)	named parameters also supported
list($.foo().bar($this.property), $p)	complex expressions can be constructed

Assignment

Assignments are single-key dictionaries with YAQL expression as key and arbitrary structure as a value. Such construct evaluated as assignment.

Assignment	Explanation
$x: value	assigns ‘value’ to local variable $x
$.x: value $this.x: value	assign value to object’s property
$.x: $.y	copy value of property 'y' to property 'x'
$x: [$a, $b]	sets $x to array of 2 values $a and $b
$x:   SomeKey:     NestedKey: $variable	structures of any level of complexity can be evaluated
$.x[0]: value`	assign value to a first array entry of property x
$.x.append(): value	append value to array in property x
$.x.insert(1): value	insert value into position 1
$.x.key.subKey: value $.x[key][subKey]: value	deep dictionary modification

Block constructs

Block constructs control program flow. Block constructs are dictionaries that have strings as all its keys. The following block constructs are available:

Assignment	Explanation
Return: value	return value from a method
If: predicate() Then:   - code   - block Else:   - code   - block	predicate() is YAQL expressions that must be evaluated to true or false. else part is optional one-line code blocks can be written as a scalars rather than array.
While: predicate() Do: | - code | - block	predicate() must be evaluated to true or false
For: variableName In: collection Do:   - code   - block	collection must be YAQL expression returning iterable collection or evaluatable array as in assignment instructions (like [1, 2, $x]) inside code block loop variable is accessible as $variableName
Repeat: Do:   - code   - block	repeat code block specified number of times
Break:	breaks from loop
Match:  case1:     - code     - block  case2:    - code    - block Value: $valueExpression() Default:    - code    - block	matches result of $valueExpression() against set of possible values (cases). the code block of first matched cased is executed. if not case matched and Default key is present (it is optional) than Default code block get executed. case values are constant values (not expressions)
Switch:   $predicate1() :       - code       - block   $predicate2() :       - code       - block Default:     - code     - block	all code blocks that have their predicate evaluated to true are executed but the order of predicate evaluation is not fixed default key is optional. if no predicate evaluated to true than Default code block get executed.
Parallel:     - code     - block Limit: 5	executes all instructions in code block in separate green threads in parallel limit is optional and means the maximum number of concurrent green threads.

Object model

Object model is JSON-serialized representation of objects and their properties. Everything user does in environment builder (dashboard) is reflected in object model. Object model is sent to App Catalog engine upon user decides to deploy built environment. On engine side MuranoPL objects are constructed and initialized from received Object model and predefined method is executed on a root object.

Objects serialized to JSON using the following template:

{
    "?": {
        "id": "globally unique object ID (UUID)",
        "type": "fully namespace-qualified class name",

        "optional designer-related entries can be placed here": {
            "key": "value"
        }
    },

    "classProperty1": "propertyValue",
    "classProperty2": 123,
    "classProperty3": ["value1", "value2"],

    "reference1": {
        "?": {
            "id": "object id",
            "type": "object type"
        },

        "property": "value"
    },

    "reference2": "referenced object id"
}

Objects can be identified as dictionaries that contain "?" entry. All system fields are hidden in that entry.

There are 2 ways to specify references. The first method ("reference1" in example above) allow inline definition of object. When instance of referenced object is created outer object becomes its parent (owner) that is responsible for the object. The object itself may require that its parent (direct or indirect) be of specified type (like all application require to have Environment somewhere in parent chain).

Second way to reference object is by specifying other object id. That object must be defined somewhere else in object tree. Object references distinguished from strings having the same value by evaluating property contracts. The former case would have $.class(Name) while the later $.string() contract.