openstack
/
murano-specs
Code Issues Proposed changes

Retire Murano: remove repo content 

Murano project is retiring
- https://review.opendev.org/c/openstack/governance/+/919358

this commit remove the content of this project repo

Depends-On: https://review.opendev.org/c/openstack/project-config/+/919359/
Change-Id: I07209efe58bb1ea60fa43026b15beea112985d2b
This commit is contained in:
Ghanshyam Mann 2024-05-10 15:52:29 -07:00
parent 637f7d6d37
commit 4b967c365b
77 changed files with 8 additions and 16074 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
.gitignore vendored
View File

@ -1,10 +0,0 @@
AUTHORS
ChangeLog
build
.tox
.venv
*.egg*
*.swp
*.swo
*.pyc
.testrepository

7
.testr.conf
View File

@ -1,7 +0,0 @@
[DEFAULT]
test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \
${PYTHON:-python} -m subunit.run discover tests $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list

3
.zuul.yaml
View File

@ -1,3 +0,0 @@
- project:
templates:
- openstack-specs-jobs

14
CONTRIBUTING.rst
View File

@ -1,14 +0,0 @@
The source repository for this project can be found at:
https://opendev.org/openstack/murano-specs
Pull requests submitted through GitHub are not monitored.
To start contributing to OpenStack, follow the steps in the contribution guide
to set up and use Gerrit:
https://docs.openstack.org/contributors/code-and-documentation/quick-start.html
This repository is for doing design review on feature proposal in Murano.
Please refer to `this document <https://opendev.org/openstack/murano-specs/src/branch/master/README.rst>`_
for more information.

3
LICENSE
View File

@ -1,3 +0,0 @@
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
http://creativecommons.org/licenses/by/3.0/legalcode

72
README.rst
View File

@ -1,66 +1,10 @@
========================
Team and repository tags
========================
This project is no longer maintained.
.. image:: http://governance.openstack.org/badges/murano-specs.svg
:target: http://governance.openstack.org/reference/tags/index.html
The contents of this repository are still available in the Git
source code management system. To see the contents of this
repository before it reached its end of life, please check out the
previous commit with "git checkout HEAD^1".
.. Change things from this point on
===============================
OpenStack Murano Specifications
===============================
This git repository is used to hold approved design specifications for additions
to the Murano project. Reviews of the specs are done in gerrit, using a similar
workflow to how we review and merge changes to the code itself.
The layout of this repository is::
specs/<release>/
Where there are two sub-directories:
specs/<release>/approved: specifications approved but not yet implemented
specs/<release>/implemented: implemented specifications
This directory structure allows you to see what we thought about doing,
decided to do, and actually got done. Users interested in functionality in a
given release should only refer to the ``implemented`` directory.
You can find an example spec in `doc/source/specs/template.rst`.
Specifications are proposed for a given release by adding them to the
`specs/<release>` directory and posting it for review. The implementation
status of a blueprint for a given release can be found by looking at the
blueprint in launchpad. Not all approved blueprints will get fully implemented.
Specifications have to be re-proposed for every release. The review may be
quick, but even if something was previously approved, it should be re-reviewed
to make sure it still makes sense as written.
Prior to the Kilo development cycle this repository was not used for
spec reviews. Reviews prior to Juno were completed entirely through
Launchpad blueprints::
http://blueprints.launchpad.net/murano
Starting from the Kilo-1 developement milestone Murano performs the pilot of
the specs repos approach.
Please note, Launchpad blueprints are still used for tracking the
current status of blueprints. For more information, see::
https://wiki.openstack.org/wiki/Blueprints
For more information about working with gerrit, see::
http://docs.openstack.org/infra/manual/developers.html#development-workflow
To validate that the specification is syntactically correct (i.e. get more
confidence in the Jenkins result), please execute the following command::
$ tox
After running ``tox``, the documentation will be available for viewing in HTML
format in the ``doc/build/`` directory.
For any further questions, please email
openstack-discuss@lists.openstack.org or join #openstack-dev on
OFTC.

274
doc/source/conf.py
View File

@ -1,274 +0,0 @@
# -*- coding: utf-8 -*-
#
# Tempest documentation build configuration file, created by
# sphinx-quickstart on Tue May 21 17:43:32 2013.
#
# This file is execfile()d with the current directory set to its containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import datetime
import sys
import os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.abspath('.'))
on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
# -- General configuration -----------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = ['redirect',
'sphinx.ext.autodoc',
'sphinx.ext.todo',
'sphinx.ext.viewcode',
'openstackdocstheme',
'yasfb',
]
# Feed configuration for yasfb
feed_base_url = 'https://specs.openstack.org/openstack/murano-specs'
feed_author = 'OpenStack Murano Team'
todo_include_todos = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Murano Specs'
copyright = u'%s, OpenStack Murano Team' % datetime.date.today().year
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = [
'_build',
'**/template.rst',
]
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
add_module_names = False
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'native'
# A list of ignored prefixes for module index sorting.
modindex_common_prefix = ['murano-specs.']
# openstackdocstheme options
openstackdocs_repo_name = 'openstack/murano-specs'
openstackdocs_bug_project = 'murano'
openstackdocs_bug_tag = ''
# -- Options for man page output ----------------------------------------------
man_pages = []
# -- Options for HTML output ---------------------------------------------------
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
html_domain_indices = False
# If false, no index is generated.
html_use_index = False
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'Murano-Specsdoc'
# -- Options for LaTeX output --------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'Murano-specs.tex', u'Murano Specs',
u'OpenStack Murano Team', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for Texinfo output ------------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'Murano-specs', u'Murano Design Specs',
u'OpenStack Murano Team', 'murano-specs', 'Design specifications for the Murano project.',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# -- Options for Epub output ---------------------------------------------------
# Bibliographic Dublin Core info.
epub_title = u'Murano Specs'
epub_author = u'OpenStack Murano Team'
epub_publisher = u'OpenStack Murano Team'
epub_copyright = u'2014, OpenStack Murano Team'
# The language of the text. It defaults to the language option
# or en if the language is not set.
#epub_language = ''
# The scheme of the identifier. Typical schemes are ISBN or URL.
#epub_scheme = ''
# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
#epub_identifier = ''
# A unique identification for the text.
#epub_uid = ''
# A tuple containing the cover image and cover page html template filenames.
#epub_cover = ()
# HTML files that should be inserted before the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_pre_files = []
# HTML files shat should be inserted after the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_post_files = []
# A list of files that should not be packed into the epub file.
#epub_exclude_files = []
# The depth of the table of contents in toc.ncx.
#epub_tocdepth = 3
# Allow duplicate toc entries.
#epub_tocdup = True

78
doc/source/index.rst
View File

@ -1,78 +0,0 @@
.. murano-specs documentation master file
=====================
Murano Specifications
=====================
Kilo specs:
.. toctree::
:glob:
:maxdepth: 1
specs/kilo/*
Liberty specs:
.. toctree::
:glob:
:maxdepth: 1
specs/liberty/*
Mitaka specs:
.. toctree::
:glob:
:maxdepth: 1
specs/mitaka/*
Newton specs:
.. toctree::
:glob:
:maxdepth: 1
specs/newton/approved/*
specs/newton/implemented/*
Ocata specs:
.. toctree::
:glob:
:maxdepth: 1
specs/ocata/approved/*
Pike specs:
.. toctree::
:glob:
:maxdepth: 1
specs/pike/approved/*
Queens specs:
.. toctree::
:glob:
:maxdepth: 1
specs/queens/approved/*
==========================
Murano apps Specifications
==========================
.. toctree::
:glob:
:maxdepth: 1
specs/murano-apps/*
==================
Indices and tables
==================
* :ref:`search`

50
doc/source/redirect.py
View File

@ -1,50 +0,0 @@
# A simple sphinx plugin which creates HTML redirections from old names
# to new names. It does this by looking for files named "redirect" in
# the documentation source and using the contents to create simple HTML
# redirection pages for changed filenames.
# Stolen from openstack/nova-specs
import os.path
from sphinx.util import logging
LOG = logging.getLogger(__name__)
def process_redirect_file(app, path, ent):
parent_path = path.replace(app.builder.srcdir, app.builder.outdir)
with open(os.path.join(path, ent)) as redirects:
for line in redirects.readlines():
from_path, to_path = line.rstrip().split(' ')
from_path = from_path.replace('.rst', '.html')
to_path = to_path.replace('.rst', '.html')
redirected_filename = os.path.join(parent_path, from_path)
redirected_directory = os.path.dirname(redirected_filename)
if not os.path.exists(redirected_directory):
os.makedirs(redirected_directory)
with open(redirected_filename, 'w') as f:
f.write('<html><head><meta http-equiv="refresh" content="0; '
'url=%s" /></head></html>'
% to_path)
def emit_redirects(app, exc):
LOG.info('scanning %s for redirects...', app.builder.srcdir)
def process_directory(path):
for ent in os.listdir(path):
p = os.path.join(path, ent)
if os.path.isdir(p):
process_directory(p)
elif ent == 'redirects':
LOG.info(' found redirects at %s' % p)
process_redirect_file(app, path, ent)
process_directory(app.builder.srcdir)
LOG.info('...done')
def setup(app):
app.connect('build-finished', emit_redirects)

1
doc/source/specs
View File

@ -1 +0,0 @@
../../specs

10
requirements.txt
View File

@ -1,10 +0,0 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
pbr>=2.0.0,!=2.1.0 # Apache-2.0
sphinx>=2.0.0,!=2.1.0 # BSD
testrepository>=0.0.18 # Apache-2.0/BSD
testtools>=1.4.0 # MIT
yasfb>=0.8.0
openstackdocstheme>=2.2.1 # Apache-2.0

12
setup.cfg
View File

@ -1,12 +0,0 @@
[metadata]
name = murano-specs
summary = OpenStack Murano Project Development Specs
description-file =
README.rst
author = OpenStack
author-email = openstack-discuss@lists.openstack.org
home-page = http://specs.openstack.org/openstack/murano-specs/
classifier =
Intended Audience :: Developers
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux

22
setup.py
View File

@ -1,22 +0,0 @@
#!/usr/bin/env python
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
import setuptools
setuptools.setup(
setup_requires=['pbr>=2.0'],
pbr=True)

542
specs/kilo/blueprint-template.rst
View File

@ -1,542 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Environment Template Catalogue
==========================================
https://blueprints.launchpad.net/murano/+spec/blueprint-template
Problem description
===================
One of powerful Murano use cases is deploying compound applications, composed by set of
different layers such as DB, web server and others, which implies the deployment of
software not just in an unique VM but in several ones. The environment template is the specification of
the set of VMs plus the applications to be installed on top of. The user can define
environment templates from scratch or reuse and customize them (e.g. including keypairs).
Environment templates not only cover instantiation, but also the specification of such templates,
store them in a catalogue and clone them from the abstract template catalog. In this way,
an abstract environment template catalogue as well as a environment template one will be stored in the
database, storing templates shared among all users and individual ones.
Proposed change
===============
In order to fullfill this functionality, a new entity can be introduced in the MURANO
database. It is the Murano environment-template, which contains the specification about what
is going to be deployed in terms of virtual resources and application information, and it can be deployed
on top of Openstack by translating it into environment. This environment template can be created, deleted,
modified and customized by the users. In fact, it can be instantiate as many times as the user wants.
For instance, the user wants to have different deployments from the same environment template: one for
testing and another for production.
The environment template is composed by a set of services/applications with the software to be installed together their
properties to work with. This software can be instantiate over an virtual service.
In this case the workflow for the creation and the instantiation of the environment template will imply:
1.- Creation of the environment template (including application information)
2.- Transformation of the environment template into the environment (creation of the environment, session and
adding applications to the environment
3.- Deploy the environment on top of Openstack.
The environment template structure and information will be similar to the environment, not including some hardware information, like
default network, or virtual server name. Mainly, the environment template information will contain:
- template name, the name for the template. In case, it is not provided, the request will not be valid.
- services, the application information. For each service it will include information about the applications to be
installed (like tomcat), including application properties like tomcat port. In addition, in case applied, the information about
the virtual server (instance) will be incorporated like keyname, flavor, image and so on. The following lines show
an environment template example.
::
{
"name": "env_template_name",
"services": [
{
"instance": {
"assignFloatingIp": "true",
"keyname": "mykeyname",
"image": "cloud-fedora-v3",
"flavor": "m1.medium",
"?": {
"type": "io.murano.resources.LinuxMuranoInstance",
"id": "ef984a74-29a4-45c0-b1dc-2ab9f075732e"
}
},
"name": "tomcat",
"port": "8080",
"?": {
"type": "io.murano.apps.apache.Tomcat",
"id": "54cea43d-5970-4c73-b9ac-fea656f3c722"
}
}
]
}
Alternatives
------------
None
Data model impact
-----------------
A environment template entity will be introduced in the MURANO object model. This implies its existence
in the database, the inclusion of Template services and the extension of the API. The template entity
can consist on:
template :-
murano:property(temp_id, "created", datetime)
murano:property(temp_id, "updated", datetime)
murano:property(temp_id, "id", ID)
murano:property(temp_id, "name", varchar)
murano:property(temp_id, "tenant-id", varchar)
murano:property(temp_id, "version", bigint)
murano:property(temp_id, "description", text)
murano:property(temp_id, "networking", text)
REST API impact
---------------
The inclusion of the environment-template entity will imply the extension of the API for the environment-template
creation, deletion, updating and translate into the environment.
**POST /templates**
*Request*
+----------+--------------------------------+--------------------------------------+
| Method | URI | Description |
+==========+================================+======================================+
| POST | /templates | Create a new environment template |
+----------+--------------------------------+--------------------------------------+
*Content-Type*
application/json
*Example Payload*
This template description can be composed just by the environment template name, or it can
include the description of all services/applications to be deployed.
1.- Just the template
::
{
'name': 'env_template_name'
}
2.- Specification of all the services
::
{
"name": "env_template_name",
"services": [
{
"instance": {
"assignFloatingIp": "true",
"keyname": "mykeyname",
"image": "cloud-fedora-v3",
"flavor": "m1.medium",
"?": {
"type": "io.murano.resources.LinuxMuranoInstance",
"id": "ef984a74-29a4-45c0-b1dc-2ab9f075732e"
},
"name": "orion",
"port": "8080",
"?": {
"type": "io.murano.apps.apache.Tomcat",
"id": "54cea43d-5970-4c73-b9ac-fea656f3c722"
}
}
]
}
*Response*
::
{
"updated": "2015-01-26T09:12:51",
"networking":
{
},
"name": "template_name",
"created": "2015-01-26T09:12:51",
"tenant_id": "00000000000000000000000000000001",
"version": 0,
"id": "aa9033ca7ce245fca10e38e1c8c4bbf7",
}
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Environment Template created successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 409 | The environment template already exists |
+----------------+-----------------------------------------------------------+
**GET /templates/{env-temp-id}**
*Request*
+----------+--------------------------------+-------------------------------------------------+
| Method | URI | Description |
+==========+================================+=================================================+
| GET | /templates/{env-temp-id} | Obtains the environment template information |
+----------+--------------------------------+-------------------------------------------------+
*Parameters:*
* `env-temp-id` - environment template ID, required
*Content-Type*
application/json
*Response*
::
{
"updated": "2015-01-26T09:12:51",
"networking":
{
},
"name": "template_name",
"created": "2015-01-26T09:12:51",
"tenant_id": "00000000000000000000000000000001",
"version": 0,
"id": "aa9033ca7ce245fca10e38e1c8c4bbf7",
}
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Environment Template created successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 404 | The environment template does not exit |
+----------------+-----------------------------------------------------------+
**DELETE /templates/{env-temp-id}**
*Request*
+----------+-----------------------------------+-----------------------------------+
| Method | URI | Description |
+==========+===================================+===================================+
| DELETE | /templates/<env-temp-id> | Delete the template id |
+----------+-----------------------------------+-----------------------------------+
*Parameters:*
* `env-temp_id` - environment template ID, required
*Response*
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Environment Template deleted successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified environment template doesn`t exist |
+----------------+-----------------------------------------------------------+
**POST /templates/{template-id}/services**
*Request*
+----------+------------------------------------+----------------------------------+
| Method | URI | Description |
+==========+====================================+==================================+
| POST | /templates/{env-temp-id}/services | Create a new application |
+----------+------------------------------------+----------------------------------+
*Parameters:*
* `env-temp-id` - The environment-template id, required
* payload - the service description
*Content-Type*
application/json
*Example*
::
{
"instance": {
"assignFloatingIp": "true",
"keyname": "mykeyname",
"image": "cloud-fedora-v3",
"flavor": "m1.medium",
"?": {
"type": "io.murano.resources.LinuxMuranoInstance",
"id": "ef984a74-29a4-45c0-b1dc-2ab9f075732e"
}
},
"name": "orion",
"port": "8080",
"?": {
"type": "io.murano.apps.apache.Tomcat",
"id": "54cea43d-5970-4c73-b9ac-fea656f3c722"
}
}
*Response*
::
{
"instance":
{
"assignFloatingIp": "true",
"keyname": "mykeyname",
"image": "cloud-fedora-v3",
"flavor": "m1.medium",
"?":
{
"type": "io.murano.resources.LinuxMuranoInstance",
"id": "ef984a74-29a4-45c0-b1dc-2ab9f075732e"
}
},
"name": "orion",
"?":
{
"type": "io.murano.apps.apache.Tomcat",
"id": "54cea43d-5970-4c73-b9ac-fea656f3c722"
},
"port": "8080"
}
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Application added successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 404 | The environment template does not exit |
+----------------+-----------------------------------------------------------+
**GET /templates/{env-temp-id}/services***
Request*
+----------+-------------------------------------+-----------------------------------+
| Method | URI | Description |
+==========+=====================================+===================================+
| GET | /templates/{env-temp-id}/services | It obtains the service description|
+----------+-------------------------------------+-----------------------------------+
*Parameters:*
* `env-temp-id` - The environment template ID, required
*Content-Type*
application/json
*Response*
::
[
{
"instance":
{
"assignFloatingIp": "true",
"keyname": "mykeyname",
"image": "cloud-fedora-v3",
"flavor": "m1.medium",
"?":
{
"type": "io.murano.resources.LinuxMuranoInstance",
"id": "ef984a74-29a4-45c0-b1dc-2ab9f075732e"
}
},
"name": "tomcat",
"?":
{
"type": "io.murano.apps.apache.Tomcat",
"id": "54cea43d-5970-4c73-b9ac-fea656f3c722"
},
"port": "8080"
},
{
"instance": "ef984a74-29a4-45c0-b1dc-2ab9f075732e",
"password": "XXX",
"name": "mysql",
"?":
{
"type": "io.murano.apps.database.MySQL",
"id": "54cea43d-5970-4c73-b9ac-fea656f3c722"
}
}
]
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Tier created successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 404 | The environment template does not exit |
+----------------+-----------------------------------------------------------+
**POST /templates/{env-temp-id}/create-environment**
*Request*
+----------+--------------------------------------------+--------------------------------------+
| Method | URI | Description |
+==========+============================================+======================================+
| POST | /templates/{env-temp-id}/create-environment| Create an environment |
+----------+--------------------------------------------+--------------------------------------+
*Parameters:*
* `env-temp-id` - The environment template ID, required
*Payload:*
* 'environment name': The environment name to be created.
*Content-Type*
application/json
*Example*
::
{
'name': 'environment_name'
}
*Response*
::
{
"environment_id": "aa90fadfafca10e38e1c8c4bbf7",
"name": "environment_name",
"created": "2015-01-26T09:12:51",
"tenant_id": "00000000000000000000000000000001",
"version": 0,
"session_id": "adf4dadfaa9033ca7ce245fca10e38e1c8c4bbf7",
}
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Environment template created successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 404 | The environment template does not exit |
+----------------+-----------------------------------------------------------+
| 409 | The environment already exists |
+----------------+-----------------------------------------------------------+
Versioning impact
-------------------------
Murano client will change to include this new functionality.
Other end user impact
---------------------
As well as a change in the API to include this new entity, the python-muranoclient will
be changed for including the environment template operations.
* env-template-create Create an environment template.
* env-template-delete Delete an environment template.
* env-template-list List the environment templates.
* env-template-rename Rename an environment template.
* env-template-show Show the information of the environment template
* env-template-add-app Add an application to the environment template
* env-template-create-environment It creates an environment from the environment template description
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
New views will be required for including the environment template functionality
Implementation
==============
Assignee(s)
-----------
Primary assignee:
hmunfru
Other contributors:
jesuspg
TBC
Work Items
----------
1.- Including the environment template entity in database
2.- Extension of the API for environment template catalogue
3.- Generation of environment from template operation
4.- Implement the changes in murano CLI
Dependencies
============
Testing
=======
TBD
Documentation Impact
====================
Environment template documentation should be included.
References
==========
https://etherpad.openstack.org/p/GLLAQ0m1H7

224
specs/kilo/conf-language-support.rst
View File

@ -1,224 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==============================
Configuration Language Support
==============================
https://blueprints.launchpad.net/murano/+spec/conf-language-support
Problem description
===================
There is a huge community of applications (opscode, puppet-labs) where
deployment installation instructions are specified in configuration
languages such as Puppet or Chef. In order to reuse these applications,
adaptors like Chef or Puppet are required on the VM side. Both chef and
puppet recipes will not be managed by centralized server (chef-server,
puppet-master), but they will use the standalone version, specifically
the usage of chef-solo and puppet apply.
Proposed change
===============
Inclusion of new executors in the murano-agent project. These executors
will be objects to be used by murano-agent. Specifically, two executors
will be implemented: Puppet and Chef. Both executors will be in charge of:
* Obtaining the required modules or cookbooks in the virtual machine.
This task can be done by passing the information from murano-engine to
murano-agent, obtaining the information from the package itself. It requires
the user to upload the package information plus the cookbooks to be used.
The second option implies the cookbooks are downloaded in the virtual
machine, so that they only need the URL to be accessible.
* Generating the required files for the configuration language.
For instance, manifests and hiera data for puppet, and, node
specifications for chef from the information stored in the
execution plan.
* Executing the chef-solo or puppet-apply process.
Previously, some work has to be done to install Chef or Puppet inside the VM.
This task can be done by using cloud-init from the murano engine. The
following is an example on how these executors work:
::
## YAML Template.
---
FormatVersion: 2.0.0
Version: 1.0.0
Name: Deploy Tomcat
Parameters:
port: $port
Body: |
return deployTomcat(port=args.port).stdout
Scripts:
deployTomcat:
Type: Chef
Version: 1.0.0
EntryPoint: mycoockbook::myrecipe
Files:
tomcat:
Name: tomcat
URL: git://github.com/opscode-cookbooks/tomcat.git
Type: Downloadable
java:
Name: java
URL: git://github.com/opscode-cookbooks/java.git
Type: Downloadable
ssl:
Name: openssl
URL: https://github.com/opscode-cookbooks/ssl.git
Type: Downloadable
Options:
captureStdout: true
captureStderr: true
In this case, a new script Type appears (instead of Application). It is
Chef type, which will execute the Chef executor. The same happens with
the Puppet Type. In addition, the EntryPoint contains the information
about the cookbook and the recipe to be installed. The Files section
is used for the cookbooks and its dependence information. The cookbooks
properties are in the Parameter section.
All the required steps to be part of the executor can be summarized as follows.
For Chef,
#. Creating the node.json with the recipes and the configuration parameters::
{
orion::ports: 1026
orion::dbname: oriondb
"run_list": [
"recipe[orion::0.13.0_install]"
]
}
#. Executing chef-solo:
chef-solo -j node.json
For puppet,
#. Generating the manifest (site.pp)::
node 'default'
{
class{
'orion::install':
}
}
#. Creating the hiera data information: hiera.yaml::
## YAML Template.
---
orion::port: 1026
orion::dbname: oriondb
#. Executing:
puppet apply --hiera_config=hiera.yaml --modulepath=/opt/puppet/modules/orion site.pp
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
The solution proposed is valid for any VM which contains the configuration
language implementation already installed. There are event chef-solo and
puppet agents for Windows.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
hmunfru
Other contributors:
jesuspg
Work Items
----------
#. Generate Chef executor
#. Generate Puppet executor
#. Work on configuration
Dependencies
============
None
Testing
=======
Integration tests will be done
Documentation Impact
====================
Information about how to defines application for Puppet and Chef will have
to be documented, explaining the different fields.
References
==========
* http://es.slideshare.net/hmunfru/fiware-and-murano-support-for-configuration-languages
* https://etherpad.openstack.org/p/conf-language-support-spec

189
specs/kilo/congress-support.rst
View File

@ -1,189 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
======================================================
Policy Guided Fulfillment - Congress Support in Murano
======================================================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/congress-support-in-murano
Problem description
===================
As a part of policy guided fulfillment we need to call Congress from Murano,
to enforce the policy on Murano environment. For current release
the enforcement will be done by Congress simulation API, where Murano will
query table *predeploy_error* in *murano_system* policy for passed update
sequence created by mapping of environment into congress schema.
Proposed change
===============
We need to provide
* python congress client in Murano
We will use simulation feature of `congress API
<https://docs.google.com/document/d/14hM7-GSm3CcyohPT2Q7GalyrQRohVcx77hx
Ex4AO4Bk/edit#heading=h.ll03wo2z9pcb>`_.
Using this API Murano will send decomposed Murano environment to Congress
tables, so simulation can evaluate *predeploy_error* rule without changing
existing data in Congress.
* mapping of Murano Environment into *update sequence* used in simulation API
Murano To Congress Mapping Details
----------------------------------
This section provides congress schema created for Murano environment mapping.
It will be created by Congress datasource drivers - see `murano driver spec
<https://blueprints.launchpad.net/congress/+spec/murano-driver>`_ .
* **Policies**
* **murano**
Dedicated policy for Murano data.
* **murano_system**
Dedicated policy for rules
* **Schema**
* *murano:objects(obj_id, owner_id, type)*
This table holds every MuranoPL object instance in an environment.
* *obj_id* - uuid of the object as used in Murano
* *owner_id* - uuid of the owner object as used in Murano
* *type* - string with full type indentifier as used in Murano
(e.g., io.murano.Environment,...)
* *murano:parent_types(obj_id, parent_type)*
This table holds parent types of *obj_id* object. Note that Murano
supports multiple inheritance, so there can be several parent types for
one object
* *murano:properties(obj_id, name, value)*
This table stores object's properties. For multiple cardinality
properties, there can be number of records here. MuranoPL properties
referencing class type (i.e., another object) are stored in
*murano:relatinoship*. Properties with structured content will be
stored component by component.
* *murano:relationships(src_id, trg_id, name)*
This table stores relationship between objects (i.e., MuranoPL property
to *class*). For multiple cardinality relationships several records
should be stored.
* *murano:connected(src_id, trg_id)*
This table stores tuples of objects connected directly and indirectly
via relationship. It is necessary since Congress does not support
recursive rules yet.
* *murano:states(end_id, state)*
This table stores *EnvironmentStatus* of Murano environment ( one of
'ready', 'pending', 'deploying', 'deploy failure', 'deleting',
'delete failure' ).
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
filip-blaha
Other contributors:
ondrej-vojta, radek-pospisil
Work Items
----------
1. Introduce Congress python client into Murano
2. Implement mapping of Murano Environment into Congress simulation API
update-sequence.
3. Provide tests
Dependencies
============
* Congress python client ( `GIT <https://opendev.org/openstack/python-congressclient>`_ )
will be added into Murano.
* *Murano datasource driver* in Congress `murano driver spec
<https://blueprints.launchpad.net/congress/+spec/murano-driver>`_
Testing
=======
Testing will use predefined Congress policy rules in order to test client and
mapping. See https://etherpad.openstack.org/p/policy-congress-murano-spec for
an example mapping and test.
Documentation Impact
====================
Documentation impact is specified in `Policy Enforcement Point <https://bluepri
nts.launchpad.net/murano/+spec/policy-enforcement-point>`_ blueprint.
References
==========
* *Murano datasource driver* in Congress
https://blueprints.launchpad.net/congress/+spec/murano-driver
* https://blueprints.launchpad.net/murano/+spec/policy-enforcement-point
* https://etherpad.openstack.org/p/policy-congress-murano-spec

309
specs/kilo/enable-category-management.rst
View File

@ -1,309 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
====================================================
Provide opportunity to manage application categories
====================================================
https://blueprints.launchpad.net/murano/+spec/enable-category-management
Murano is an application catalog, where new applications can be easily added.
Those applications may belong to a category, that is not on predefined list.
Also, some categories may not be needed, and user can delete such kind
of categories.
All operations should be available only for admin users.
Problem description
===================
Category management should be provided during:
* Packages uploading in dashboard;
* Packages modifying in dashboard;
* Packages uploading or modifying via command line;
Adding new category:
* Category name may contain spaces and other special characters;
* Category name limit is equal to 80 characters;
* Only admin users can add new category;
Deleting category:
* Category may be deleted only if no packages belong to this category;
* Only admin users can delete category;
Proposed change
===============
Changes required to support this feature:
* Add new panel named 'Categories' under 'Manage' section;
It should be available only for admin users.
This panel should represent table with categories.
The table will contain 'name' column and 'assigned packages' column.
"Add category" will be a table action and "Delete Category" will be
row action.
Delete button should be hidden for those categories connected to
the packages.
* Provide corresponding methods in the python-muranoclient;
Category manager will be added to v1 module;
Alternatives
------------
Admin can edit a database to add or delete categories.
Data model impact
-----------------
None
REST API impact
---------------
**GET /catalog/categories**
The previous call /catalog/packages/categories is will still be valid to
support backward compatibility.
*Request*
+----------+----------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==================================+==================================+
| Get | /catalog/categories | Get list of existing categories |
+----------+----------------------------------+----------------------------------+
*Response*
::
{"categories": [
"id": "3dd486b1e26f40ac8f35416b63f52042",
"updated": "2014-12-26T13:57:04",
"name": "Web",
"created": "2014-12-26T13:57:04",
"package_count": 0
},
{
"id": "k67gf67654f095gf89hjj87y56g98965v",
"updated": "2014-12-26T13:57:04",
"name": "Databases",
"created": "2014-12-26T13:57:04",
"package_count": 0
}]
}
**GET /catalog/categories/<category_id>**
*Request*
+----------+-----------------------------------+----------------------------------+
| Method | URI | Description |
+==========+===================================+==================================+
| Get | /catalog/categories/<category_id> | Get category detail |
+----------+-----------------------------------+----------------------------------+
*Parameters*
* `category_id` - category ID, required
*Response*
::
{
"id": "0420045dce7445fabae7e5e61fff9e2f",
"updated": "2014-12-26T13:57:04",
"packages": [
"Apache HTTP Server",
"Apache Tomcat",
"PostgreSQL"
],
"name": "Web",
"created": "2014-12-26T13:57:04"
}
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Category retrieved successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified category doesn`t exist |
+----------------+-----------------------------------------------------------+
**POST /catalog/categories**
+----------------------+------------+--------------------------------------------------------+
| Attribute | Type | Description |
+======================+============+========================================================+
| name | string | Category name |
+----------------------+------------+--------------------------------------------------------+
*Request*
+----------+----------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==================================+==================================+
| POST | /catalog/categories | Create new category |
+----------+----------------------------------+----------------------------------+
*Content-Type*
application/json
*Example*
{"name": "category_name"}
*Response*
::
{
"id": "ce373a477f211e187a55404a662f968",
"name": "category_name",
"created": "2013-11-30T03:23:42Z",
"updated": "2013-11-30T03:23:44Z",
}
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Category created successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 403 | Forbidden. Category with specified name already exist |
+----------------+-----------------------------------------------------------+
**DELETE /catalog/categories**
*Request*
+----------+-----------------------------------+-----------------------------------+
| Method | URI | Description |
+==========+===================================+===================================+
| DELETE | /catalog/categories/<category_id> | Delete category with specified id |
+----------+-----------------------------------+-----------------------------------+
*Parameters:*
* `category_id` - category ID, required
*Response*
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Category deleted successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to access this session |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified category doesn`t exist |
+----------------+-----------------------------------------------------------+
| 403 | Forbidden. Category with specified name is assigned to |
| | the package, presented in the catalog. Only empty |
| | categories can be removed |
+----------------+-----------------------------------------------------------+
Versioning impact
-----------------
Murano dashboard will support only the version of the client, that includes
corresponding changes in the client. 'Categories' panel will not work
with the old murano version, but application catalog and package management
will work fine.
Other end user impact
---------------------
None
Murano-dashboard / Horizon impact
---------------------------------
Category management will be available in dashboard.
Areas to be changed: (described in sections above)
* Manage section will have new panel;
Deployer impact
---------------
None
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Ekaterina Chernova
Primary assignee:
<efedorova@mirantis.com>
Work Items
----------
* Introduce 2 additional calls in API
* Update API specification
* Provide these calls in python-muranoclient
* Implement changes in dashboard
* Enable CLI to manage categories
Dependencies
============
None
Testing
=======
New tests should be added in dashboard integration tests
Documentation Impact
====================
API specification should be updated.
All changes are already represented here, just need to copy.
References
==========
None

165
specs/kilo/murano-agent-timeouts.rst
View File

@ -1,165 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==================================
Add timeouts to murano-agent calls
==================================
https://blueprints.launchpad.net/murano/+spec/murano-agent-timeouts
Now there is no way to be sure that the agent successfully started execution
on a VM. Also there is no control of the execution time of scripts on agent.
This process should be more controllable. It can be done by adding timeouts
in Murano engine.
Problem description
===================
* During the agent`s work there could be some problems with execution of scripts and
VM may hang. In this case user will wait indefinitely without knowing
what's happened.
* Currently there is no feedback from agent so, it`s impossible to determine
whether agent is ready to accept execution plans or not.
It is proposed to provide mechanism of timeouts which solves these problems.
Proposed change
===============
1) First of all add timeout to method *call* of agent on engine-side.
Optional parameter *timeout* will be added to method *call* of class *Agent*
in agent.py. This parameter is the time in seconds with default value *600*.
Developer can set up custom value during developing apps for example in this
way::
- $.instance.agent.call($template, $resources, 300)
If the agent plan execution time exceeds the limit, it will be terminated.
2) Add method *waitReady* in Agent class.
Method will be added to *Agent* class in agent.py. It has optional parameter
*timeout* with default value *100*. *waitReady* creates test plan with trivial
body::
template = {'Body': 'return', 'FormatVersion': '2.0.0', 'Scripts': {}}
and sends this plan once to agent by method *call*. It can be used by developer
to stop deployment before sending template of app if agent is inaccessible as
follows::
- $.instance.agent.waitReady()
- $.instance.agent.call($template, $resources)
If the agent test plan execution time exceeds the time limit,
*TimeoutException* will be raised and deployment terminates. *TimeoutException*
will be created in murano/common/exceptions.py.
3) Add new method *isReady* in Agent class.
This method will simply call the *waitReady*. Method *isReady* returns:
* *True*, if test plan is executed on time;
* *False*, if the agent plan execution time exceeds the
limit.
* and raise *PolicyViolationException*, which will be created in
murano/common/exceptions.py, if the agent disabled by the server.
The method can be used during development Murano-applications. For example,
developer can check if the agent is running and ready to execute templates
before sending the execution plan of application:
::
- If: $.instance.agent.isReady()
Then:
- $._environment.reporter.report($this, 'Murano Agent is ready')
The message in above example will be reported to Murano Dashboard.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:lk
ddovbii
Work Items
----------
The change is simple and can be done in one Gerrit patch. Implementation is
acutally completed.
Dependencies
============
None
Testing
=======
Unit and integration tests must be done.
Documentation Impact
====================
MuranoPL specification should be updated.
References
==========
None

141
specs/kilo/murano-mistral-integration-1.rst
View File

@ -1,141 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================
murano-mistral-integration
==========================
https://blueprints.launchpad.net/murano/+spec/murano-mistral-integration-1
The purpose is to add integration between Murano and Mistral.
We would like to allow invocation of a Mistral workflow from Murano.
No prerequisite for uploaded Mistral workflow.
Problem description
===================
A new capability in Murano modeling process should be added in order to answer
several use cases, including:
#. The application modeler wishes to leverage existing workflow that deploys
a specific component as part of a new application model creation process.
#. The application modeler wishes to add post-deployment logic to an
application model.
For example:
a. Check that the application has been successfully deployed.
b. Inject initial data to the deployed application.
Proposed change
===============
Adding a new system class for Mistral Client that allows to call Mistral
APIs from the Murano application model.
The system class will allow you to:
#. Upload a Mistral workflow to Mistral.
#. Trigger the already-deployed Mistral workflow, wait for completion and
return the execution output.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Natasha Beck
Work Items
----------
#. Add mistral client system class that can trigger pre-deployed Mistral
workflow.
#. Add ability to the client to upload the Mistral workflow.
#. Add test that deploys application which uploads the Mistral workflow from
Resources, triggers it and gets output as a result.
Dependencies
============
Openstack Mistral component.
Testing
=======
Functional test will be added to Murano.
The test deploys application which uploads the Mistral workflow from
Resources, triggers it and gets output as a result.
Documentation Impact
====================
Murano-Mistral integration must be documented from the following perspectives:
* Setup configuration (for example with and without mistral)
* Include usage of Mistral workflow as a part of an application model
The following Murano documentation will be affected:
* Murano Installation Guide
Add section on Mistral requirement
* Murano Workflow
Add section on Murano-Mistral integration
* Murano Article (new)
Article on Murano-Mistral integration
References
==========
None

190
specs/kilo/muranoclient-marketplace-support.rst
View File

@ -1,190 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=========================
Murano Repository Support
=========================
https://blueprints.launchpad.net/murano/+spec/muranoclient-marketplace-support
Murano applications provide a powerful and flexible way to move workloads
from other cloud environments to OpenStack. In order to accelerate
application migrating we need a way to deliver Murano
packages and Glance images to customers incrementally, independent from major
releases of OpenStack.
Problem description
===================
Typical use cases:
* After the end users installed and configured Murano they would need to
install murano-enabled applications. To do so they would use `murano` CLI
client. Invoking a command like
`murano install-package io.apps.application --version=2.0` would
install the application in question alongside with all requirements
(applications and glance images)
* A developer would want to provide murano applications to end users, by
hosting them on a web server and providing http-access to application
packages. In that case end users would be able to install the application by
invoking a CLI command `murano install-package http://path/to/app`
* End user would want to install bundles of applications that are often
required to work together, but not necessarily depend on each other.
In that case end user would invoke a command
`murano install-bundle bundle_name` and murano-client would download all
the applications mentioned in the bundle alongside with their requirements.
* End users would want to perform same operations through
murano-dashboard instead of CLI tools.
Proposed change
===============
The proposition is to implement a set of features in python-muranoclient and
murano-dashboard, that would allow end users to install applications from
http application repository.
* Enable `package-import` command to support url as a parameter for import by
modifying the way package creation of murano-client works.
* Enable `package-import` command to support package name as a parameter for
import, introduce `--murano-repo-url` as a base path to the repository.
* Introduce `bundle-import` command and allow it to support local-files, urls
and bundle names as parameters for import. The command should parse the
bundle file and import all the packages, mentioned in the file.
Bundle should be a simple json and/or yaml-structured file.
* Introduce `Require:` section to `manifest.yaml`, that would specify which
packages are required by the appication.
* Enable package creation suite to automatically import all the packages
mentioned in `Require:` section.
* Introduce `images.lst` file in the package structure, that would contain a
list of images, required for the application to work.
* Enable package creation suite to automatically import required images into
glance image service.
* Allow `bundle-import` command to import bundles from local-files. In that
case first search for package files and image files in the local filesystem,
relative to the location of the bundle file. If file is not found locally
attempt to connect to the repository.
* Enable murano-dashboard to support changes made to client and introduce a way
to upload packages via url/name to dashboard.
* Implement importing of bundles in murano-dashboard by url or by name.
Since bundles are currently a simple json/yaml optionally we could optionally
support direct input.
* Implement error handling both for CLI-tools and dashboard, that would inform
end users about any errors that might have happened along the way of
importing.
* Optionally implement a progress marker and a ETA-marker for CLI `import`
commands.
Alternatives
------------
Implementing a server, that would hold versions and paths to package files
and implementing a client to that server might be a good alternative to a
simple http-server, although it seems a bit excessive at the moment.
Another idea would be to implement a service, that would download packages
asynchronously. This would allow client to download large package files and
image files with possibly better error handling mechanisms. This also seems a
bit excessive at the moment.
Data model impact
-----------------
It might be a good idea to store information about required apps and required
images, although it is not strictly required for the task.
REST API impact
---------------
It might be a good idea to warn the end user if the client installed a
package, that depends on other packages, not present in app catalog,
although it is not strictly required for the task.
Versioning impact
-----------------
This proposition adds functionality both to python-muranoclient and to
murano-dashboard. It should be fully backward compatible.
Minor version of python-muranoclient should be increased, because we add
functionality.
Other end user impact
---------------------
See *Proposed Change* section, as it describes all the ways users would
interact with the feature
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
Package import dialog should be changed to reflect different ways of
importing an application.
Additional bundle-import dialog should be implemented.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<kzaitsev@mirantis.com>
Work Items
----------
* Add support for importing packages by url in client
* Add support for importing packages by name in client
* Add support for importing bundles by name/url in client
* Add support for recursive parsing of Require section of `manifest.yaml`
file in client
* Add support for image download/upload from `images.lst` file in client
* Handle `package exists` error in CLI client
* Add support for different ways to import a package in murano-dashboard
* Add support for bundle-import in murano-dashboard
* Add support for image/package requirements while importing packages and
bundles in murano-dashboard
Dependencies
============
None
Testing
=======
Unit testing should be sufficient to cover most of the use cases.
However an integration test, that would setup a simple repository is very
desirable, since we add changes to both `python-muranoclient` and
`muranodashboard`.
Documentation Impact
====================
Additional capabilities of the CLI client should be documented, otherwise
there should be no impact, since we do not change the API.
References
==========
None

1
specs/kilo/placeholder.rst
View File

@ -1 +0,0 @@
../template.rst

221
specs/kilo/plugable-classes.rst
View File

@ -1,221 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
====================================
Plugable pythonic classes for Murano
====================================
https://blueprints.launchpad.net/murano/+spec/plugable-classes
One of the key features of Murano is extensibility, and we need to push this
feature even further and give our customers a way to extend Murano with new
functions (e.g. support for F5 BigIP API) in a drag-n-drop manner. This spec
proposes a solution which will add this extensibility option.
Problem description
===================
Currently all the functionality which is available to user is limited to the
features of MuranoPL language which just provides data transformation
capabilities and flow control primitives. The language itself does not contain
any functions for I/O operations, hardware access or interaction with host
operating system, other OpenStack or third-party services. This is an
intentional design feature: MuranoPL code is provided by users and cannot be
always trusted. All the external communications and low-level interactions are
done via python code which is bundled with Murano Engine and is accessible to
MuranoPL code via MuranoPL wrappers. Any interactions which are not supported
by that Python classes are impossible.
Some deployment scenarios may need to extend this set of allowed low-level
interactions. They may include some customer-specific logic, custom software
bindings etc, so trying to bundle all of them into the standard Murano Engine
classes is not a good idea. Instead, there should be a way to dynamically
add extra interactions to any existing deployment of Murano without modifying
its core components but rather with installing some plugin-like components.
Installing these plugins is supposed to be a maintainer-only operation,
requiring administrative access to nodes running Murano services. It is
supposed that the maintainer is always aware about the contents of the plugins
and is able to verify them from security, performance and other sensible points
of view.
Proposed change
===============
It is proposed to implement each extension as independent Python Package built
using setuptools library. Each package should define one or more entry-point in
a specific namespace (``io.murano.extensions`` is suggested). Each of this
entry- points should export a class, which may be registered as MuranoPL class
when the engine loads.
Each package should be installed on Murano nodes into the same Python
environment with Murano engine service.
Murano will get a PluginLoader class which will utilize stevedore library [1]
to discover classes registered as entry-points in ``io.murano.extensions``
namespace.
Murano Engine will use PluginLoader to register all the loaded plugins in its
class loader (i.e. will call ``import_class`` with a class imported from the
plugin as a parameter). As the result, the classes will become available for
the MuranoPL code being executed in the Engine.
To prevent potential name collisions, MuranoPL names for the loaded classes
will be assigned automatically: the name will consist of the namespace
(``io.murano.extensions`` as suggested above) and the name of entry-point.
To guarantee this naming rule the imported classes should not define their
MuranoPL names on their own (i.e they should not have
``@murano_class.classname`` decorators or other code which modifies their
``_murano_class_name`` field). If they do, the PluginLoader will discard
that information and will log a warning message.
As the entry-point name will eventually become a name of MuranoPL class, the
PluginLoader will validate it accordingly.
As neither stevedore nor setuptools enforce any uniqueness constraints on the
entry-point names (i.e. several packages may define entry-points with the same
name within a same namespace, and all of them will be correctly loaded by
stevedore), then this enforcement should also be done by the Murano's Plugin
Loader. If two or more plugin packages attempt to register classes with the
same endpoint name, then a warning will be logged and no classes from all the
conflicting packages will be loaded.
PluginLoader will also ensure that objects being exported in these entry-points
are indeed classes, and will check if they define a classmethod called
``init_plugin``. If such method exists, the PluginLoader will execute it before
loading it.
The plugins which are already installed in the environment may be prevented
from being loaded by a configuration option. This new option called
``enabled_plugins`` will be added to ``murano.conf``. If it has its default
value ``None`` or is omitted from the config, there will be no restriction on
the plugins which are being loaded (any plugin registered within the
environment will be loaded). If it exists and is not None, then it is expected
to contain a list of names of the packages from which the plugins will be
loaded. If the package is not mentioned there, all its endpoints will be
ignored and no classes from it will be imported. Empty value of
``enabled_plugins`` will mean that no plugins may be loaded and only bundled
system classes are accessible from the MuranoPL code.
The ``enabled_plugins`` setting will be implemented using
``EnabledExtensionManager`` class of the stevedore library [2], so the disabled
plugins will be excluded from entry-point name analysis. Thus if there are
plugins which define conflicting entry-point names, then the conflict may be
resolved with this setting instead of uninstalling the plugin from the
environment.
Currently stevedore is unable to load packages which were installed after the
start of the current process. So, in current proposal it is required to restart
Murano services after plugin package is installed, removed or upgraded and
after the changing of ``enabled_plugins`` value in configuration file.
As the restart of the services is not a good thing for production solutions, it
may be a good idea to design a "graceful restart" solution which will make the
service to stop listening for incoming requests, finish its current tasks and
then exit and restart, loading the updated configuration and plugins. However
such solution is out of scope of the current spec and is left for future
blueprints.
Alternatives
------------
Instead of using stevedore to discover and load the plugins, some home-made
solution may be invented to load Python modules from some directory. This
solution may have its benefits (e.g. it does not require restarts to load new
plugins), however stevedore is currently a de-facto standard for building
plugable solutions in Openstack, so it is suggested to use it.
Data model impact
-----------------
This proposal does not affect data model.
REST API impact
---------------
N/A
Versioning impact
-----------------
N/A
Other end user impact
---------------------
N/A
Deployer impact
---------------
The change itself does not have any immediate impact on deployer: a new
configuration option is optional and has meaningful default. However
registering new plugins will require to restart the Services, which may bring
up some concerns in production environments.
Developer impact
----------------
Developers who build their own plugins should be aware about setuptools entry-
points and should inherit their exported classes from
`murano.dsl.murano_object.MuranoObject`.
Murano-dashboard / Horizon impact
---------------------------------
No immediate changes required.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ativelkov
Work Items
----------
* Implement the PluginLoader class
* Modify MuranoEngine to register plugin-imported classes in class loader.
Dependencies
============
This requires stevedore library as a dependency. It is already part of
OpenStack Global Requirements, so no problems are expected.
Testing
=======
The unit-tests have to cover PluginLoader class using the make_test_instance
method of stevedore.
Separated tests should cover API method call.
Tempest tests are out of the scope of this spec.
Documentation Impact
====================
There should be created a "Plugin developer's Manual" which will describe the
process of plugin package creation.
References
==========
[1] http://docs.openstack.org/developer/stevedore/
[2] http://docs.openstack.org/developer/stevedore/managers.html#enabledextensionmanager

207
specs/kilo/policy-enforcement-point.rst
View File

@ -1,207 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
====================================================
Policy Guided Fulfillment - Policy Enforcement Point
====================================================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/policy-enforcement-point
Problem description
===================
As a part of policy guided fulfillment we need to implement *predeploy* policy enforcement point - i.e., Murano calls Congress to evaluate *predeploy* policy rules on data representing Murano environment being deployed. If evaluation returns *predeploy error* data (i.e., enforcement failed), then deployment of Murano environment fails.
*Predeploy* policy rules are represented by *predeploy_error(env_id, obj_id, message)* congress table.
It means
* Congress administrator is responsible for creating predeploy rules, which has this table on the left side (see examples).
* Murano is not involved in policy rule evaluation, except that Murano provides data about Murano environments. Thus user can use any data (e.g., datasource tables, policy rules) available in Congress to define when environment can be deployed.
Table *predeploy_error(env_id, obj_id, message)* reports list of found errors:
* *env_id* environment id where error was detected
* *obj_id* object id (in environment id) on which error was detected
* *message* text message of error
Murano environment serialization/decomposition is described in `Congress Support in Murano <https://blueprints.launchpad.net/murano/+spec/congress-support-in-murano>`_ and `Murano Congress Driver <https://blueprints.launchpad.net/congress/+spec/murano-driver>`_
Example (generic):
::
predeploy_error(env_id, obj_id, message) :-
murano:state(env_id,"PENDING"),
my-rule-table1(env_id, obj_id),
concat("", "some error message 1", message)
predeploy_error(env_id, obj_id, message) :-
murano:state(env_id,"PENDING"),
my-rule-table2(env_id, obj_id),
concat("", "some error message 2", message)
my-rule-table1(env_id, obj_id) :- ....
my-rule-table2(env_id, obj_id) :- ....
Example (allow only environments where all VM instances has flavor with max 2048MB RAM):
::
predeploy_error(eid, oid, msg) :-
murano:object(oid, eid, type),
checkIfError(oid),
concat("", "Instance flavor has RAM size over 2048MB", msg)
checkIfError(oid) :-
murano:parent_type(oid, "io.murano.resources.Instance"),
murano:property(oid, "flavor", fname),
nova:flavors(i,fname,v,r,d,e,rx),
gt(r,2048)
Enforcement point will use Congress simulation api to evaluate rules on passed mapped environment into Congress schema.
Proposed change
===============
When user executes *deploy* action on an environment following steps will be done
* environment will be mapped into *update sequence*
* Congress simulation API will be executed:
::
openstack congress policy simulate murano_system 'predeploy-error(envId, objId, message)' 'env+(1000,"myEnv") obj+(1,1000,"VM") prop+(100,1,"name", "vm1") prop+(101,1,"image", "ubuntulinuximg") obj+(2,1000,"Tomcat") prop+(110,2,"name", "tomcat") prop+(111,2,"port", 8080) rel+(200,2,1, "instance") ' action
* if response will contain non empty result, then deployment will fails
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
Environment deployment may fail due to policy validation failure.
Deployer impact
---------------
Policy enforcement will be used only if
* Enforcement is enabled in *murano.conf*
* Congress is available in Keystone catalog (i.e., it is deployed in OpenStack)
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ondrej-vojta
Other contributors:
filip-blaha, radek-pospisil
Work Items
----------
1. Use implementation of *Congress Support in Murano* in order to implement policy enforcement point as advised by Stan (see below). The Congress support must correctly deal with following setups
* Openstack with Congress installed
* Openstack without Congress
::
Stan: such approach makes PolicyEnforces (and thus dependency on Congress) be mandatory for Murano. Better approach for now could be just insert
at https://github.com/stackforge/murano/blob/master/murano/common/engine.py#L112
something like
if config.CONF.enable_policy_enforcer:
policyenforcer.validate(self.model)
2. Provide Developer and User Documentation (see Documentation section).
Dependencies
============
* *Congress Support in Murano* https://blueprints.launchpad.net/congress/+spec/murano-driver
* *Murano datasource driver* in Congress https://blueprints.launchpad.net/congress/+spec/murano-driver
* *Policy enforcement specification* https://etherpad.openstack.org/p/policy-congress-murano-spec
Testing
=======
Unit and integration tests must be done.
Integration tests must cover following setups
* Openstack has Congress installed
* situations when Congress is running (i.e., responding) and not running (i.e., not responding) must be tested
* Openstack has not Congress installed
Documentation Impact
====================
Policy enforcement must be documented from following perspectives
* Setup configuration (e.g., with and without congress)
* Murano rules (i.e., Murano environment data decompositiion) in Murano policy
* How Murano policy affects environment deployment
Following Murano documentation will be affected
* Murano Installation Guide
Add section on Congress requirement and section on enabling policy enforcement
* Murano Workflow
Add section on Murano policy enforcement
* Murano Article (new)
Article on Murano policy rules (e.g., Murano environment decomposition to Congress)
References
==========
* *Congress Support in Murano* https://blueprints.launchpad.net/congress/+spec/murano-driver
* *Murano datasource driver* in Congress https://blueprints.launchpad.net/congress/+spec/murano-driver
* *Policy enforcement specification* https://etherpad.openstack.org/p/policy-congress-murano-spec

215
specs/liberty/add-support-for-heat-environments.rst
View File

@ -1,215 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=================================
Add support for heat environments
=================================
https://blueprints.launchpad.net/murano/+spec/add-support-for-heat-environments-and-files
Add support for using Heat environments, when saving and deploying Heat
templates.
Problem description
===================
Today there is no option to create stacks neither with an environment nor with
nested stacks via Murano. Only basic stacks are supported.
Use cases for deploying a stack with an environment:
* Supporting resource registry
* Supporting saving 'test' and 'production' parameter profiles for the same
template
Proposed change
===============
The change described here will focus on supporting Heat environments. Another
spec was submitted for Heat files.
In this spec no UI changes are included. A UI part that can be added in future
specs, and in any case is out of the scope of this spec.
Allowing Heat environments to be saved and deployed as part of a Murano Heat
package. Adding environments to a package should be optional. Heat environment
files should be placed in the package under '/Resources/HotEnvironments' When a
user request to add a Heat-generated package to a Murano environment, he should
have an option to specify one of the hot-environment-files located at
'/Resources/HotEnvironments' in the package, and this heat environment file
should be sent to Heat with the template during Murano environment deployment.
If no heat environment file was specified, the template will be deployed
without a Heat environment.
The stage where the Heat environment for the deployment should be specified
is when adding a package to a Murano environment as part of a configuration
session.
The heat environment should be referenced by name.
In the future, there can be an API to list the hot environments of a package.
This might be done by using the existing API of getting the UI of a package.
Alternatives
------------
* The package structure can be different.
* Words that can replace the word environments to reference heat environments:
configurations, profiles, settings. The problem is that in Heat they use the
word environments.
* Specifying the heat environment to be deployed with the Heat template as
part of a Murano environment when deploying the environment (instead of
specifying it when adding a package to an environment). If we will wait to
this point, we will have to give a map of packages and the environments to be
deployed with them. this alternative requires more validations.
Data model impact
-----------------
new data objects:
* hotEnvironment - This parameter will be passed as part of the
/environments/{env-id}/services POST API request body. The value of this
parameter will be an environment file name.
* templateParameters - All heat parameters that were passed in the root of the
/environments/{env-id}/services POST API request body, will be moved under
this property.
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
User will now have the option to add heat environments and additional files to
the package, and have them deployed with the package as part of the Murano
environment. The Heat environment will be deployed when it is requested in the
relevant API, while the additional files will always be deployed.
At this point there will be no change in the python-muranoclient. If the user
will wish to add environments or additional files to a heat generated package.
He can edit the package and continue via API. If the user only added additional
files, he can continue via python-muranoclient/UI as well.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
New features must not break UI. So since it was proposed to move the user
defined parameters from the root of the request body to the property
"templateParameters", the UI must change the way it build the request to add a
package to a Murano environment.
The new feature for sending a hot environment with the template will be exposed
in the UI the following way:
The user will be able to choose an environment file from a drop-down list
during the same wizard actions he uses to enter heat template parameters.
Implementation
==============
new data objects:
* envs
* env_name
envs will be generated when parsing the package, so if there is nothing in the
package, the new data objects will be empty. If an API to list the environments
will be implemented an empty list will return, same as in other Murano APIs.
A new object called envs of type dictionary will be added in HeatStack class
in the init method. It can be referenced from inside HeatStack by using
self._envs before/when sending the create request to Heat. This object will be
initialized to be empty.
A new method called setEnvs will be added in HeatStack class. It will allow to
enter a value into _envs just like setParameters allows to enter values into
_parameters.
env_name will take the value passed by the user in the parameter
heatEnvironment. It will be initialized and passed to class HeatStack the same
way parameters are passed today.
When sending the create stack request to heat the selected environment content
should be passed. If all is configured and passed correctly the environment
file content can be accessed from class HeatStack by using the next command:
self._envs[self._env_name]
A new method called _translate_envs will be add to HotPackage class. it will
get a path to the envs directory and will return a dictionary of environments
locations and files values. It will be in the next format:
environmentRelativePathStartingHeatEnvironmentsDirectory -> stringFileContent
For example if there is an environment with a full path of
/Resources/HeatEnvironments/my_heat_env.yaml and content of:
"heat_template_version: 2013-05-23\n\nparameters:\n" and it
is the only file in the folder, than this will be the dictionary returned:
{"my_heat_env.yaml": "heat_template_version: 2013-05-23\n\nparameters:\n"}
A very similar function was proposed for the Heat files feature. There will be
a reuse of the code there, if it will be implemented first.
Assignee(s)
-----------
Primary assignee:
michal-gershenzon
Other contributors:
noa-koffman
Work Items
----------
* Add support for adding a package to a Murano environment with a Heat
environment specified in the request.
* Add support for Heat environments when deploying a Murano environment. If
a Heat environment is saved in the session for the package, it should be
parsed and send with the template, when deploying a Murano environment.
* make sure that when ui generate a POST request for API:
/environments/{env-id}/services the user defined parameters are located under
templateParameters in the request body.
Dependencies
============
None
Testing
=======
Unit tests should cover API calls changes:
* Test sending a Heat environment when adding a package to a Murano environment
(positive and negative).
* Test that the request for Heat is build correctly with Heat environment
Documentation Impact
====================
None
References
==========
* http://docs.openstack.org/developer/heat/template_guide/environment.html

168
specs/liberty/add-support-for-heat-files.rst
View File

@ -1,168 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===========================================
Add support for heat environments and files
===========================================
https://blueprints.launchpad.net/murano/+spec/add-support-for-heat-environments-and-files
Add support for using Heat additional files, when saving and deploying Heat
templates.
Problem description
===================
Today there is no option to create stacks neither with an environment nor with
nested stacks via Murano. Only basic stacks are supported.
Use cases for deploying a stack with additional files:
* Supporting Heat nested stacks
* Supporting scripts
For more information see references section of this spec
Proposed change
===============
The change described here will focus on supporting additional files. Another
spec will be submitted for Heat environments.
Allowing Heat additional files to be saved and deployed as part of a Murano
Heat applications. Adding additional files to a package should be optional.
Such files should be placed in the package under '/Resources/HeatFiles'. When a
Heat generated package is being deployed, if the package contains Heat files,
they should be sent to Heat together with the template and params. If there are
any Heat additional files located in the package under '/Resources/HeatFiles',
they should all be sent with the template during stack creation in Heat.
There can be more nested directories under '/Resources/HeatFiles', and if they
have files in them, those should be send to heat as well together with their
relative path.
This part does not require any UI nor API changes.
Alternatives
------------
* The package structure can be different.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
User will have the option to add Heat files to the package, and have them
deployed with the package as part of the Murano environment.
At this point there will be no change in the python-muranoclient. If the user
will wish to add files to a heat generated package, he can edit the package and
continue normally from there.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None.
Implementation
==============
new data objects:
* files
This new data object will be generated when parsing the package, so if there
is nothing in the package, the new data objects will be empty.
A new object called files of type dictionary will be added in HeatStack class
in the init method. It can be referenced from inside HeatStack by using
self._files when sending the create request to Heat. This object will be
initialized to be empty.
A new method called setFiles will be added in HeatStack class. It will allow to
enter a value into _files just like setParameters allows to enter values into
_parameters.
A new method called _translate_files will be add to HotPackage class. it will
get a path to the files directory and will return a dictionary of files
locations and files values. It will be in the next format:
fileRelativePathStartingHeatFilesDirectory -> stringFileContent
For example if there is a file with a full path of
/Resources/HeatFiles/my_heat_file.yaml and content of "echo hello world" and it
is the only file in the folder, than this will be the dictionary returned:
{"my_heat_file.yaml": "echo hello world"}
A new parameter of called files will be added to _generate_workflow method that
can be found inside HotPackage class. It will be included in the deploy
variable in the same way the template_parameters is included.
Assignee(s)
-----------
Primary assignee:
michal-gershenzon
Other contributors:
noa-koffman
Work Items
----------
* Add support for Heat additional-files. If such files exist in the package,
they should be parsed and send with the template, when deploying a Murano
environment.
Dependencies
============
None
Testing
=======
Unit tests should cover API calls changes:
* Test that the request for Heat is build correctly with Heat files
Documentation Impact
====================
None
References
==========
* http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#get-file

214
specs/liberty/artifact-repository-support.rst
View File

@ -1,214 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===========================
Artifact Repository Support
===========================
https://blueprints.launchpad.net/murano/+spec/artifact-repository-support
Glance has recently got a new feature - ability to store not only the VM images
but also data assets and their metadata for other OpenStack projects.
This specification defines the usage of this feature in Murano, so Murano may
store its packages in Glance and benefit from all its features.
Problem description
===================
The current implementation of Murano Package Repository as part of murano-api
has the following major drawbacks:
* Code duplication. Lot's of the logic to manage the packages is very similar
for other OpenStack projects. Sharing that code with others will help to
improve the overall quality of OpenStack ecosystem.
* Database storage is a bad idea. Currently we store all the murano-packages as
blob fields in the database tables. This dramatically decreases the
performance and limits the maximum size of the packages, thus preventing us
from putting actual application binaries inside the packages. Using glance
for this purpose will allow us to utilize all the backing stores supported by
glance, including Swift, Ceph, S3 and others.
* No cross-package dependencies. Current implementation of package repository
does not have any concept of cross-package dependency, so MuranoPL package
loaders have to detect package dependencies only in runtime. This leads to
issues when some of the package's dependencies are not present in the repo,
but this cannot be detected until the actual deployment starts. Using glance
which has a notion of cross-artifact dependencies solves this issue natively
at the repository level.
* Image dependencies are manual. Current implementation has some manual way of
managing dependencies of murano packages to glance images. When glance is
user dependencies on Images will have the same nature as cross-package
dependencies.
* No versioning of packages. Current implementation does not allow to attach
semver-based version strings to the packages, and so no filtering is possible
by this info. Glance Artifact Repository provides a rich versioning tools and
allows to filter by specific version spec and range.
* No ability to query packages based on dependency hierarchy. We need a way to
query the repository for the classes which inherit some specific base class.
Glance Artifact Repository provides such queries out of the box.
Proposed change
===============
It is proposed to create a Glance Plugin which will define the artifact type
"Murano Package" with the following type-specific metadata attributes:
* **type** - a string defining the type of the package, may have values of
either `Application` or `Library`. Immutable. Required.
* **author** - a string defining the author of the package. Immutable.
Required.
* **display_name** - a display name for the package to be shown in the catalog.
Mutable. Required.
* **enabled** - a boolean indicating if that package is available for usage.
Proposed temporary, until Glance v3 has a support of "disable" operation for
all the types of artifacts. Mutable, True by default.
* **categories** - a set of strings containing the categories attached to the
package. Mutable.
* **class_definitions** - a set of strings containing the fully-qualified names
of muranoPL classes contained within the package. Immutable.
* **inherits** - a dictionary in which the keys are the interfaces inherited by
the classes of the packages and the values are the list of the names of these
classes. Immutable.
* **keywords** - a set of keywords to simplify the search of the package in the
catalog. Mutable.
..and the following binary objects:
* **logo** - a blob containing the logo of a package
* **archive** - a blob containing the zip archive with the package
* **ui_definition** - a blob containing the yaml-based definition of app's UI
if the one is present.
If worth noting that currently v3 of Glance API is tagged as EXPERIMENTAL, and
both its implementation and interfaces may (and will) change in Mitaka release
cycle. So, some of the plugin implementation details may have to be changed as
well in the next cycle.
Because of this, it is suggested to add this functionality as EXPERIMENTAL as
well, maintaining the old behavior as a preferred alternative, selected by
default.
Alternatives
------------
We may continue using murano-api to manage our packages, at least till the
Glance v3 is STABLE, however this will require us to introduce lots of
temporary code to support things like versioning and dependencies. So, it is
better to add the Glance support as an experimental feature, facade it under
the python-muranoclient bindings, and migrate to it completely in the next
release cycle.
Data model impact
-----------------
No impact. When the migration is complete and Glance is the only storage of
packages, we'll need to drop the old data tables which are currently storing
the old package repositories.
REST API impact
---------------
No impact.
Versioning impact
-----------------
No impact.
Other end user impact
---------------------
No impact.
Deployer impact
---------------
The deployer will have to add the glance plugin package into their environment.
It should be deployed into the same python env with the regular Glance. They'll
have to enable the V3 API in the glance-api config as well.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
Only the configuration changes should be made in murano-dashboard.
Implementation
==============
Assignee(s)
-----------
Primary assignee: ativelkov
Work Items
----------
* Implement a glance plugin
* Copy the artifact-specific access logic from python-glanceclient into the
python-muranoclient, as glance is not going to release v3-aware clients until
the API is stable.
* Implement artifacts adapter in python-muranoclient
Dependencies
============
* Most of the artifacts functionality has been merged to Glance by L-1
milestone. However, several important bugfixes were merged only into the RC1.
So, the latest Glance master is required.
Testing
=======
As the new backend substitutes the old one, the regular testing of
package-related actions should be sufficient.
Documentation Impact
====================
New configuration settings have to be documented.
References
==========
None

139
specs/liberty/bundle-save.rst
View File

@ -1,139 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=================================================================
Download bundle of packages to local directory using muranoclient
=================================================================
https://blueprints.launchpad.net/murano/+spec/bundle-save
The purpose is to add command to muranoclient which allows to download the
bundle from application catalog to local dir.
Problem description
===================
There are cases when there is no Internet access cloud with murano installed.
Then if user wants to add some bundle of packages into murano, he has to
download all of them one-by-one from application catalog using local computer
with access to the Internet. After that he saves them somewhere on data storage
device and moves all files to cloud.
It is necessary to simplify process of saving packages to avoid manual
downloading.
Proposed change
===============
It's proposed to add new CLI command *bundle-save* to murano-client.
Method *do_bundle_save* will corresponds with new command. It will take three
arguments:
* *filename* is a bundle name, bundle url or path to the bundle file;
* *--path* (optional) is a path to directory in which user wants save packages.
If it is not specified, current directory will be used;
* *--no-images* (optional) is flag. If it is set, downloading of all required
images will be skipped.
Method will build whole list of packages and its dependencies. This ability is
already implemented and used in 'bundle-import'. Then method will save bundle
file and each package to specified path. For this, method 'save' will be added
to *FileWrapperMixin* -- the parent class for *Bundle* and *Package* classes.
This method will take one argument *dst* -- destination for file. It will copy
already downloaded file to the specified path.
Method *do_bundle_save* will also save images which packages require.
*save_image_local* method will be used for that. All images will be downloaded
if *--no-images* is not set.
After bundle saving directory with packages can be moved to lab with murano
where all of them can be imported to murano application catalog in one
command.
CLI command *package-save* also must be implemented. It will give to user the
opportunity to download specific package or several packages he need.
The implementation of command will be based on the methods described above.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
User will have access to a new command.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Dmytro Dovbii
Primary assignee:
ddovbii
Work Items
----------
* Add method *save_image_local*
* Add method *save()* to *FileWrapperMixin* class
* Implement CLI command *bundle-save*
* Implement CLI commamd *package-save*
Dependencies
============
None
Testing
=======
Unit tests for CLI client must be updated
Documentation Impact
====================
CLI reference should be updated manually
References
==========
None

158
specs/liberty/change-murano-class-loader.rst
View File

@ -1,158 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=================================
Rework package class loader logic
=================================
https://blueprints.launchpad.net/murano/+spec/change-murano-class-loader
This spec describes rework of murano package class loader as part of the
another blueprint `simulated-execution-mode-murano-engine
<https://blueprints.launchpad.net/murano/+spec/simulated-execution-mode-murano-engine>`_.
The detailed logic regarding class modification would be described in this
specification.
Problem description
===================
Class loader should be able to load packages not only from the external
repository, but from a local directory. It's needed not only for testing new
packages, but for accepting packages on-the-fly.
During the development phase the package gets frequently updated, and the need
to upload it to the repository on each update complicates the development.
Ability to load it from the local filesystem will streamline the process
Another case, when there is no connection to the external repository.
Proposed change
===============
Need to create one more class, responsible for loading packages.
It will look up at the provided directory for a package, that name was requested.
It worth noting, that packages should not be zipped.
Class loader from repository (RCL) and new class loader from local dir (LCL)
will provide the following logic:
* If local dir is not provided, LCL is not operating.
Logic stays same as it's now. RCL do all the work.
* If directory path or several paths to load local packages from are provided,
LCL will check all packages in dir and compare with requested name.
If the desired class is found, package gets loaded. If not - next dir in the
provided list will be scaned.
If is not found in all the provided directories - RCL sends API call to find
it in the repository as usual.
We do have both class loader implemented, we need an ability to combine two
(or more) package loaders in one, with the ability to prioritize the queries.
For example, a ``CombinedPackageLoader`` may be created with an instances of
``DirectoryPackageLoader`` and ``ApiPackageLoader`` underneath. When a
request for a package or class comes in, it is first executed against loader
with the highest priority (say, ``DirectoryPackageLoader``), and if it is not
found there, then goes to the next one.
Alternatives
------------
Use separate class loader in test framework.
But support two different class loaders is not good.
Also as an alternative we can also mention
`this <https://docs.djangoproject.com/en/1.7/ref/settings/#std:setting-TEMPLATE_LOADERS>`_
approach.
That is: introduce package_loaders config parameter. It can be a list of
python-dot-notation class names, that murano would import and try to import pkg
from each loader. This could further be modified to a list of lists, to allow params
like this:
`package_loaders: [pkg_loader1, [pkg_loader2, param1, param2]]`
This would allow to easily add custom loaders without changing murano code itself.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
Enabling loading packages from local directory will be set up in config file.
New key in config file will be added under *[engine]* section and look
like that:
# Directory used as a way to load packages from. (string value)
# local_packages_dir = <None>
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<efedorova>
Other contributors:
<ativelkov>, <slagun>
Work Items
----------
* Implement ``CombinedPackageLoader``:
* Perform testing
Dependencies
============
* Murano Simulation Mode:
https://blueprints.launchpad.net/murano/+spec/simulated-execution-mode-murano-engine
Testing
=======
Unit tests should be added/updated.
Documentation Impact
====================
Opportunity to load apps from local directory should be described in the
documentation.
References
==========
None

396
specs/liberty/cloud-foundry-service-broker-api.rst
View File

@ -1,396 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Implement Cloud Foundry Service Broker API
==========================================
https://blueprints.launchpad.net/murano/+spec/cloudfoundry-api-support
Cloud Foundry is PaaS which supports full lifecycle from initial development,
through all testing stages, to deployment. Most well known Cloud Foundry
flavours is Cloud Foundry OSS, Pivotal Cloud Foundry and Pivotal Web Services.
If we implement Cloud Foundry Service Broker API in murano, murano apps will
be available at Cloud Foundry as services. So Cloud Foundry users will be
granted an ability to operate murano applications.
Problem description
===================
Typical scenario of Cloud Foundry and murano collaboration will look like:
1. While configuring murano enable murano service broker in config file. Provide
host and port for it.
2. Add murano Service Broker in Cloud Foundry and grant access to murano apps for
Cloud Foundry organization.
3. Provision murano app as Cloud Foundry service instance using Cloud Foundry tools.
Proposed change
===============
We need to write Cloud Foundry Service Broker implementation for murano. One of
the parts of this implementation should be some mapping function between Cloud
Foundry and OpenStack resources. Now it's planned to map Cloud Foundry
Organizations to OpenStack tenants and Cloud Foundry spaces to murano
environments. So, all tenant users will be granted with the privileges based on
their existing roles in OpenStack tenant. Each Cloud Foundry space will be linked
wih murano environment. The parameters which is needed to murano for successful
application deployment will store in service object section parameters. The
Service Broker itself will parse them as soon as Cloud Foundry can't do it.
It will be parsed during Provision request. The request body will look like that:
.. code-block:: javascript
{
"service_id": "service-guid-here",
"plan_id": "plan-guid-here",
"organization_guid": "org-guid-here",
"space_guid": "space-guid-here",
"parameters":{
"parameter1": 1,
"parameter2": "value"
}
}
It's planned to setup a Service Broker as a separate service. Additional
options should be added to the configs. Also we want to use Cloud Foundry
experimental asynchronous operations[3].
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
No existing API of murano service is going to be changed. New service will be
created implementing the standard Cloud Foundry Service Broker API. Its API
methods are as follows:
**GET /v2/catalog**
*Request*
+----------+----------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==================================+==================================+
| GET | /v2/catalog | List all available apps |
+----------+----------------------------------+----------------------------------+
*Parameters:*
* None
*Response*
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. The expected resource body as below |
+----------------+-----------------------------------------------------------+
::
{
"services": [{
"id": "service-guid-here",
"name": "mysql",
"description": "A MySQL-compatible relational database",
"bindable": true,
"plans": [{
"id": "plan1-guid-here",
"name": "small",
"description": "A small shared database with 100mb storage quota and 10 connections"
},{
"id": "plan2-guid-here",
"name": "large",
"description": "A large dedicated database with 10GB storage quota, 512MB of RAM, and 100 connections",
"free": false
}],
"dashboard_client": {
"id": "client-id-1",
"secret": "secret-1",
"redirect_uri": "https://dashboard.service.com"
}
}]
}
**PUT /v2/service_instances/:instance_id?accepts_incomplete=true**
*Request*
+----------+-----------------------------------------------------------+-------------------------------------------+
| Method | URI | Description |
+==========+===========================================================+===========================================+
| PUT | /v2/service_instances/:instance_id?accepts_incomplete=true| Create new service resources for developer|
+----------+-----------------------------------------------------------+-------------------------------------------+
::
{
"service_id": "service-guid-here",
"plan_id": "plan-guid-here",
"organization_guid": "org-guid-here",
"space_guid": "space-guid-here"
}
*Response*
+----------------+------------------------------------------------------------+
| Code | Description |
+================+============================================================+
| | OK. May be returned if the service instance already exists |
| 200 | and the requested parameters are identical to the existing |
| | service instance. |
+----------------+------------------------------------------------------------+
| 202 | Accepted. Service instance creation is in progress. |
+----------------+------------------------------------------------------------+
| 409 | Conflict. Should be returned if the requested service |
| | instance already exists. The expected response body is “{}”|
+----------------+------------------------------------------------------------+
| 422 | Should be returned if the request did not include |
| | ?accepts_incomplete=true |
+----------------+------------------------------------------------------------+
::
{
"dashboard_url": "http://example-dashboard.com/9189kdfsk0vfnku"
}
**PATCH /v2/service_instances/:instance_id?accepts_incomplete=true**
*Request*
+----------+-----------------------------------------------------------+----------------------------------+
| Method | URI | Description |
+==========+===========================================================+==================================+
| PATCH | /v2/service_instances/:instance_id?accepts_incomplete=true| Update existing service instance |
+----------+-----------------------------------------------------------+----------------------------------+
::
{
"plan_id": "plan_guid_here"
}
*Response*
+----------------+------------------------------------------------------------+
| Code | Description |
+================+============================================================+
| 200 | Return if only the new plan matches the old one completely |
+----------------+------------------------------------------------------------+
| 202 | Accepted. Service instance update is in progress. |
+----------------+------------------------------------------------------------+
| 422 | Should be returned if the request did not include |
| | ?accepts_incomplete=true |
+----------------+------------------------------------------------------------+
**DELETE /v2/service_instances/:instance_id?accepts_incomplete=true**
+----------+-----------------------------------------------------------+-----------------------------------+
| Method | URI | Description |
+==========+===========================================================+===================================+
| DELETE | /v2/service_instances/:instance_id?accepts_incomplete=true| Delete all resources create during|
| | | the provision. |
+----------+-----------------------------------------------------------+-----------------------------------+
*Response*
+----------+--------------------------------------------------+
| Code | Description |
+==========+==================================================+
| 202 | Accepted. Service instance deletion in progress. |
+----------+--------------------------------------------------+
| 410 | Returned if service does not exist |
+----------+--------------------------------------------------+
| 422 | Should be returned if the request did not include|
| | ?accepts_incomplete=true |
+----------+--------------------------------------------------+
**PUT /v2/service_instances/:instance_id/service_bindings/:binding_id**
*Request*
+----------+----------------------------------------------------------------+----------------------------------+
| Method | URI | Description |
+==========+================================================================+==================================+
| PUT | /v2/service_instances/:instance_id/service_bindings/:binding_id| Bind service |
+----------+----------------------------------------------------------------+----------------------------------+
::
{
"plan_id": "plan_guid_here",
"service_id": "service_guid_here",
"app_guid": "app_guid_here"
}
*Response*
+----------------+------------------------------------------------------------------+
| Code | Description |
+================+==================================================================+
| 201 | Binding has been created. The expected response body is below. |
+----------------+------------------------------------------------------------------+
| 200 | May be returned if the binding already exists and the requested |
| | parameters are identical to the existing binding. The expected |
| | response body is below. |
+----------------+------------------------------------------------------------------+
| 409 | Should be returned if the requested binding already exists. The |
| | expected response. body is `{}`, though the description field can|
| | be used to return a user-factorin error message. |
+----------------+------------------------------------------------------------------+
::
{
"credentials": {
"uri": "mysql://mysqluser:pass@mysqlhost:3306/dbname",
"username": "mysqluser",
"password": "pass",
"host": "mysqlhost",
"port": 3306,
"database": "dbname"
}
}
**DELETE /v2/service_instances/:instance_id/service_bindings/:binding_id**
*Request*
+----------+----------------------------------------------------------------+----------------------------------+
| Method | URI | Description |
+==========+================================================================+==================================+
| DELETE | /v2/service_instances/:instance_id/service_bindings/:binding_id| Unbind service |
+----------+----------------------------------------------------------------+----------------------------------+
*Response*
+----------+-----------------------------------+
| Code | Description |
+==========+===================================+
| 200 | Binding was deleted |
+----------+-----------------------------------+
| 410 | Returned if binding does not exist|
+----------+-----------------------------------+
**GET /v2/service_instances/:instance_id/last_operation**
*Request*
+----------+--------------------------------------------------+-------------------------------+
| Method | URI | Description |
+==========+==================================================+===============================+
| GET | /v2/service_instances/:instance_id/last_operation| Polling status of the last 202|
| | | operation |
+----------+--------------------------------------------------+-------------------------------+
*Response*
+----------+--------------------------------------------------------+
| Code | Description |
+==========+========================================================+
| 200 | OK |
+----------+--------------------------------------------------------+
| 410 | GONE. Appropriate only for asynchronous delete requests|
| | Cloud Foundry will consider this response a success and|
| | remove the resource from its database. |
+----------+--------------------------------------------------------+
::
{
"state": "in progress",
"description": "Creating service (10% complete)."
}
Versioning impact
-------------------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
Service Broker should be deployed and enabled in the murano config.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
starodubcevna
Work Items
----------
Changes can be split to this parts:
* Implement the stub of Service Broker itself. Add needed config opts and starting point.
* Implement basic Cloud Foundry API calls such as list and provision. Also on this step we
should add murano specific API calls.
* Series of extensions for Cloud Foundry API support:
* Add update and deprovision API calls
* Add bind/unbind API calls
Dependencies
============
None
Testing
=======
Unit tests should cover new API calls.
Documentation Impact
====================
Document "Murano and Cloud Foundry HowTo". It should be step by step guide for
Cloud Foundry and murano cooperation.
References
==========
[1] https://youtu.be/ezq9P1WN2LY
[2] http://docs.cloudfoundry.org/services/api.html
[3] https://docs.cloudfoundry.org/services/asynchronous-operations.html

146
specs/liberty/dynamic-ui-specify-no-explicit-name-field.rst
View File

@ -1,146 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
============================================================
Remove name field from fields and object model in dynamic UI
============================================================
https://blueprints.launchpad.net/murano/+spec/dynamic-ui-specify-no-explicit-name-field
Now name is a required parameter in every single form definition.
But murano-engine doesn't know anything about this parameter. It mostly used
for dymanic UI purposes. So we can insert this field automatically in every
form.
Problem description
===================
* 'name' property has no built-in predefined meaning for MuranoPL classes
or applications.
Now all unknown parameters are ignored, but the error will be spawned and
all applications will be invalid in the near future.
To prevent global failure this change is suggested.
Proposed change
===============
Add automatic field inserting into the first form and store 'name' field value
in object header:
* Dynamic UI version will be increased to 2.1;
* 'name' property is not required in MuranoPL class definition anymore.
If user still have 'name' property in class definition, he should supply
corresponding field in UI definition. It will have no special meaning for
murano dashboard.
* New field will be inserted to the first form if Dymanic UI version is higher
or equal to 2.1;
* If Dymanic UI version is higher or equal to 2.1 'name' field value will be
placed to the object header ('?'), in "done" method of application creation
wizard;
* In get-mode, application 'name' parameter will be checked in:
* Primary under '?' parameter;
* Secondary in the object model root;
* Add new YAQL function to dashboard's YAQL to be used by dynamic UI.
The function will be called 'name' and will allow to use
automatically-inserted name field in object model description.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
This change introduces new version of dynamic UI form definition.
Only backward compatibility will be supported: new dashboard can work with old engine.
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
This spec is supposed to be implemented in murano-dashboard only.
User will see new field for adding app name.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<efedorova@mirantis.com>
Work Items
----------
* Implement automatic 'name' field insertion if version satisfies the requirements;
* Put 'name' field value to the object header area;
* Use 'name' attribute from object header or inside object modal root
in environments table;
* Implement new YAQL function, which returns applications name.
Dependencies
============
This change is a dependency for a future changes in engine
Testing
=======
CI tests should be updated and catch all errors.
Documentation Impact
====================
Dynamic UI form definition examples should be updated
References
==========
None

237
specs/liberty/env-configuration-from-cli.rst
View File

@ -1,237 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===============================
Configure environments from CLI
===============================
Blueprint for this specification:
https://blueprints.launchpad.net/python-muranoclient/+spec/env-configuration-from-cli
Currently there is no way to configure and/or deploy a murano environment from
the command line. This specification describes how this issue can be addressed
and what steps have to be taken in order to implement this abilities.
Problem description
===================
Currently the only possible way to deploy a murano environment is through
interaction with horizon. CLI client currently lacks commands to add apps to
environment or deploy an environment. This makes murano useless without horizon
and murano-dashboard.
It also means, that scripting and/or automating deployment of murano
apps/environments is currently problematic to say the least.
Proposed change
===============
This spec proposes to add several new CLI commands and modify existing to
allow user to add apps to the env, configure it and send it to deploy.
Command ``environment-create`` currently only supports adding the name of the
environment, while the api call supports setting aditional params, such as
defaultNetworks. ``environment-create`` should be updated to allow setting
custom keys to environment model.
``environment-edit`` command should perform all the editing (adding, deleting,
modifying of existing apps)
This command should be non-interctive, adding an interactive mode is a
good idea as of itself, but is out of scope of this spec.
``environment-edit`` would use jsonpatch format for input (as described by
RFC 6902). This is a well known and powerfull instrument to manipulate json,
and object model is currently stored in json.
This would allow easy addition, modification and deletion of apps.
Editing of environment attributes is not allowed by API, so it's beyond the
scope of this spec.
During command execution, env configuration parameters should be read from
stdin or a file.
Supporting stdin as input source means it should be possible to issue a
command like ``murano environment-edit env_id < app_patch.json``
An example ``app_patch.json`` might look like::
[
{ "op": "add", "path": "/services/-",
"value":
{'?':
{'id': '2fc3005d-b4f8-420d-9e15-f961b41f49ee',
'type': 'io.murano.apps.App1'
},
'instance':
{'?': {'id': '53366263-04e9-49d1-829d-50e27158749c',
'type': 'io.murano.resources.LinuxMuranoInstance'},
'assignFloatingIp': True,
'availabilityZone': 'nova',
'flavor': 'm1.medium',
'image': 'afc1aa61-f623-4c66-bbd8-5359261c5272',
'name': 'ilphvibwqyu6h1' },
'name': 'App1'
}
},
]
All operations would only apply to services field, so the `/services/` part of
the path can be ommitted.
note::
This means, that a user has to generate unique ids. This can be a difficult
task and may hinder scripting capabilities. We might think of a way to
mitigate this issue, by introducing a way to generate ids. This can be done
by introducing an optional parameter ``--id-format``. It would accept a
template for strings to be replaced with generated ids. For example:
``--id-format=##id##`` would replace all occurrences of ##id##-1 with fist
generated id, all occurrences of ##id##-2 with second generated id and all
occurrences of ##id## with a unique id. uuids should be used for id
generation.
Command should perform validation of its input. This can be done using
MuranoPL classes and property contracts.
If an error happens during non-interactive mode (for example validation fails
for some field) ``murano`` should return a non-zero status, to facilitate
scipting and error-detecting.
Command ``environment-edit`` should support optional ``--session`` parameter,
that allows user to specify a session for editing an environment.
It should also
warn user in case more than one session is open. If the session parameter is
omitted: last open session for given environment is used.
If there are no open sessions — a new session would be opened.
note::
This means that to deploy a simple application one has to know jsonpatch
syntax. We might think about optional `syntax sugar` commands, that would
allow adding a single app to the environment. These would use predefined
jsonpatch patch documents and would accept only value to be added to
/Objects/services/- path.
We should implement session controlling commands during
implementation of this spec, for example:
#. ``session-list`` Show list of open sessions for a given environment.
#. ``session-show`` Show details of currently configured session..
#. ``session-delete`` Discard an open session for a given environment.
In case an error occurs during environment deployment this would allow to
rollback the changes to the previous version of environment.
This should also allow to handle cases, when a single environment is edited
simultaneously by different users.
``environment-action`` command should be implemented to allow performing
actions against an environment. One example of actions is a deploy action, so
``murano environment-action deploy`` should deploy the environment.
Alternatives
------------
Alternatively we can implement an interactive mode, that would mimic current
dashboard behaviour. This would not allow scripting, but would allow us to
build a more user-friendly CLI interface. The dowside is that current UI
definitions are scheduled to be changed and would most likely be replaced in
the forseable future. This means, that most of the work would go to waste.
Data model impact
-----------------
None
REST API impact
---------------
None, CLI should reuse API calls, already used by dashboard
Versioning impact
-------------------------
Since we're adding functionality — None
Other end user impact
---------------------
None
Deployer impact
---------------
It is possible that implementation of this spec would require setting and
reading intermediate environment variables to work correctly with sessions,
during app addition, env deployment.
Overall deployment of murano would not be affected.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<kzaitsev>
Work Items
----------
#. ``environment-edit`` command
(should create session, check permissions, add
patch resulting model into ).
#. Input validation for ``envieonment-edit``
(request packages from API, check ID
uniqueness, check input field adequacy).
#. (optional) ``--format-id`` option support
#. (optional) syntax sugar command, that would allow easy addition of a package
to an environment.
#. Session controlling commands.
#. ``environment-action`` command.
#. Shell unit tests.
#. Integration tests.
Dependencies
============
None
Testing
=======
We shall need Unit tests for new commands introduced.
Also, since this change introduces a way to deploy an env from CLI. This means
that integration tests for murano client should be implemented. A typical test
should upload and app, configure a simple environment with 1-2 apps and set
some custom parameters, like access port and optionally deploy the env in
question. This tests should probably take place in murano-ci.
Documentation Impact
====================
New python-muranoclient commands would have to get a proper documentation. It's
also possible, that we would want to document the whole process of deploying an
app or scripting of such a deployment as a separate article in murano
documentattion.
References
==========
* http://jsonpatch.com
* https://tools.ietf.org/html/rfc6902
* https://tools.ietf.org/html/rfc7159
* https://pypi.org/project/jsonpatch

180
specs/liberty/environment-abandon.rst
View File

@ -1,180 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==============================
Environment abandoning support
==============================
https://blueprints.launchpad.net/murano/+spec/environment-abandon
The purpose is to add an ability to abandon the environment, which hangs during
deployment or deleting in order to hide it from the list of other environments.
Problem description
===================
There are cases when the process of deleting or deploying environment may hang.
Then the user has to release all used resources and manually clean the database
in order to get rid of the list of failed environments.
Manual cleaning of the database is unsafe and inconvenient operation. That's
why `abandon` feature has to be provided. It should improve usability and
allows to user to avoid direct database editing. But it is necessary to notify
the user that all resources used by abandoned environment must be also released
manually as previously.
Proposed change
===============
The implementation of this feature consists of three stages:
1) Modify environment-delete API endopoint to have *abandon* feature
The method *delete* must be changed. Depending on the parameter "abandon",
obtained from request data, the method should work differently. This parameter
has a boolean type. If it equals *True* enviroment will be directly removed
from the database without object model cleanup.
2) Provide corresponding changes in the python-muranoclient
Method *delete* of environment manager should be modified. New boolean
parameter *abandon* with default value *False* should be added. The value of
parameter affects the building of url, which is sent to murano-api.
3) Add new button *Abandon* to murano-dashboard
This button should be available with any environment state.
Proposed change doesn't solve problem of deployment process hanging.
Murano-engine may continue to deploy abandoned environment. It is
necessary to find a way how to stop murano-engine in this case.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
**DELETE /environments/<environment_id>?abandon**
*Request*
+----------+----------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==================================+==================================+
| DELETE | /environments/{id}?abandon | Remove specified environment. |
+----------+----------------------------------+----------------------------------+
*Parameters:*
* `abandon` - boolean, indicates how to delete environment. *False* is used if
all resources used by environment must be destroyed; *True* is used when just
database must be cleaned
*Response*
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Environment deleted successfully |
+----------------+-----------------------------------------------------------+
| 403 | User is not allowed to delete this resource |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified environment doesn`t exist |
+----------------+-----------------------------------------------------------+
Versioning impact
-------------------------
Murano dashboard will support only the version of the client, that includes
corresponding changes.
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
New action `Abandon` will be added to murano-dashboard. It will be always
available in the row of other action.
Dialog with warning should appear when user executes action
Implementation
==============
Assignee(s)
-----------
Dmytro Dovbii
Primary assignee:
<ddovbii@mirantis.com>
Work Items
----------
* Modify method 'delete' of environment API to support two delete modes
* Implement adding 'abandon' parameter to url in 'delete' method of environment
manager in muranoclient
* Add flag '--abandon' to CLI command 'environment-delete'
* Add new class 'AbandonEnvironment' which provide new button 'Abandon' in
murano-dashboard
Implementation is acutally completed.
Dependencies
============
None
Testing
=======
Functional tests for murano-dashboard must be updated.
Unit tests should cover API call and CLI client
Tempest tests are out of the scope of this spec.
Documentation Impact
====================
API specification should be updated
References
==========
None

172
specs/liberty/logging-api-for-muranopl.rst
View File

@ -1,172 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
========================
Logging API for MuranoPL
========================
https://blueprints.launchpad.net/murano/+spec/logging-api-for-muranopl
The purpose is to add an ability to log actions while developing MuranoPL
applications
Problem description
===================
It is good practice to log basic stages during application execution.
Logging API should make debugging and troubleshooting processes easier.
Proposed change
===============
Implemention key points
1) New MuranoPL class `io.murano.system.Logger`
Class `Logger` will be part of the MuranoPL core library.
This class should contain basic logging functionality.
Usage example in MuranoPL::
$.log: logger('logger_name')
$.log.info('message: {0}', 'checkpoint')
that code should be equivalent of python code::
from oslo_log import log as logging
LOG = logging.getLogger('logger_name')
LOG.info(_LI('message: {0}').format('checkpoint'))
Others logging methods are `Logger.debug`, `Logger.info`, `Logger.warning`,
`Logger.error`, `Logger.critical`. Each method corresponds to the oslo.log
logging level.
There is also separate method for exception stack trace output described
below.
2) Exceptions logging
Method `Logger.exception` intended to log a stack trace::
$.log.exception(exc, 'Something bad happened: {0}', 'Oops!')
`exception` method uses the same log level as `Logger.error`.
3) New global MuranoPL function `logger()`
Call of the function `logger('logger_name')` returns new instance of the
`io.murano.system.Logger`. If function with the same `logger_name` was
called previously then the same logger instance should be returned instead of
building new one.
4) Logging configuration
Logging should use standard Murano Engine logging subsystem.
Application itself cannot set logging settings at runtime.
All of appenders, formatters an etc. may be configured via standard way
as others loggers in Murano Engine.
Prefix `applications` should be added to each logger name which created by
application. As example, logger named `active_directory` in the MuranoPL
should be identified as `applications.active_directory` at the python side
and in the system config. This is for separation loggers used by
applications and engine system loggers. Also it will allow us to specify
settings for the application loggers separately from others loggers.
5) Default configuration
All logs written in the one file by default. Log rotation should be used,
so maximum size of logs is limited.
6) Logging naming conventions
A note about the logging naming conventions should be added to the MuranoPL
tutorial.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
Deployer will get more information about application execution stages and additional
tool for more precise troubleshooting
Developer impact
----------------
Logging API will allow developer to debug application in a more effective manner
getting information from logs.
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Alexey Khivin
Primary assignee:
<akhivin@mirantis.com>
Work Items
----------
* Create new MuranoPL class `io.murano.system.Logger`
* Create new global MuranoPL function `logger()`
* Create method `Logger.exception`
* Add new section for logging parameters into the Murano Engine config
* Describe naming conventions for loggers in the Murano docs
Dependencies
============
None
Testing
=======
Functional tests for MuranoPL must be updated.
Documentation Impact
====================
MuranoPL
References
==========
None

239
specs/liberty/migrate-to-yaql-vnext.rst
View File

@ -1,239 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=====================
Migration to yaql 1.0
=====================
https://blueprints.launchpad.net/murano/+spec/migrate-to-yaql-vnext
yaql 1.0 has a long list of improvements over yaql 0.2 used by Murano.
But it is not a drop-in replacement for yaql 0.2 and requires to perform
serious changes to Murano in order to switch to it.
-------------------
Problem description
-------------------
While yaql 1.0 is more or less similar in terms of query language to that in
v0.2 it is very different in how its functionality exposed to a hosting
project. Most notable differences are:
* syntax for Python function declaration for yaql is different from that in
yaql 0.2.
* yaql 0.2 had so called #resolve() function. When it encountered function
call expression (say ``foo()``) it tried to call ``#resolve()`` so that it
will do the function resolution. If #resolve was not present in context of
failed to resolve function standard yaql 0.2 resolution procedure was used.
Murano used #resolve to catch attempts to call ``$obj.something()`` where obj
is a MuranoObject because it is impossible to register all possible methods
of all possible objects in advance. yaql 1.0 doesn't have neither
``#resolve()`` nor anything similar to it.
* yaql 1.0 doesn't have tuples syntax any more. In v0.2 ``foo(a => b)`` meant
``foo(tuple(a, b))``. In v1.0 ``=>`` syntax means keyword argument value.
Murano used to convert tuples to keyword arguments when calling MuranoPL
methods.
* in yaql 0.2 parser was a global object so that you could say yaql.parse().
In v1.0 parser itself is customizable and need to be constructed by the
hosting project so you need a parser instance (called YaqlEngine in v1.0)
to parse expressions.
* yaql 1.0 doesn't have ``#validate()`` function used to implement ``:`` in
ns:TypeName syntax. yaql 1.0 also doesn't have ':' operator but has a way
to define custom operators that can be used instead.
* there is a great overlap between MuranoPL and yaql 1.0. yaql have a rich
and extensible framework to decorate function parameters to specify their
type and requirements, to inject hidden parameters and to control parameter
laziness. Not only it used for validation but also for function overload
resolution. In MuranoPL there are argument contracts serving similar purpose.
However their implementation is completely independent from each other as for
the whole function invocation process. As a result for MuranoPL methods that
are writen in Python (system classes) it is impossible neither contracts nor
yaql specs, have additional injected parameters and so on.
---------------
Proposed change
---------------
In Murano Engine:
=================
* Create and store in a global variable YaqlEngine instance that will be used
through the application to parse expressions. Update all the places where
parsing takes place to use that object
* extend list of yaql 1.0 parameters specifications (smart-types) with
additional that will validate parameter value using contract expression.
* generate yaql 1.0 FunctionDefinitions from MuranoPL yaql methods using
introduced smart-type. As a result all MuranoPL methods will be directly
exposed as yaql functions. Actual function payload will still point to
MuranoDslExecutor code to perform correct context switching, locking and
logging
* do similar thing for native MuranoPL methods (those that are written in
Python). Python methods could use any yaql parameter declarations and full
potential of yaql 1.0 specs. So now yaql functions, Python methods and YAML
methods will all be unified as yaql functions/methods with full
interoperability. For native methods there will be 2 FunctionDefinition:
one is the interface stub that gets registered in yaql context and calls
executor.invoke_method() under the hood. And another one for a native method
itself to call methods via yaql mechanics (and apply all the smart-type
specs) rather that via custom reflection-based implementation that is found
in dsl code. For YAML methods method.body will still remain the dictionary
as it is now
* implement ``#operator_.`` (``.`` operator) in such a way that for expressions
of a form ``$obj.foo()`` where ``$obj`` is a MuranoObject it will discover
all possible methods of ``$obj``'s type (via added MuranoClass functionality)
and continue with ``foo(sender=$obj)`` execution in a child context with all
``$obj``'s method registered in it. This is a way to dynamically register
functions in context overcoming absence of yaql 0.2's ``#resolve()``
* configure yaql engine to have additional operator ``:`` with a highest
priority. Expressions of a form ns:Type should be resolved in place using
current Namespaces specification and resulting type name will be used to
obtain MuranoClass from class loader. As a result of operation return
special object (MuranoClassReference) that will wrap MuranoClass and could
be used to reference type in YAML MuranoPL code without providing access
to MuranoClass internals. That can be used later to implement static methods
with ``ns:Type.foo()`` syntax
* add one more yaql smart-type so that it will be possible to declare that
function requires type reference. This smart type should accept both strings
and MuranoClassReference objects. new() and class() functions should be
implemented with this smart-type. This will allow transparency between
string-based syntax new('io.murano.MyClass') and new(ns:MyClass) since
ns:MyClass will not be a string anymore
* change dsl's and engine's yaql functions to have new declaration syntax.
Remove functions that are already present in yaql 1.0 standard library.
* use yaql 1.0 legacy mode for backward compatibility with existing
MuranoPL applications
In python-muranoclient:
=======================
* create global YaqlEngine and update code to use it (and new yaql module
structure)
* update YaqlExpression code to use that global parser object and new yaql's
module structure
In Murano Dashboard:
====================
* make dashboard either use parser object from updated python-muranoclient or
create one for itself
* update yaql functions that were written for dynamicUI forms to have new
declaration syntax
* because dashboard uses expression serialization (pickling) and yaql 1.0
expressions are not serializable because of a outer AST expression contains
reference to YaqlEngine used to create it and YaqlEngine cannot be made
serializable there is a need for custom pickling code that will make pickle
not to try to serialize YaqlEngine instance
* use yaql 1.0 legacy mode for backward compatibility with existing
MuranoPL applications. In the future we can switch to non-legacy mode on
newer form format versions.
------------
Alternatives
------------
Try to fix some of the bugs in yaql 0.2 and not to switch to v1.0
Data model impact
=================
None
REST API impact
===============
None
Versioning impact
=================
yaql migration should not break existing applications so there is no versioning
impact here. However it may have an impact on how plugins/system classes for
MuranoPL are written. But since versioning is not implemented yet there is
nothing to increment and backward compatibility will be broken for those
classes.
Also this yaql legacy mode may be switched off for next engine version and
still be turned on for old applications
Other end user impact
=====================
Because new yaql is not 100% compatible with yaql 0.2 there is still a
possibility that some applications will break even with compatibility mode
turned on. For example there might be bugs in applications that remained
unnoticed with yaql 0.2 but cause exception to be raised with v1.0 or
application might have relied on yaql 0.2 buggy behavior that is not present
anymore.
Deployer impact
===============
None
Developer impact
================
Plugins will have to be updated to use yaql-style method declarations. But
because yaql can infer many things automatically changes are going to be
minor and will not require much efforts to do. However without any
modifications old plugins will likely to break after migration will be made.
Murano-dashboard / Horizon impact
=================================
dashboard code need to be updated to use new yaql. This will not affect user
experience
--------------
Implementation
--------------
Assignee(s)
===========
Primary assignee:
Stan Lagun <slagun>
Other contributors:
Ekaterina Chernova <efedorova>
Work Items
==========
Bullets from "proposed changes" section may be used as a work items directly
------------
Dependencies
------------
This spec depends on yaql 1.0 be released and the latest changes in it.
-------
Testing
-------
Because applications can break they should be re-tested. When incompatibility
found it is better to try to fix it in engine's code rather that in application
to capture similar cases in other applications.
--------------------
Documentation Impact
--------------------
Documentation for how to write plugins/system classes need to be updated.
Most of the newly introduced features to MuranoPL are direct consequence
of new yaql features and should be documented in yaql's scope rather than
copying it to MuranoPL documentation.
----------
References
----------
None

219
specs/liberty/murano-api-all-tenants-search.rst
View File

@ -1,219 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===============================
Murano API - All Tenants Search
===============================
https://blueprints.launchpad.net/murano/+spec/murano-api-all-tenants-search
Congress Murano datasource driver pulls environments from one tenant only. The goal
is to pull all environments from all tenants (as nova driver does for servers).
Problem description
===================
Murano - Congress integration is part of a part of Policy Guided Fulfillment.
It uses Congress policy framework to define and evaluate restrictions on Murano
environments. So Murano environments are pulled by Congress Murano datasource driver,
so Congress policy rules can be evaluated.
The problem is that Murano REST API returns environments of one tenant of authenticated user's token only.
Thus Congress policy rules evaluation is run on data from one tenant only.
Other Congress datasource drivers are dealing with similar requirements also - for example
Nova datasource driver pulls data about all servers across all tenants
in its *nova* policy. It is possible because Nova REST API supports search option *all_tenants*.
Note that *Congress policy* is a place of both rules and data related to one *service*.
If *policy* is defined by datasource driver, then its configuration have *user*, *password* and *tenant*,
which are used to get token to access the service.
Proposed change
===============
Search option *all_tenants* will be added to operation *List Environments* of
Murano REST API. When set, the returned list will contain all environments accessible
by the user (specified by token) regardless of tenant.
Listing environments from all tenants can only admin user.
Alternatives
------------
The requested behavior can be also achieved by iterating operation
*List Environment* over all tenants available to the configuration user.
This solution has following performance issues:
* each pull cycle executes the REST operation for every tenant where user is member,
instead of one execution in case of *all_tenants*
* user's tenant assignment has to be periodically updated, so it leads to another
requests to keystone each such period
Data model impact
-----------------
None
REST API impact
---------------
* List Environments
* *all_tenants* parameter is added. When set to *true*, then search over all tenants
is executed, otherwise search on token's tenant is done
Example (without *all_tenants*):
::
GET http://<server-name>:8082/v1/environments
{
"environments": [
{
"status": "deploying",
"updated": "2015-05-06T08:14:06",
"networking": {},
"name": "test",
"created": "2015-05-06T08:08:40",
"tenant_id": "cd9e218f9b894ebdb421e9906fbec15e",
"version": 1,
"id": "8cc3187c763f4ca9bc58cdaf89f926d3"
}
]
}
Example (with *all_tenants* - note different *tenant_id*):
::
GET http://<server-name>:8082/v1/environments?all_tenants=true
{
"environments": [
{
"status": "deploying",
"updated": "2015-05-06T08:14:06",
"networking": {},
"name": "test",
"created": "2015-05-06T08:08:40",
"tenant_id": "cd9e218f9b894ebdb421e9906fbec15e",
"version": 1,
"id": "8cc3187c763f4ca9bc58cdaf89f926d3"
},
{
"status": "deploying",
"updated": "2015-05-08T09:34:16",
"networking": {},
"name": "test 2",
"created": "2015-05-08T08:18:20",
"tenant_id": "8908989abbeec239023489023ccc1234f",
"version": 1,
"id": "abecbf88328932bbecbefe82348238b"
}
]
}
Versioning impact
-------------------------
None
Other end user impact
---------------------
*python-muranoclient* will be changed as follows:
* *--all-tenants* on CLI
Example:
::
$ murano environment-list --all-tenants
* *search options* will be supported on API level
Example:
::
class EnvironmentManager(base.ManagerWithFind):
def list(self):
...
def list(self, search_opts):
...
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
filip-blaha
Work Items
----------
* Introduce *all_tenants* search option in
* file *murano/api/v1/environments.py*
* Modify *policy.json* file with rules
* file *etc/murano/policy.json*
* Add support for search options in *python-muranoclient*
* file *muranoclient/v1/environments.py*
* Add support for *--all-tenants* in *python-muranoclient* CLI
* file *muranoclient/shell.py*
Dependencies
============
None
Testing
=======
Unit tests should cover server API side also client and shell should be covered.
Documentation Impact
====================
REST API documentation will be modified to mention *all_tenants* search option.
References
==========
* https://wiki.openstack.org/wiki/PolicyGuidedFulfillmentLibertyPlanning
* https://wiki.openstack.org/wiki/PolicyGuidedFulfillmentLibertyPlanning_MuranoAPI

171
specs/liberty/murano-core-model-integration-improvements.rst
View File

@ -1,171 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=========================================================
Murano API - Core Model Component Integration Improvement
=========================================================
https://blueprints.launchpad.net/murano/+spec/murano-core-model-integration-improvements
Core Model can be seen as API, because user is using it when writing Datalog queries
in Congress, or integrating with Mistral workflows. Current Murano Core model does not
provide means to easy link them with realized OpenStack entities (for example Murano
Instance does not provide UUID of provisioned Nova Server).
Problem description
===================
Congress datalog queries are one of core features used by Policy Guided Fulfillment.
These queries are used to express validity of Murano environment either
in pre-deployment and/or runtime. In order to evaluate environment validity it is
necessary to work with realized entities by core Murano components - for example
* *I want to check if Murano Instance's Nova server exists and is running*
* In kilo I have to use IP address of the Murano Instance and do multiple joins of
neutron tables to identify Nova server.
* *I want to check if owner of network used by Murano Environment is from given group of
users*
* In kilo it is impossible because Murano network object contains only name
of the network (*a-network*), while realized network (via Heat) contains name
with Murano object id (*a-network-bed7a70ed791434c8acdd53a52a8d4ca*)
Proposed change
===============
Changes of core Murano model:
* **io.murano.resources.Instance**
* add property *openstackId* and fill the property with value once Heat provisioned
Instance
::
openstackId:
Contract: $.string()
Usage: Out
* **io.murano.resources.Network**
* add property *openstackId* and fill the property with value once Heat provisioned
network (as part of Instance provisioning)
::
openstackId:
Contract: $.string()
Usage: Out
Alternatives
------------
Instead of adding the same property to each class aware of openstack ID we can create
mixin class ( e.g. **OpenstackIdMixin**) with this property and all classes aware of
openstack ID will extend that mixin.
::
Name: OpenstackIdMixin
Properties:
openstackId:
Contract: $.string()
Usage: Out
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
Murano package versioning is currently analyzed and it is planned for Liberty.
It makes sense to introduce it (modifications of the core Murano packages) as new
versions of core Murano packages.
On the other hand, proposed changes are backward compatible, so they can be
done prior versioning.
Other end user impact
---------------------
None
Deployer impact
---------------
If proposed changes will be done prior Murano package versioning, then after upgrade
the Murano objects won't have initialized introduced properties (*openstackId*).
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
The change will simplify implementation of Horizon UI (instance detail)
Implementation
==============
Assignee(s)
-----------
Primary assignee:
filip-blaha
Work Items
----------
* introduce *openstackId* properties to
* *meta/io.murano/Classes/resources/Instance.yaml*
* *meta/io.murano/Classes/resources/Network.yaml*
* implemented instance and network *openstackId* property population
* *meta/io.murano/Classes/resources/Instance.yaml* , *deploy* method
* *meta/io.murano/Classes/resources/NeutronNetwork.yaml* , *deploy* method
* *meta/io.murano/Classes/resources/NovaNetwork.yaml* won't be modified, as nova
networking is deprecated
Dependencies
============
https://blueprints.launchpad.net/murano/+spec/murano-versioning
Testing
=======
Both unit and tempest tests of policy guided fulfillment will be enhanced to test properties *openstackId*.
Documentation Impact
====================
None
References
==========
* https://wiki.openstack.org/wiki/PolicyGuidedFulfillmentLibertyPlanning
* https://wiki.openstack.org/wiki/PolicyGuidedFulfillmentLibertyPlanning_MuranoAPI

168
specs/liberty/murano-log-guildelines.rst
View File

@ -1,168 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
======================
Murano unified logging
======================
https://blueprints.launchpad.net/murano/+spec/unified-style-logging
Rewrite murano logging in unified OpenStack style proposed by
https://blueprints.launchpad.net/nova/+spec/log-guidelines
Problem description
===================
Now log levels and messages in murano are mixed and don't match the OpenStack
logging guideliness.
Proposed change
===============
The good way to unify our log system would be to follow the major guidelines.
Here is a brief description of log levels:
* Debug: Shows everything and is likely not suitable for normal production
operation due to the sheer size of logs generated (e.g. scripts executions,
process execution, etc.).
* Info: Usually indicates successful service start/stop, versions and such
non-error related data. This should include largely positive units of work
that are accomplished (e.g. service setup, environment create, successful
application deployment).
* Warning: Indicates that there might be a systemic issue;
potential predictive failure notice (e.g. package execution problems,
problems with categories listing).
* Error: An error has occurred and an administrator should research the event
(e.g. deployment failed, app add failed).
* Critical: An error has occurred and the system might be unstable, anything
that eliminates part of murano's intended functionality; immediately get
administrator assistance (e.g. failed to access keystone/database, plugin
load failed).
As far as murano-dashboard has it own notification system all notifications
should be duplicated at log messages and should follow this spec in the selection
of log level.
Here are examples of log levels depending on environment execution:
* Action execution:
.. code:: python
LOG.debug('Action:Execute <ActionId: {0}>'.format(action_id))
..
* Environment creation:
.. code:: python
LOG.info(_LI('Environments:Create {id} succeed>'.format(id=environment.id)))
..
* Package execution problems
.. code:: python
LOG.warning(_LW("Class is defined in multiple packages!"))
..
* Environment is not found
.. code:: python
LOG.error(_LE('Environment {id} is not found').format(id=environment_id))
..
Additional step for our logging system should be usage of pep3101 as unified
format for all our logging messages. As soon as we try to make our code more
readable please use {<smthg>} instead of {0} in log messages.
Alternatives
------------
We need to follow OpenStack guidelines, but if needed we can move plugin logs
to DEBUG level instead of INFO. It should be discussed separately in each case.
Data model impact
-----------------
None
REST API impact
---------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
murano-image-elements impact
----------------------------
None
murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
starodubcevna
Work Items
----------
* Unify existing logging system
* Unify logging messages
* Synchronize dasboard notifications and log entries
* Add additional logs if needed
Dependencies
============
None
Testing
=======
None
Documentation Impact
====================
None
References
==========
https://blueprints.launchpad.net/nova/+spec/log-guidelines
https://www.python.org/dev/peps/pep-3101/

410
specs/liberty/murano-versioning.rst
View File

@ -1,410 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=================
Murano versioning
=================
https://blueprints.launchpad.net/murano/+spec/murano-versioning
This specification proposes set of arrangements that are going to allow Murano
keep backward compatibility with applications written for older version while
still making possible to introduce incompatible changes to any part of the
project.
Problem description
===================
Murano is a complex system that comprises from a number of services and
technologies including several domain specific/markup languages. All of them
evolve and change from release to release. New features are introduced, code is
being refactored and not always remains compatible with previous release.
But applications written for previous Murano versions must still work on newer
releases otherwise all applications would require at least retesting and often
code changes on each release.
Proposed change
===============
Because there are many different parts in Murano that can be broken it is
necessary to have versioning support to manage backward compatibility.
Murano packages versioning
''''''''''''''''''''''''''
Murano packages will have new optional attribute in their manifest called
``Version``. It will be standard SemVer format version string consisting
of 3 parts: ``Major.Minor.Patch`` and optional SemVer suffixes
``[-dev-build.label[+metadata.label]]``.
All MuranoPL classes are said to have version of the package they contained in.
If no version specified package version will be equal to "0.0.0"
Version number will become very important not just to distinguish modifications
of the same package but to determine compatibility between them. Two classes
(packages) said to be compatible for particular operation if class with newer
version number can be used instead of older class version without breaking
application/component written for that older version.
Compatibility rules differ for various class usages but they all assume
semantic versioning rules be strictly followed. Changes which break
backwards-compatibility should increment the major segment, non-breaking new
features increment the minor segment and all non-breaking bug fixes increment
the patch segment.
Changes that require major version segment to be incremented:
* class was removed from package
* public method or property was removed from any class in the package
* public method changed argument signature to incompatible with previous one.
For example argument removed or added without default value
* input property (or initializer argument) was added without ``Default`` value
and restrictive contract (e.g. one must provide its value)
* any other change (including in class behavior) that can break class
consumers (callers)
Minor version segment should be incremented upon:
* new class was introduced in the package
* new method or property was introduced in any package's class
* list of parents (``Extends`` section) changed for any class
* any other interface (including behavior) changes that don't break external
class users but can possibly break class inheritors
For all other backward compatible changes patch segment must be incremented.
One should not make changes to package contents without also updating version
number. However there is no requirement to do it on each change. Only released
packages need to differ in version. As a result:
#. several breaking changes can be accumulated into one version number change
#. major segment change may also imply changes that would normally affect minor
and/or patch segments and the same goes for minor version segment change
#. during one development iteration packages may be incompatible with each
other even if they have compatible version number
SemVer suffixes have no special meaning for Murano and affect how two version
strings compare to each other.
Murano core library is also a package which has its own version. The first
version to have a number will be 0.1.0 and be released in Liberty.
Murano Extension Plugins (i.e python packages which declare
setuptools-entrypoints in 'io.murano.extensions' namespace) also will have
similar versioning semantics: they will have a fully qualified name (defined
by the setuptools' package name) and a version (also defined by setuptools).
From the class-loader perspective the MuranoPL classes defined in the plugins
are no difference from the classes defined in a regular package. However
for initial implementation we may just assume that all exposed plugin
classes reside in a single MuranoPL package having a name of the Python package
and version `0.0.0` to keep backward compatibility until we find a better
way to represent MuranoPL packages using Python counterpart.
Package requirements
''''''''''''''''''''
Packages may require other packages for their work. Those requirements are
expressed via ``Require`` key in package manifest. It has a form of
.. code-block:: yaml
Require:
package1_FQN: version_spec_1
...
packageN_FQN: version_spec_N
``version_spec`` here denotes allowed version range.
It can be either in `semantic_version.Spec` pip-like format (for example
`>=1.2,<2.0,!=1.5` or `*`) or as partial version string. In the later case:
* x <==> x.0.0 <= version < (x+1).0.0
* x.y <==> x.y.0 <= version < x.(y+1).0
* z.y.z <==> version == x.y.z
* empty (null) <==> 0.0.0 <= version < 1.0.0
Upon package upload/validation it is checked that at least one package in
specified version range exists.
All packages considered to be dependent on ``io.murano`` package (core
library). If such requirement is absent from the list (or the list is empty or
even there is no ``Require`` key in package manifest) then dependency
``io.murano: 0`` will be automatically added (for package types other than
MuranoPL default version may differ).
Packages must explicitly request all other packages which types are mentioned.
This includes class() contracts, inheritance (``Extends``) and dynamic object
construction (``new()``). For any type name use type should not be resolved
by the engine if its not contained either in the class package itself nor in
any of its requirements. Transitional dependencies doesn't have to be
explicitly required.
Murano Extension Plugins treated as regular MuranoPL packages and thus need
to have a way to specify requirement via Python means.
Each package will also have implicit (unless specified explicitly) requirement
on itself of a form ``PackageFQN: X`` where ``X`` is the package major version
segment meaning package can tall to other versions of itself having the same
major version. This requirement is automatically satisfied by the package
itself.
Object version
''''''''''''''
Object in Object Model are also versioned by the version of object's class.
The version will be stored in ``?/classVersion`` attribute of each object.
When model loader tries to create object from its JSON representation it is
going to ask class loader for class of specified version. Versions of object's
parts that belong to parent classes are determined by normal package dependency
resolution rules (parent classes must be in package requirements for the class
version being loaded)
If no version specified then latest available version should be used.
If version is specified but is not present in catalog then:
#. latest class with the same major and minor segments used
#. if there is no match use latest class having the same major version segment
only
#. if there is no matching class then fail
class() contracts must validate object version against package requirements.
If class A has a property with contract $.class(B) then object passed in this
property when upcasted to B must have a version compatible with requirement
specification in A's package (requesting B's package)
Side by side versioning of packages
'''''''''''''''''''''''''''''''''''
There exist cases when several version of the same package may live in the
same environment:
* there are different versions of the same MuranoPL class inside single object
model (environment)
* several class versions encounter within class parents. For example class A
extends B and C and class C inherits B2 where B and B2 are two different
versions of the same class
This implies that class loader needs to:
#. be able to load/keep in memory several versions of the same class
independently
#. when asked for a class object (i.e. MuranoClass object) from MuranoPL code
(via ``class()`` contract or ``new()`` function analyzes caller's package
requirements and returns latest class version matches those requirements
(or fail otherwise)
#. when accessed from object model loader be able to return class version
specified in object header (and find compatible replacement when the request
cannot be satisfied)
The first case when 2 different versions of the same class need to talk to each
other is handled by the fact that in order to do that there must be a
``class()`` contract for that value and it will be validated by the rules from
previous section.
For the second case where single class will attempt to inherit from two
different versions of a same class engine (dsl) should attempt to to find a
version of this class which satisfies all parties and use it instead.
However if it is impossible all remained different versions of the same class
will be threatened as if they be unrelated classes.
For example: classA inherits classB from packageX and classC from packageY.
Both classB and classC inherit from classD from packageZ, however packageX
depends on the version 1.2.0 of packageZ, while packageY depends on the
version 1.3.0. This leads to a situation when classA transitively inherits
classD of both versions 1.2 and 1.3. So, an exception will be thrown.
However, if packageY's dependency would be just "1" (which means any of the
1.x.x family) the conflict would be resolved and a 1.2 would be used as it
satisfies both inheritance chains.
Engine versioning
'''''''''''''''''
Each package has a manifest attribute named ``Format`` of a form
[PackageType/]Version where PackageType is a language used for the package
with “MuranoPL” as a default and Version is a 3-component version string
with shortening rules ``null = 1 = 1.0 = 1.0.0``. Version indicates minimum
engine/language version to run.
Language, engine and manifest format versions are tied together into a
single version.
We assume that all of above are backward compatible or internally can handle
differences between versions. Engine internally knows what changed in each
version and can use that knowledge to emulate behavior of older versions.
Engine must ensure that for each incompatibility in execution behavior or in
YAQL functions particular class method gets properly initialized context with
functions and flags from requested engine version.
UI forms versioning
'''''''''''''''''''
UI forms are versioned using Format attribute inside YAML definition.
The only way to stay compatible with older format versions is either to
support several different formats internally in dashboard or have
auto-conversion utility that will upgrade forms on package upload.
API versioning
''''''''''''''
If single API service can handle several API versions simultaneously it
should be done via endpoint prefixes (/v1/, /v2/ etc.). Otherwise there
should be several separate services listening on different ports and
registered with different names in Keystone.
We assume that dashboard-API-engine are always consistent and upgraded as
the whole. We also assume to be acceptable for different API versions to
have separate databases or tables so that environments created with one
version would not be visible to APIs of other versions.
Alternatives
------------
Keep strict backward compatibility without ability to introduce breaking
changes.
Data model impact
-----------------
For current package API:
* need to store version number for each versioned entity in the database
* in many cases uniqueness must be constrained by name, version number
and tenant ID rather than by name and tenant ID alone
* requirements for each package need to be stored in database as well
However there is an ongoing process to move packages from Murano API to
Glance v3 Artifact Repository (GLARE).
In GLARE package manifest attributes like FQN and Version are going to be
naturally mapped to corresponding artifact attributes. The package dependencies
will be stored in Glance as cross-artifact dynamic dependencies (i.e.
dependencies not on a particular artifact but on the last artifact matching
the given name and the version range query) as soon as that feature is
implemented in Glance (currently only static dependencies are implemented
there). Until that dependencies are going to be stored as a regular list of
strings, and the Murano engine will process it and query Glance to fetch the
packages.
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
All versions of released packages should be available simultaneously.
That implies that for core library and other essential packages instead
of importing only those packages at system installation all versions of
those libraries need to be imported.
If there would be several independent API services it would require additional
deployment efforts.
Developer impact
----------------
Development process need to be changed for correct version tracking.
Here is an example how it can be organized:
* for each released version of the package there is a separate folder called
after version number plus one additional folder called “dev”
* during development all changes go to “dev” folder. All packages and
requirements have a version “0.0.0” effectively turning versioning off.
* after code freeze release manager assign version number to the version about
to be released by examining commits happened between releases. It is advised
to mark such commits somehow (for example [changes major version],
[changes minor version], [changes revision] or something similar).
* requirements may be defined more accurately from the same source
* new folder for released version is created and final version of the “dev”
package is put there.
* bug/fix and backport commits can be made in between modifying either already
released package or making a copy of it with different revision version
number according to versioning requirements.
* we should write a script to package/upload packages from such versioned
folder (e.g. producing a package for each released version)
Murano-dashboard / Horizon impact
---------------------------------
In L cycle we are not going to show multiple versions of the same app in
Murano dashboard: only the last one will be shown if the multiple versions
are present. This is to minimize the changes at Dashboard side: in future
releases we'll add the ability to select the proper version.
Implementation
--------------
Assignee(s)
```````````
Primary assignee:
Stan Lagun <istalker2>
GLARE integration (API to access packages with versions and GLARE plugin):
Alexander Tivelkov <ativelkov>
Work Items
``````````
* Make Murano Engine access package manifest information
* Implement more flexible processing of package format string (currently it
requires strict version number)
* Make engine be able to setup independent YAQL context chain depending on
package's engine version requirements
* Update class loader to work with versions
* Add support for version numbers and package requirements to
DirectoryPackageLoader (including version-aware directory structure as in
``murano-apps`` repo)
* Update APIPackageLoader to work with GLARE Murano plugin
* Implement version compatibility rules for contracts and inheritance
Dependencies
============
None
Testing
=======
In general, after each release we should try to test old applications on new
release to make sure that nothing broke (which may happen if versioning rules
were not followed correctly). But it is clear that in practice this cannot be
done for all applications in catalog. So we may peek selected set of
applications and to try to deploy all versions of those applications on current
release. This set should be representative to test the most possible set of
futures.
Subset of those applications can be used for per-commit tests.
Documentation Impact
====================
Changes introduced by each version change of each component should be
documented. There should be possible to see documentation for specific previous
Murano version as it remains usable.
References
==========
None

174
specs/liberty/object-construction.rst
View File

@ -1,174 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
============================
MuranoPL object construction
============================
https://blueprints.launchpad.net/murano/+spec/object-construction
Object construction is ability to create instances of MuranoPL classes
(both YAML and Python-based) and initialize it property. It also directly
affect relations between those 2 types of classes just because they can
inherit from each other and have initializer of their own
Problem description
===================
There are several major problems with current object initialization:
* object created by ``new()`` function will always have an owner set to null.
Because of this you cannot create objects that expect themselves to be
nested in something (e.g. say ``find(std:Environment).require()``)
* there are ``initialize()`` method in Python classes and method with the same
name in YAML part, but they work completely different. YAML version is
currently doesn't invoked at all for ``new()`` making it useful for Python
classes only
* Python initializer may me called instead of YAML version in some cases as
they have the same name
* Python classes inherit from MuranoObject even if they not aware of it.
Because MuranoObject has ``__init__`` with many parameters and user
classes have constructor parameters of their own user class cannot have
``__init__`` or it will break MuranoObject. It looks very strange for a class
to inherit from Python's object and not being able to have ``__init__``.
* also because MuranoObject has many methods and properties of its own if user
class will define method with the same name it will break how MuranoPL work.
Proposed change
===============
* rename YAML initialize and destroy methods to ``.init"`` and ``.destroy``
as they are special. Make class loader automatically replace old names with
new ones for backward compatibility. Dot-prefixed names cannot be called from
MuranoPL directly as this will be a YAQL syntax error so there is no more
need to have special treatment for those names in MuranoDslExecutor and
block them from being called explicitly
* ``.init`` will be allowed to have parameters. During object initialization
object model values will go into .init parameters instead of properties
if parameter name matches key name in object model. Usual parameter contract
system applies here
* because ``.init`` need to be able to save values it receives in parameters to
corresponding objects it should be allowed to modify properties with
usage=In that are immutable in normal operation mode. The same extends to
functions that get called during initialization
* separate Python classes from MuranoObject. Classes will no longer inherit
from MuranoObject and thus will have ``__init__`` instead of initialize.
MuranoObject will store their native part in a property called ``extension``
* using YAQL injected parameters provide Python class with interface to access
its MuranoObject counterpart and invoke methods defined in YAML
* initialization should go as following:
1. YAML-defined properties initialized (in 1 pass for ``new()``, 2 passes
for object model load) except for those that can be found in ``.init``
parameters
2. Python ``__init__`` get called if present. The same property value used
to initialize object used as ``__init__`` parameters (i.e. subset of
them since ``__init__`` may have less parameters then number of declared
properties or not to have parameters at all)
3. For MuranoObject counterpart all ``.init`` methods in hierarchy get
invoked starting from the top-level classes and down to the class that
we are initializing. It should be invoked with special flag passed in
context that will allow it to write to even read-only properties.
Property values that were skipped on first stage become ``.init``
parameters on each hierarchy level. It is ``.init``'s job to set
corresponding properties (that might have different contract)
* Add ability to explicitly pass created object owner as a ``new()`` parameter
* Update object model serializer so that if object A is specified as an owner
of object B but doesn't references it and instead referenced by some other
object C that is nested inside A then reattach B to C. This will not break
object A since it remains nested in B. If A is referenced by several objects
conforming this criteria any one of them can be used.
Alternatives
============
None
Data model impact
=================
None
REST API impact
===============
None
Versioning impact
=================
* initialize and destroy will automatically be renamed to .init and .destroy
by the class loader so previously written apps won't break
* because currently initialize cannot have parameters at all backward
compatibility retained
* the same is true for ability to modify read-only properties in initializer
Other end user impact
=====================
Users gets ability to implement things like auto-scaling with
ability to create instances from within MuranoPL
Deployer impact
===============
None
Developer impact
================
None
Murano-dashboard / Horizon impact
=================================
None
Implementation
==============
Assignee(s)
```````````
Primary assignee:
Stan Lagun <slagun>
Work Items
``````````
Bullets from "proposed changes" section may be used as a work items directly
Dependencies
============
https://blueprints.launchpad.net/murano/+spec/migrate-to-yaql-vnext
Testing
=======
Please discuss how the change will be tested. We especially want to know what
tempest tests will be added. It is assumed that unit test coverage will be
added so that doesn't need to be mentioned explicitly, but discussion of why
you think unit tests are sufficient and we don't need to add more tempest
tests would need to be included.
Is this untestable in gate given current limitations (specific hardware /
software configurations available)? Is this untestable in murano-ci? If so,
are there mitigation plans (3rd party testing, gate enhancements, etc).
Documentation Impact
====================
Mentioned changes need to be included into MuranoPL documentation
References
==========
None

164
specs/liberty/policy-based-env-modification.rst
View File

@ -1,164 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
========================================
Policy Based Modification of Environment
========================================
https://blueprints.launchpad.net/murano/+spec/policy-based-env-modification
Goal is to be able to define modification of an environment by Congress policies prior
deployment. This allows to add components (for example monitoring), change/set properties
(for example to enforce given zone, flavors, ...) and relationships into environment,
so modified environment is after that deployed.
Problem description
===================
Currently it is possible to reject deployment of an environment if it does not follows
set of so called pre-deployment policies set by admin. Administrator wants to also modify
environment prior it is deployed:
* add/set/remove component properties
* add/remove relationships
* add/remove objects
**Example Use Case: Policy Based Monitoring**
Admin wants to monitor an environment, so he wants to
* install monitoring agent on each Instance
* it is done by adding component with the agent and creating relationship between
agent and Instance. It is done at pre-deploy time
* register monitoring agent on Monitoring server
* it is done by calling monitoring server API during deployment of monitoring agent.
Proposed change
===============
**Changes**
* Introduce new Congress policy rule *predeploy_modify(eid,oid,modify-action-id,priority,
[key-val]\*)*
*predeploy_modify* policy rule is queried on all actions.
Simulation Congress API is used like in case of *predeploy_errors* policy rule.
If it returns non empty list of *modifications* for given environment, then
* *deploy* action is temporarily paused, until all modifications are processed
* if any of modification fails, then environment *deploy* fails
* Pluggable modification actions
Modification actions can be plug using setup *entry_points*.
Out of box, there will be following modification actions
* add_property( name=name, value=value)
* remove_property( name=name)
* set_property( name=name, value=value)
* add_relationship( name=name, source=source-uuid, target=target-uuid)
* remove_relationship( name=name, object=object-uuid)
* add_object( type=type, owner=owner-uuid, owner-rel-name=name, [name=val]*)
* remove_object( object=object-uuid)
Alternatives
------------
Alternative can be usage of *executes[]* of Congress policy, which executes modify
actions. In this approach
* modify action has to be implemented as Congress datasource action
* triggering of executes[] has to be solved
* it is not possible to order modify action ordering
* Murano session-id of REST API must be passed to Congress
* actions can be executed only as asynchronous, so it is not possible to postpone
*deploy* environment action until all modify actions are finished
Thus it is not alternative.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
User (admin) can control modification by creating *predeploy_modify* Congress policy
rules.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
filip-blaha
Work Items
----------
* design API of modify actions
* framework for pluggable modify actions - registering and managing available actions
* implement out-of-box actions
* add point to engine where congress called and returned action list is processed on given environment
Dependencies
============
None
Testing
=======
We need to cover by unit tests:
* framework for registering/managing modify actions
* applying modify actions on environment
* processing action list returned by congress
We need to create functional tests covering end-to-end scenario.
Documentation Impact
====================
It is documented as part of policy guided fulfillment.
References
==========

206
specs/liberty/simple-software-configuration.rst
View File

@ -1,206 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=============================
Simple Software Configuration
=============================
https://blueprints.launchpad.net/murano/+spec/simple-software-configuration
The purpose is to add to Murano core-library new functionality allowing to
simplify the process of software configuration.
Problem description
===================
At the moment a developer of murano applications often has difficulties with
configuration of software on the instance. They are related with fact that
developer always has to create execution plans even in situations when some
short scripts must be executed on VM.
For example, developer wants to run mysql service. In this case he should:
* Prepare template file for execution plan
* Prepare script containing only something like this:
..
service mysql-server start
..
* Describe in code of class the sending of plan to the murano agent
It is proposed to update the murano core-library to allow to user make express
software configuration instructions without writing explicit configuration
scripts and execution plans for Murano Agent in order to develop applications
faster, and have concise and clear workflows.
Proposed change
===============
The extension of library will support only Linux instances.
The new class **io.murano.configuration.Linux** will be added. In initial
implementation this class has two methods:
::
runCommand:
Arguments:
- agent:
Contract: $.class(sys:Agent)
- command:
Contract: $.string().notNull()
- helpText:
Contract: $.string()
* **runCommand** method sends specified string containing CLI command to
murano-agent and then it will be executed.
Arguments:
* *agent* - instance of murano-agent
* *command* - string with CLI command
* *helpTest* - string (optional), description for logging. Will be used as
name of execution plan. If it is *Null* value of *command* will be used.
::
putFile:
Arguments:
- agent:
Contract: $.class(sys:Agent)
- fileContent:
Contract: $.string().notNull()
- path:
Contract: $.string().notNull()
- helpText:
Contract: $.string()
* **putFile** method takes content of file and writes it to specified path
on VM
Arguments:
* *agent* - instance of murano-agent
* *fileContent* - string, content of file
* *path* - string, path for writing
* *helpTest* - string (optional), description for logging. Will be used as
name of execution plan. If it is *Null*, value of *path* will be used.
The both methods actually use the same procedure of sending the execution plans
to the agent and require corresponding templates for that, but hide from a
developer this routine.
**Example of usage**
The next example describes how new feature can be used. This code demonstrates
workflow of method in WordPress application, which used for re-configuration
of database settings.
::
changeDatabaseConnection:
Arguments:
- dbHost:
Contract: $.string().notNull()
- dbName:
Contract: $.string().notNull()
- dbUser:
Contract: $.string().notNull()
- dbPassword:
Contract: $.string().notNull()
Body:
- $resources: new(sys:Resources)
# Creating instance of Linux class
- $linux: new(conf:Linux)
# First we need to stop server. 'runCommand' can be used here
- $linux.runCommand($.instance.agent, 'service apache2 stop')
# Creating a dictionary for replacement
- $configReplacements:
"%DB_HOST%": $dbHost
"%DB_NAME%": $dbName
"%DB_USER%": $dbUser
"%DB_PASS%": $dbPassword
# Making a replacement. `wp-config.php' is included to package
- $confFileContent: $resources.string('wp-config.php').replace($configReplacements)
# Putting ready content to necessary path on VM
- $linux.putFile($.instance.agent, $confFileContent, '/var/www/html/wordpress/wp-config.php')
# Now we can start Apache again
- $linux.runCommand($.instance.agent, 'service apache2 start')
Alternatives
------------
Instead of using the common procedure with creating execution plans and
communication with murano-agent some software configuration resources of heat
probably can be used. During updating of library it can be used in the future.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
Application developers will be able to use new functionality in their apps.
Existing apps will not be affected.
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ddovbii
Work Items
----------
* Create new class *io.murano.configuration.Linux*
* Implement methods *putFile* and *runCommand*
* Update Murano PL docs
Dependencies
============
None
References
==========
None

282
specs/liberty/simulated-execution-mode-murano-engine.rst
View File

@ -1,282 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Simulated Execution Mode For Murano Engine
==========================================
https://blueprints.launchpad.net/murano/+spec/simulated-execution-mode-murano-engine
Problem description
===================
As an Application Developer I'd like to execute my workflows without actual
deployment and interaction with murano-dashboard in order to verify my workflow
before actual deployment by those increasing my application development
speed.
Proposed change
===============
Verifying application packages should be simple and fast.
User doesn't have to re-upload package and add app to the environment on every
change.
* Allow application author to validate his application using unit-tests
* Those tests will be put to the application package to allow anyone to test
this app at any time
* Tests will look like regular unit tests. Testing framework, which will
run unit-tests will support commands, that will allow to load test package
from directory, to call class methods and configure deployments parameters.
That will make deployment test run easier. Also, test writer may run
deployment several times, examine and compare results with different
parameters
* Tests should be able to produce complete object model with some parameters
of deployment:
* environment attributes (such as tokens) (or overwriting values, defined in
config);
* mock the methods of system classes which include various kinds of external
communications. Dependent applications, system resources and various API
clients and also be mocked. It should be allowed to specify a returned
value. There would be separate specification for mocking, where the
details will be described.
Test-case prototype may look like that:
::
Namespaces:
=: io.murano.apps.foo.tests
sys: io.murano.system
pckg: io.murano.apps.foo
Extends: io.murano.tests.TestFixture
Name: FooTest
Methods:
initialize:
Body:
# - $.appJson: new(sys:Resources).json('foo-test-object-model.json')
- $.appJson:
- ?:
id: 123
type: io.murano.apps.foo.FooApp
name: my-foo-obj
instance:
?:
type: io.murano.resources.Instance
id: 42
...
setUp:
Body:
- $.env: $.createEnvironment($.appJson) # creates an instance of std:Environment
- $.myApp: $.env.applications.where($.name='my-foo-obj').first()
- mock($.myApp.instance, "deploy", "mockInstanceDeploy", $this)
- mock(res:Instance, deploy, "mockInstanceDeploy", $this)
testFooApp:
Body:
- $.env.deploy()
- $.assertEqual(true, $.myApp.getAttr('deployed'))
tearDown:
Body:
mockInstanceDeploy:
Arguments:
- mockContext
Body:
- Return:
# heat template
Alternatives
------------
Provide one CLI command, that will mock creation of VMs and other things and
returns the deployment result.
Cons:
Impossible to verify deployments, where execution plan returns a value, which
is used in future app workflow. Compare results of several deployments would be
inconvenient Real VM’s can’t be
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
Tests will be placed to a package, so manifest version need to be updated.
This functionality should be described in a separate spec.
For now, there will be no impact on project itself.
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<efedorova@mirantis.com>
Work Items
----------
#. Add ‘simulation’ mode (new entry-point) to Murano Engine, where
packages would be uploaded from the provided path there would be no
interconnection with RabbitMQ
#. Make changes to the class-loader, located in engine, to not use API.
Separate spec is provided for this change (https://review.opendev.org/#/c/198745/).
#. Implement testing framework, written in MuranoPL that will include the
classes, described below. The structure would be taken from python unittest
module. Framework will include test-runner
#. Implement mock support.
#. Define what is needed to change in MuranoPL itself
Testing framework may contain the following classes and methods.
This are base classes for simple testing framework.
* ``TestCase`` class
+------------------+-------------------------------------------------------------+
| Method | Description |
+==================+=============================================================+
| setUp() | Method called immediately before calling the test method. |
+------------------+-------------------------------------------------------------+
| tearDown() | Method called immediately after the test method has been |
| | called and the result recorded. |
+------------------+-------------------------------------------------------------+
| run(result=None) | Run the test, collecting the result into the test result |
| | object passed as result. |
+------------------+-------------------------------------------------------------+
| assert... | Different asserts (assertEqual, assertNotEqual, assertTrue, |
| | assertFalse). |
+------------------+-------------------------------------------------------------+
* ``TestResult`` class: This class is used to compile information about which tests have succeeded and which have failed.
+--------------+-------------------------------------------------------------+
| Attribute | Description |
+==============+=============================================================+
| errors | A list containing 2-tuples of TestCase instances and strings|
| | holding formatted tracebacks. Each tuple represents a test |
| | which raised an unexpected exception. |
+--------------+-------------------------------------------------------------+
| failures | A list containing 2-tuples of TestCase instances and strings|
| | holding formatted tracebacks. Each tuple represents a test |
| | where a failure was explicitly signalled using the |
| | TestCase.assert*() methods. |
+--------------+-------------------------------------------------------------+
| testsRun | The total number of tests run so far. |
+--------------+-------------------------------------------------------------+
* ``TestRunner(stream=sys.stderr, descriptions=True, verbosity=1)`` A basic test runner
implementation which prints results on standard error.
Has *run* method, witch executes the given test case. Also stores the execution result.
For the fist time test may be run only one by one. Later we can add ``TestSuite`` class and
``TestLoader`` class:
* ``TestLoader`` class is responsible for loading tests according to various criteria
and returning them wrapped in a TestSuite (or TestSuite if will add this class).
+--------------------------------------+----------------------------------------------------+
| Methods | Description |
+======================================+====================================================+
| loadTestsFromTestCase(testCaseClass) | Return a suite of all tests cases contained in the |
| | TestCase-derived testCaseClass. |
+--------------------------------------+----------------------------------------------------+
#. Implement simple mocking machinery
All mockes are separated into NonCallable and Callable mocks
``Mock`` class
Public methods
+-----------------------------+-----------------------------------------------------------------+
| Methods | Description |
+=============================+=================================================================+
| start() | Activate a patch, returning any created mock. |
+-----------------------------+-----------------------------------------------------------------+
| stop() | Stop an active patch. |
+-----------------------------+-----------------------------------------------------------------+
| patch(target) | The `target` is patched with a `new` object. `target` should be |
| | a string in the form `package.module.ClassName`. |
+-----------------------------+-----------------------------------------------------------------+
| attach_mock(mock, attribute)| Attach a mock as an attribute of this one, replacing its name |
| | and parent |
+-----------------------------+-----------------------------------------------------------------+
| configure_mock(kwargs) | Set attributes on the mock through keyword arguments |
+-----------------------------+-----------------------------------------------------------------+
Private methods:
initialize, __call__, _patch, __enter__, __exit__
Dependencies
============
None
Testing
=======
None
Documentation Impact
====================
New testing framework will be documented from scratch.
References
==========
Discussions in IRC will be provided

267
specs/liberty/ui-network-selection.rst
View File

@ -1,267 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
========================================
Add network selection element to UI form
========================================
Include the URL of your launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/ui-network-selection
Sometime a VM should be placed to existing network rather then to a new network
created during deployment. While our workflows support this, there is no way
for the end user to select this network in the easy way in the UI. It will be
great if there is a special form element which will pre-populate a list of
available networks and provide an easy option to select desired network for the
application.
Problem description
===================
Currently murano supports only so called "Default Scenario" when it comes to
networking: it creates a single network per environment, picks a router,
allocates an IP range and creates a subnet with this range within a created
network. This behavior is fine for most of the cases, however it may be
insufficient in complicated scenarios and topologies.
For example, an application may require some pre-configured networks to exist
so it may manipulate with the resources associated to them, allocate floating
IPs from the specific net etc. In this case, the existing net becomes a valid
input property for the application, so its developer may ask the users to
specify it before the deployment.
Another scenario is the need to have all the VMs of the environment to join
some pre-configured network - regardless of their configuration and the
applications they run for. This may be caused by some specific networking
requirements of a particular cloud (a frequent example is a custom proxy to
access the internet which is reachable only from a specific network segment).
When the Network Management scenarios were initially introduced in murano
`[1] <https://wiki.openstack.org/wiki/Murano/Specifications/Network_Management>`_
we planed to have a so-called "Advanced scenario", i.e to provide the users
with an ability to use existing networks, subnets, routers etc, or configure
some sophisticated combination of them.
This scenario was properly supported at engine and at the level of the Core
Library: the `io.murano.resources.Instance` class has a `networks` field which
allows to specify `customNetworks` as a collection of objects inheriting from
`io.murano.resources.Network`, which may include the existing networks or
new networks with non-default configuration.
However, there is no support of this functionality at the UI level: the object
model being generated by the Dynamic UI contains the default networking
definitions only: i.e. the directive to join Environment's default network,
which is - in its turn - is hardcoded to be a newly created network and a
subnet in it.
Proposed change
===============
A new field will be added to the Dynamic UI framework which will allow to pick
the network and a subnet from the ones available to the current user. This will
be a drop-down list populated when the form is rendered.
The value selected by the user in this field will be a tuple, consisting of
the network id and a subnet id. This ids may be passed to the application
either as plain strings or as part of a more complicated Object Model, for
example as the properties of `io.murano.resources.ExistingNeutronNetwork`
objects. It is up to application developer to properly interpret and use these
values. The existing applications will not be affected by this change, as their
"configure instance" step of the UI dialog will not include any networking
settings. In future some of our standard apps may be updated to utilize this
new field, but those updates are out of the scope of this spec.
A similar field (but a static one rather then defined as part of Dynamic UI
framework) should be added to the `New Environment` dialog form, so the user
may choose an existing network to be the default network of a given
environment. The default value in that choice should lead to the creation of a
new environment's network (i.e. to replicate the existing behavior), while any
other choice should lead to a generation of a new object of type
`io.murano.resources.ExistingNeutronNetwork` which will be passed to murano-api
as part of `defaultNetworks` dictionary as the default environment's network.
Both these fields should share the same logic for value population.
Additionally, the dynamic UI field should have the following options, defined
as constructor arguments and exposed to dynamic ui as yaml attributes:
* *include_subnets* - `True` by default. If `True`, the dropdown includes all
the possible combinations of network and subnet. E.g. if there are two
available networks X and Y, and X has two subnets A and B, while Y has a
single subnet C, then the dropdown will include 3 item: (X, A), (X, B),
(Y, C). If set to `False` the subnet info will not be retrieved, and `None`
values will be returned as second items in output tuples, so only network ids
are available.
* *filter* - `None` by default. If set to a regexp string, will be used to
display only the networks with names matching the given regexp.
* *murano_networks* - `None` by default. May have values `None`, `exclude` or
`translate`. Defines the handling of networks which are created by murano.
Such networks usually have very long randomly generated names, and thus look
ugly when displayed in dropdown. If this value is set to `exclude` then these
networks are not shown in the dropdown at all. If set to `translate` the
names of such networks are replaced by a string `Network of %env_name%`.
Note that this functionality is based on the simple string matching of the
network name prefix and the names of all the accessible murano environments.
If the environment is renamed after the initial deployment this feature will
not be able to properly translate or exclude its network name.
* *allow_auto* - `True` by default. Defines if the default value of the
dropdown (labeled "Auto") should be present in the list. The default value is
a tuple consisting of two `None` values. The logic on how to treat this value
is up to application developer. It is suggested to use this field to indicate
that the instance should join default environment network. For use-cases
where such behavior is not desired, this parameter should be set to `False`.
The string representation of the dropdown items should look like
`%NetworkName%: %cidr% (%SubnetName%)`, where `%SubnetName%` part is optional
and may be not present if the subnet's name is not set.
If neutron is not available (so murano falls back to nova-network support) the
dropdown (both the static and dynamic ones) are not populated and appropriate
hints are available in the `New Environment` dialog.
Alternatives
------------
Currently the only way to change the default networking behavior is the usage
of `networking.yaml` file which allows to override the networking setting at
the environment level, for all the murano environments of all the tenants. This
is not flexible enough and does not provide the desired user experience.
However this method will remain, as it allows to override the network setting
globally.
Data model impact
-----------------
No impact. The existing data structures will be used.
REST API impact
---------------
No impact. The existing API methods will be used.
Versioning impact
-----------------
As this feature adds a new type of Dynamic UI fields this will bump the minor
version of Dynamic UI format version. The version will change from 2.0 to 2.1
Other end user impact
---------------------
The user will see the new field in the "Create Environment" dialog. It will
also be shown when the environment is created inline in the environments grid.
The default value of this new field will follow the old behavior.
The user experience with the existing applications will not be changed.
Deployer impact
---------------
The dropdown in Dashboard will be calling public neutron APIs. If they are not
accessible due to some reason, the UI will guess that neutron is not installed
at all so nova network usage will be assumed.
However, the actual decision on the fallback to nova-network is done at the
murano-api. So, if the dashboard is unable to connect to neutron while the api
is then the behavior is inconsistent: the UI tells user that nova-network is
used, while this is not true. No error occur in this case though.
Developer impact
----------------
The application developers may need to modify their apps to use the new
feature. Patch [2] may be used as an example.
Existing apps will not be affected, they will just have the old default
behavior.
Murano-dashboard / Horizon impact
---------------------------------
The whole change proposed in this spec is a change of murano-dashboard.
No other components are modified.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ativelkov
Other contributors:
ddovbii
Work Items
----------
* Implement the shared logic to retrieve and filter the list of networks
* Implement the DynamicUI control to select networks in apps
* Add a dropdown field to a static Create Environment form to select the
default network of the environment.
* Add the support of the new control in the existing murano apps in murano-apps
repository.
Dependencies
============
* Include specific references to specs and/or blueprints in murano, or in other
projects, that this one either depends on or is related to.
* If this requires functionality of another project that is not currently used
by Murano, document that fact.
* Does this feature require any new library dependencies or code otherwise not
included in OpenStack? Or does it depend on a specific version of library?
Testing
=======
There should be an acceptance testing implemented on this feature:
* We should test deploying the apps with existing network selected and with the
default option.
* Modified application (for example [2]) should be deployed both with "Auto" as
instance network or with some existing network selected.
* The test cases above should verify the ability to assign floating IPs to the
VMs
* The networks being used as an options for the manual selection should be
connected to a router uplinked to the external network (otherwise app
deployment will fail). Also the DNS nameservers has to be manually assigned
on those networks.
Documentation Impact
====================
A new Dynamic UI field type has to be documented at `Dynamic UI definition
specification` guide at [3]
References
==========
* [1] https://wiki.openstack.org/wiki/Murano/Specifications/Network_Management
* [2] https://review.opendev.org/#/c/201659/
* [3] https://murano.readthedocs.org/en/latest/draft/appdev-guide/muranopackages/dynamic_ui.html#dynamicuispec

33
specs/mitaka/README.rst
View File

@ -1,33 +0,0 @@
Mitaka specifications
======================
This directory is supposed to hold approved specifications for the 'Mitaka' release.
You are welcome in contributing to Murano!
Murano
------
Murano Project introduces an application catalog, which allows application
developers and cloud administrators to publish various cloud-ready
applications in a browsable‎ categorized catalog. Cloud users
-- including inexperienced ones -- can then use the catalog to
compose reliable application environments with the push of a button.
Murano Project Resources
------------------------
* `Murano Official Documentation <http://murano.readthedocs.org>`_
* Project status, bugs, and blueprints are tracked on
`Launchpad <https://launchpad.net/murano>`_
* Additional resources are linked from the project
`Wiki <https://wiki.openstack.org/wiki/Murano>`_ page
* `Python client <https://github.com/openstack/python-muranoclient>`_
License
-------
Apache License Version 2.0 http://www.apache.org/licenses/LICENSE-2.0

262
specs/mitaka/actions-authentication-and-visibility.rst
View File

@ -1,262 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=====================================
Actions authentication and visibility
=====================================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/actions-authentication-and-visibility
Application actions is a way for external systems to perform action related
procedures such as maintenance, scaling etc. However those 3rd party
systems are incapable to authenticate themselves to Keystone to call this
API securely.
Problem description
===================
Currently in order to call an action caller needs to have a valid Keystone
token or trust. It is not always possible for cases where 3rd party system is
supposed to be such caller.
There can be a 4 types of authentication for application actions:
#. Public actions. Anybody can call them. No authentication required.
Usually those are the actions without side effect, but if they do
they do that on environment owner behalf.
#. Actions that require authentication and could be invoked from both Murano
UI/CLI (considering user has valid OpenStack credentials) and automation
systems that cannot authenticate to Keystone.
#. Actions that are supposed to be called from automation systems only.
#. Actions that are supposed to be called by admin only and thus require
valid OpenStack token.
It must be possible to associate actions with one of the categories above
and to handle authentication according to type that is chosen.
Proposed change
===============
It is proposed to implement pre-authenticated URLs for the actions.
We assume that any external system will be able to execute POST query
on a fixed HTTP URI that doesn't require authentication. Thus application
during deployment phase (when it has a valid Keystone token or trust)
may allocate such a secret URI on murano API server than when invoked would
trigger particular action on behalf of the user that owns the application
(environment).
There should be an ReST API call to allocate such URI and there also should
be a MuranoPL method to call it from inside of applications.
This API call works as following:
#. Creates a keystone trust for special service user (specified in config file)
to perform particular action
#. Creates a record in database that holds trust ID, project ID, environment
ID, target object ID, action name and action parameters - everything that
is necessary to call particular action on particular application with
specified input.
#. Returns URI that has ID of an inserted record.
#. Murano API should be configured to bypass authentication for such URIs.
#. When such URI is accessed, murano API should extract record ID from the URI.
Then obtain trust ID and action parameters from the database and call
the actions using trust ID putting it instead of Keystone token into
object model.
#. Existing API for calling actions should also work without authentication.
If it is accessed with Keystone token in HTTP header then it must be
validated in the API code. Otherwise only public actions (those that do
not require authentication) must be allowed there.
#. Pre-authenticated URIs can be revoked by revoking the trust and then
deleting the record from database. This should be also possible from within
MuranoPL code.
Alternatives
------------
None
Data model impact
-----------------
New database table holding:
* Record ID
* Trust ID
* OpenStack project ID
* Murano environment ID
* Target object (application) ID
* Action name
* Action parameters (JSON)
REST API impact
---------------
Several new API endpoints are added:
**POST /environments/<env_id>/actions-auth/<action_id>**
*Request*
+--------+-------------------------------------------------+----------------------------------------------+
| Method | URI | Description |
+========+=================================================+==============================================+
| POST | /environments/<env_id>/actions-auth/<action_id> | Creates pre-authenticated URI for the action |
+--------+-------------------------------------------------+----------------------------------------------+
::
{
"arg1": "value1",
"arg2": "value2"
}
*Response*
+----------------+-------------------------------------+
| Code | Description |
+================+=====================================+
| 200 | Authentication URI has been created |
+----------------+-------------------------------------+
::
{
"uri": "full pre-authenticated URI",
"id": "auth_id"
}
**DELETE /environments/<env_id>/actions-auth/<auth_id>**
*Request*
+-----------+-----------------------------------------------+------------------------------------+
| Method | URI | Description |
+===========+===============================================+====================================+
| DELETE | /environments/<env_id>/actions-auth/<auth_id> | Revokes authentication for auth_id |
+-----------+-----------------------------------------------+------------------------------------+
*Response*
+----------------+----------------------------+
| Code | Description |
+================+============================+
| 200 | Authentication was revoked |
+----------------+----------------------------+
**POST /environments/<env_id>/actions/pre-auth/<auth_id>**
*Request*
+--------+---------------------------------------------------+----------------------------------------------------+
| Method | URI | Description |
+========+===================================================+====================================================+
| POST | /environments/<env_id>/actions/pre-auth/<auth_id> | Calls action by its auth_id. If Keystoke header is |
| | | provided in HTTP readers it takes presendence over |
| | | trust_id. |
+--------+---------------------------------------------------+----------------------------------------------------+
*Response*
+----------------+----------------------------+
| Code | Description |
+================+============================+
| 200 | Action was executed |
+----------------+----------------------------+
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
Service user credentials should be configured in murano.conf.
Also base part of the URI need to be there as well so that API server
would know load balancer IP it stands behind. Keystone v3 should be enabled.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
There should be a way for the user to create pre-auth URI from Murano
dashboard.
Implementation
==============
Assignee(s)
-----------
Stan Lagun <slagun@mirantis.com>
Work Items
----------
#. Define a markup for action visibility in MuranoPL.
#. Improve object model serialization to include action visibility.
#. Create database migration and model code to work with new table.
#. Add support for missing config parameters.
#. Create API endpoint to create pre-authenticated URI.
#. Create API endpoint to revoke pre-authenticated URI.
#. Create API endpoint to invoke pre-authenticated URI.
#. Configure authentication in paste.ini.
#. Improve python-muranoclient with support for new endpoints
#. Implement MuranoPL functions to create/revoke pre-authenticated URIs for
particular action. Function should return existing Action API endpoint
for public and token-only actions.
#. Provide the same capabilities in Murano dashboard.
Dependencies
============
None
Testing
=======
Develop sample application that would deploy some 3rd part system
and provide it with pre-authenticated URI that it could call periodically.
Observe action invocation fact by its side-effects (logs etc.)
Documentation Impact
====================
New ReST and MuranoPL APIs need to be documented.
References
==========
None

207
specs/mitaka/engine-package-cache.rst
View File

@ -1,207 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
====================
Engine Package Cache
====================
Include the URL of your launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/murano-engine-package-cache
This specification provides means to improve caching of packages to fasten
consecutive deployments and action executions, that use the same package.
Problem description
===================
When murano-engine performs a deployment or executes an action on a deployed
environment it needs access to the code of the package. Currently it downloads
a package from API for each deployment and stores it in a local cache for the
duration of the deployment, but deletes it after the deployment has finished.
This means, that a consecutive deployment of the environment that uses the same
package would have to download it again. Same is true for actions. This puts an
undesired strain on murano-api and network, as the same package is downloaded
many times even if it didn't change
Currently murano-engine stores packages on disc, using package-id as cache key.
This further means that if two environments are using the same package the
first to finish would delete the cache, leaving the second environment in an
unexpected state. This can potentially lead to unexpected and hard to pinpoint
bugs.
Among other things the cache system should be able to satisfy and correctly
work around the following corner cases:
#. Several different versions of single package may be used simultaneously by
parallel deployments/actions.
#. Package may be deleted or re-uploaded to API (with or without
version change) while there are ongoing deployments that use previous
package version.
#. Deployment/action may require to download the package that is currently
being downloaded for another deployment/action.
#. Packages might need to be eventually invalidated and space constrained.
Generally there are 2 situation when a package cache needs to be invalidated:
#. A new package of the same version and FQN has been updated, meaning old one
is no longer available, but might still be used by an ongoing deployment
#. A package has not been used for long time (but may be exactly the same as
the most recent package in the API)
Proposed change
===============
Proposed change includes modifying current cache mechanism to allow it to
persist the packages on disk and not clean them up after deployment/action has
ended.
#. Change cache storage directory path from {id} to {fqn}/{version}/{id}
This would allow to easily detect outdated versions of the same package.
Listing {fqn}/{version} directory *before* asking API for an id would give
engine exact list of packages that are in cache and should be invalidated
#. Whenever engine starts executing a task it would acquire shared *usage*
lock on 2 levels: eventlet-based lock (to synchronise tasks from the
current execution task with other execution tasks) and file-system based
lock, using flock or similar primitives (to synchronise use of package
cache between different murano-engine processes). The lock should be
released after the package is no longer required.
#. If the package is not available in cache execution task would attempt to
acquire exclusive *download* lock on 2 levels (eventlet/file-based), thus
allowing only one download per id at a time. The lock should be released
after the download is finished.
#. The task/process, downloading the package would be the one responsible
for deleting outdated versions of the same package.
To do so it would acquire exclusive
*usage* lock on 2 levels for the packages it wishes to delete as part of
cleanup. This would ensure, that ongoing deployments would not be affected
by the cleanup.
Usage lock has to be acquired before download lock. Otherwise there is a
race condition where 2 versions of the same package were downloaded within
a very short interval of time by 2 tasks, and the newer version has finished
downloading before the first one did (for example the newer package is
significantly smaller) and engine started cleanup. This could lead to
newer version being the first to acquire the exclusive *usage* lock and
cleaning the package, that is in use.
(Alternatively we could have used just one *usage* lock and upgrade it from
shared to exclusive if the operation would be available for all types of
locks. Unfortunately this is not true for flock/fcntl based locks in
linux/bsd system. While the upgrade/downgrade operation is available it is
not guaranteed to be atomic, therefore we risk a race condition. See
respective man pages for more clarifications. Therefore it's not an option)
If the process crashes or is killed all the
locks held by it would be released. This is ensured by nature of flock/fcntl
locks and is obviously true for eventlet-based locks. Therefore new process
would not get deadlocked by those locks.
Limitations
-----------
This spec only aims to address the problem of invalidating package cache of
packages, that are outdated.
Solving the problem of invalidating of packages, that have not been used in a
while requires a separate cache-manager, process/service, which in turn would
require additional more fine grained locks to be implemented and it doesn't
look like a real problem right now as the size of the packages is relatively
small and the number of packages in a typical murano installation is not that
large to consume all of the space on the server.
However we should be aware of this situation and probably work on that solution
some time in the future.
Alternatives
------------
One of the alternatives would be to add HTTP headers for caching control, for
example If-Modified-Since. While this is a good idea as of itself it would
impact murano-api, python-muranoclient and murano-engine, thus making it a lot
harder to implement and test.
Instead of having 2-layer locking we could implement spin-locks around file
locks, which doesn't look like a good idea though.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
Since package cache would persist on disk — it would be possible to drain disk
resources by it. This is still possible to do today by creating a really large
amount of simultaneous deployments, although in the current situation the
package cache would be eventually deleted, and space would be reclaimed.
If we believe that this is a serious security flaw — we need to implement cache
invalidation/caps of max cache storage before M release
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
kzaitsev
Work Items
----------
#. Implement caching mechanisms.
#. Implement unit/functional tests.
Dependencies
============
None
Testing
=======
Looks like unit tests would be enough for proper coverage, although
functional tests against race conditions might be benefitial.
Documentation Impact
====================
Usual docs update required
References
==========
* `FreeBSD flock man page
<https://www.freebsd.org/cgi/man.cgi?query=flock&sektion=2>`_
* `Linux flock man page <http://linux.die.net/man/2/flock>`_

106
specs/mitaka/external-request-middleware.rst
View File

@ -1,106 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
================================================
Middleware for external (non-OpenStack) requests
================================================
https://blueprints.launchpad.net/murano/+spec/external-request-middleware
When request is coming to murano (or any other service) from outside of OpenStack
it has no OpenStack-specific requests headers at all. So, it's hard to use
standart middlewares and authentication methods for exteranal requests.
Problem description
===================
Now we need to recreate keystoneclient and get authentication information for each request
which comes from outside of OpenStack using basic auth credentials. This can be look
better if we can use standart keystone middleware for external service requests,
but we don't have enough info (at least token) in the external requests.
Now you can see this behaviour in murano service broker for Cloud Foundry.
Proposed change
===============
Create a new middleware which will handle external requests and add X-Auth-Token
header. This should be enough for keystone middleware and murano context middleware.
Middleware should be added to all external services adaptors (now we have only
service broker). It's not recommended to add it directly to murano-api because
it can be real security issue.
Alternatives
------------
Take everything as it is.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
Developers which will create adapters for external service shouldn't worry about
how it will authenticate and work with murano. They can simply add this middleware
to their applications.
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
starodubcevna
Work Items
----------
* Implement external requests middleware
Dependencies
============
None
Testing
=======
Now this can be tested in a bunch of functional tests for Cloud Foundry service
broker.
Documentation Impact
====================
None

218
specs/mitaka/mocking-machinery.rst
View File

@ -1,218 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
================================================
Mocking machinery for MuranoPL testing framework
================================================
https://blueprints.launchpad.net/murano/+spec/mocking-machinery
Currently we have separate executor, that allows to run MuranoPL code in
testing mode without dashboard. But the main purpose of this executor is to
provide 'mock' availability. This specification describes how this can be done.
Problem description
===================
It's not possible to provide unit-testing without mocking machinery.
The whole idea of murano simulation mode was in deployment imitation.
During the deployment of murano applications, a lot of external objects,
that are not connected to the app itself are involved, such as *murano agent*,
*networks*, *heat stacks* and etc.
All of the classes that should be mocked can be divided into:
* python classes
* yaml classes
* dependent applications
Mocks can really help with new application development (especially compound
ones).
Mock implementation will enable:
* Real deployment simulation
* Run several deployments with different parameters
Actually, list of use-cases is unlimited, since user will be allowed to provide
any alternative function implementation to speed up the development.
Proposed change
===============
Test-runner executor should have new context manager, that will take into
consideration specified mocks. Mocks will be specified with new global YAQL
function.
This will be first step to enable mocking. Later improvements will be added and
described in a separate specification.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None. Will be not available in the previous murano versions.
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None, only new opportunity for developers
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<efedorova@mirantis.com>
Work Items
----------
#. Create MockContextManager.
Context manager is responsible for providing a valid context with
corresponding attributes and functions for the current object.
Modified context manager will be used for mock implementation. It will be
inherited from the original one and will be called ``MockContextManager``.
It will store instructions which objects or classes are needed to be
replaced with mocks and return context with mock definition instead of the
original function. For that purpose several dictionaries should be used:
one for mapping objects with mocks and other one for class mapping.
If mock for the current object exist, new linked context
(``murano.dsl.linked_context.LinkedContext``) will be returned. It links
existing context with the new context, where mock definition is presented.
So if mocked function will be called, context will contain two
definitions with that function: mock and original one, but mock will have
higher priority.
If there is no mock for the current object or class, existing context
will be returned. If there is no existing context, *None* will be returned.
#. Add global YAQL functions (it will be available only in test-runner mode) to
set scope of mocking and mock definition. It will be called ``inject`` and
will have several declarations for different purposes:
* ``inject`` for set up mock for *class* or *object*, where mock definition
is a *name of the test class method*
* `def inject(target, target_method, mock_object, mock_name)`
* ``inject`` for set up mock for *a class* or *object*, where mock
definition is a *YAQL expression*
* `def inject(target, target_method, yaql_expr)`
Description:
* *target*: MuranoPL class name (namespaces can be used or full class name
in quotes) or MuranoPL object
* *target_method*: Method name to mock in target
* *mock_object*: Object, where mock definition is contained
* *mock_name* Name of method, where mock definition is contained
* *yaql_expr*: YAQL expression, parameters are allowed
So user is allowed to specify concrete method to use instead of original,
or to provide YAQL expression from which new function will be composed.
Advantages of defining mock with YAQL expression:
* Simplicity
Thus, if you need your methods to return different constants, you can
return it inline instead of creating different methods for each constant.
* Restricted context
By default Local variables are not seen in the mock function scope, but
it's possible to specify which variables to pass to the expression.
#. Add `withOriginal` YAQL function
* `withOriginal(a => $x, b => $y)`
YAQL function, registered in the mock context.
Allows to pass values from the original context to the mock context,
where mock function is executed. Suppose, we have ``$x: 2`` in the mock
function and ``$x: 1`` in the original function. We can not just combine
the contexts since we want to use both 'x' variables and it will be
unclear which. With new YAQL function original variables can be passed to
a mock context with configurable name. So original context need to be
saved in advanced and ``withOriginal`` function should have access to it.
* `originalMethod()`
YAQL function, registered in the context of inject function.
Calls the original method and can be used in a mock function.
#. Add `OneOf` Smart type
In new function for injection mock parameter can be class or object.
So we need to accept one of those. This new type will check function
parameter for belonging to one of type in the provided list.
Dependencies
============
None
Testing
=======
New code should be 100% covered by unit tests.
Documentation Impact
====================
Separate documentation for the whole test-runner and mock machinery will
be provided.
References
==========
None

161
specs/mitaka/multiple-engine-workers.rst
View File

@ -1,161 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=======================
Multiple engine workers
=======================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/multiple-engine-workers
This specification is to implement multiple workers to murano-engine.
Problem description
===================
murano-engine is currently single process. It's a problem for scalability.
Proposed change
===============
Implement multiple workers to murano-engine by oslo.service library.
When starting service, murano-engine forks the number of workers which is
written in configuration file. When a configuration reload is required, restart
service is needed. If child process is killed, murano-engine forks new worker.
Most of the OpenStack projects use oslo.service.
Alternatives
------------
There are some external tools.
Configuration management tool such as Puppet, Chef, Ansible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Configuration management tool controls services which is managed by init systems
like systemd, upstart. Configuration management tool does not manage processes.
Init systems
~~~~~~~~~~~~
Systemd, upstart are init systems of Unix like systems. Upstart is default init
system until Ubuntu 14.10 or RHEL6. Systemd is default init system of Ubuntu
15.04 or RHEL7, and other Linux systems.
Systemd manages processes based on configuration file. If process spawns child
processes, systemd recognizes child processes properly and manages under single
control group by a configuration file. Systemd can be configured automatically
restart process on crash but does not have ability to spawn process.
Supervisor
~~~~~~~~~~
Supervisor is time-proven process management tool written in python.
Supervisor can control a number of processes on UNIX-like operating systems.
Supervisor starts processes as subprocesses , so can true up/down and can be
configured automatically restart them on a crash.
Supervisor has good function itself, but must managed by init systems and
considered high availability. Supervisor is not working under Python 3. The
whole Openstack and murano are going to support python 3. This is not
appropriate for alternatives.
Combination of configuration management tool and systemd can be altenative for
implementing multiple processes. configuration management tool deploys services,
each service manages single murano-engine process.
Both proposed change and alternative can process restart if process crashed.
Proposed change can manage multi processes easier than alternative. Proposed
change has main process which spawns child processes. If you change the number
of processes, update configuration file and then restart service. On the other
hand, alternative needs to rewrite conf file of configuration managenment tool
and redeploy.
Proposed change can manage control group of murano-engine processes easier than
alternative. Because proposed change can change control group settings by update
a configuration file of systemd. On the other hand, alternative needs to update
each confifiguration file of systemd.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
The number of murano-engine workers config option is added in
murano.conf under engine section named workers. The default value
is oslo_concurrency.processutils.get_worker_count().
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<nakamura-h>
Work Items
----------
* Add the number of murano-engine workers config option to murano.conf
* Replace oslo_service.service.ServiceLauncher with
oslo_service.service.launch in murano.engine.cmd module.
Worker parameter of oslo_service.service.launch method takes
the number of murano-engine workers config option.
Dependencies
============
* oslo_concurrency
Testing
=======
* Unit tests on murano-engine
Documentation Impact
====================
None
References
==========
* `Oslo.service <https://github.com/openstack/oslo.service>`_
* `Puppet <https://github.com/puppetlabs/puppet>`_
* `Chef <https://github.com/chef/chef>`_
* `Ansible <https://github.com/ansible/ansible>`_
* `Upstart <http://upstart.ubuntu.com/>`_
* `Systemd <http://www.freedesktop.org/wiki/Software/systemd/>`_
* `Supervisor <http://supervisord.org/>`_

222
specs/mitaka/muranopl-metadata.rst
View File

@ -1,222 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=================================================
MuranoPL metadata to properties, classes, methods
=================================================
https://blueprints.launchpad.net/murano/+spec/metadata-in-muranopl
MuranoPL metadata is a way to attach additional information to various MuranoPL
entities such as classes, packages, methods etc. That information might be used
by both applications (to implement dynamic programming techniques) or by the
external callers (API consumers like UI or even by the Murano Engine itself
to impose some runtime behavior based on well known meta values).
Problem description
===================
With time and development of the language new challenges arrived and MuranoPL
need to be extended with new keywords for new features. For example, it may
be needed to mark some method as the one that serves special purpose to
distinguish it from other methods in the program workflow. Let's say one
wants to obtain the list of all methods within the class and choose only
methods that can be called by the unprivileged user. The one way to do
it is to introduce a new possible value for the `Usage` attribute, something
like `Usage: Non-admin`. Another way is to introduce even new keyword. The
problem is that every application developer may need to attach different kinds
of such information depending on the needs of their apps, and the range of
these needs can be endless. With the current approach, every new sort of
additional information requires modification of the core MuranoPL language
code.
In order to solve this problem, metadata will be used to store new
information about various MuranoPL entities. With this new feature MuranoPL
will become more flexible and go away from the "new keyword for new feature"
rule. All the developer will need to do is to define his own kind of data he
wants to apply to the elements of his application (in the form of the custom
MuranoPL class) and use it.
Proposed change
===============
The proposed solution is to introduce another type of classes - meta-classes.
Meta classes are similar to regular classes but has additional attributes
that control how and where instances of that meta-class can be attached.
To distinguish meta-classes from regular classes new class-level attribute
will be introduced called `Usage`. When `Usage` is `Class` (which is a default)
the rest of markup is interpreted as a class. Usage `Meta` is used to define
meta-class.
In addition to Usage the following attributes are available for meta-classes:
#. `Cardinality` - either `One` or `Many` - controls if there can be more than
one instance of the meta-class attached to a single language entity.
Default is `One`.
#. `Applies` - one of `Package`, `Type`, `Method`, `Property`, `Argument` or
`All` - controls to which of the language entities instances of the meta-
class can be attached. It is possible to specify several values using YAML
list notation. Default is `All`.
#. `Inherited` - `true` or `false` - specifies if the metadata retained for
child classes, overridden methods and properties. Default is `false`.
Now, let's take a look at the examples of the meta-class in MuranoPL:
.. code-block:: yaml
Name: FooMetaOne
Usage: Meta
Applies: Property
Cardinality: One
Properties:
description:
Contract: $.string()
Default: null
count:
Contract: $.int()
Default: null
.. code-block:: yaml
Name: FooMetaMany
Usage: Meta
Applies: [Property, Method]
Cardinality: Many
The instances of meta-classes will never have an owner and thus cannot use
`find()` function.
Instances of `FooMetaOne` class can be attached to properties only and each
property may have at most on attached `FooMetaOne` instance.
To attach this class to a property is used `Meta` keyword in a property
description.
.. code-block:: yaml
Namespaces:
=: io.murano.apps.apache
std: io.murano
res: io.murano.resources
sys: io.murano.system
meta: io.murano.meta
Name: ApacheHttpServer
Extends: std:Application
Properties:
enablePHP:
Contract: $.bool()
Default: false
instance:
Contract: $.class(res:Instance).notNull()
Meta:
meta:FooMetaOne:
description: "Stub metaclass"
count: 2
In example above Meta keyword has a scalar value because it is only one
instance get attached. However it can also be an array:
.. code-block:: yaml
Meta:
- meta:FooMetaOne:
description: "Stub metaclass"
count: 2
- meta:FooMetaMany:
- meta:FooMetaMany:
Metadata can be accessed from MuranoPL using reflection capabilities and from
Python code using existing yaql mechanism (additional yaql smart type/helper
interface may be needed to simplify the task).
Alternatives
------------
Add new keyword for new feature when we need it.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
starodubcevna
Work Items
----------
#. Create a new basic class instead of `MuranoType`.
#. Create 2 new classes which will inherit from new basic class - one for regular
data structures and the other one will be `MetaAttribute`.
#. Provide possibility to create instances of meta-classes.
#. Provide an access to meta-classes.
#. Create a mechanism to attach instances of meta-classes to related objects and
store it.
Dependencies
============
None
Testing
=======
New unit tests should be added to packages.
Documentation Impact
====================
New documents about metadata usage should be added to the documents pool.
References
==========
None

169
specs/mitaka/pluggable-package-types.rst
View File

@ -1,169 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=======================
Pluggable Package Types
=======================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/pluggable-package-types
This spec is about making possible to support different package formats
(besides `MuranoPL/1.x` and `Heat.HOT/1.0`) via package format plugins.
Problem description
===================
There are many different formats that can be used to describe applications
besides MuranoPL. Currently Murano supports 2 package types `MuranoPL/1.x` and
`Heat.HOT/1.0`. However because they both are parts of Murano source codes
it is impossible to add additional types without merging them into main source
tree thus making Murano team responsible for all of them.
Proposed change
===============
It is proposed to have support for non-MuranoPL packages the same way HOT
support is implemented: by dynamic generation of MuranoPL code at run time.
However the package types themselves need to be pluggable so that anyone
could extend Murano with additional package type by installing corresponding
plugin.
It is proposed to use stevedore library and similar approach to how MuranoPL
python plugins are currently handled.
Package types plugins will be identified by dedicated namespace
`io.murano.plugins.packages`. To specify package type one should append to
plugin's `setup.cfg` file
.. code-block:: ini
io.murano.plugins.packages =
FORMAT_STRING = CLASS_IDENTIFIER
For example:
.. code-block:: ini
io.murano.plugins.packages =
Cloudify.TOSCA/1.0 = murano_cloudify_plugin.cloudify_tosca_package:CloudifyToscaPackage
If target package type requires some utility to construct it (e.g. put right
files in right folders, zip them, generate manifest and so on) than it
should also be included in the plugin as additional shell endpoint.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None. There cannot be 2 versions of the same plugin simultaneously.
However single plugin may support several different package formats including
severals different versions of the same format using single of several Python
classes. Version number remains part of format string.
Other end user impact
---------------------
None
Deployer impact
---------------
Plugins need to be deployed to each Murano node (or to all machines running
either Murano API or Murano Engine) in order to support particular package
type. Installation is done as usually in python (`pip install PATH` or
`python setup.py install` etc.)
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Stan Lagun <slagun@mirantis.com>
Work Items
----------
1. Refactor current package class hierarchy, change order of `__init__`
parameters so that format name and version will become the first two.
This is required because they will be supplied automatically by plugin
loader.
#. Implement plugin loader that would find all installed stevedore plugins
in `io.murano.plugins.packages` and build internal mapping
format name/version -> Python implementation class
#. Add function to register in that mapping implementations that remain inside
Murano sources like MuranoPL package type
#. Add method to retrieve `Package` instance of supplied format name/version.
Plugin-loader should return class factory that will instantiate appropriate
Python class for particular format by substituting first 2 arguments
(format name and version) while the rest arguments will be supplied by the
caller.
#. Refactor `load_utils` to use plugin-loader rather than its own hardcoded
mapping.
Dependencies
============
stevedore library
Testing
=======
Testing can be performed by attempt to deploy application that is not in
native Murano format like TOSCA is.
Documentation Impact
====================
Guide that tells how to develop custom package type plugins need to be
published.
References
==========
None

260
specs/mitaka/public-env-template.rst
View File

@ -1,260 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===========================
Public Environment Template
===========================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/abstract-env-template
Continuing with the environment template catalog blueprint, which allowed
to store complex architecture for applications, an abstract environment
template catalog can be included.
It involves the existence of an environment template catalogue independent
on any user, which can be customized by anyone to include their concrete
information (e.g. keypair).
Concretely, it involves all the operations to manage environment templates
(creation, deletion, updating), plus the customization done by the user.
At the end, the customized environment template can be launched and converted
into an environment
Problem description
===================
MURANO environment template catalogue is composed by environment templates
which belong only to its owner. Thus, it is not possible to share or
reuse environment templates among tenants, since all are privates. The
idea is to have the possibility to have public templates beside private one,
which can be reused and customized by other tenants. In this
way the abstract environment template catalogue will be composed by all public
environment template from all tenants.
Proposed change
===============
Including public environment templates in MURANO requires to modify the
environment template entity in the model to include a property "is_public",
which describes whether the template is public or private. In addition, it is
required to extend the API to allow for obtaining all public environment
templates from all tenants, or customizing a concrete environment template
from another tenant.
Alternatives
------------
None
Data model impact
-----------------
A new property will be include in the environment template model.
environment-template :-
murano:property(temp_id, "is_public", bool)
REST API impact
---------------
The inclusion of the environment-template entity will imply the extension of
the API for the environment-template creation, deletion, updating and
translate into the environment.
**GET /templates?is_public=true/false**
*Request*
+----------+-----------------------------+-----------------------------------+
| Method | URI | Description |
+==========+=============================+===================================+
| GET |/templates?is_public | Get all public/privates template |
| | | from the tenant or all tenants. |
+----------+-----------------------------+-----------------------------------+
*Parameters:*
+----------------+-----------------------------------------------------------+
| Parameter | Description |
+================+===========================================================+
| `is_public` |boolean, indicates whether public environment templates are|
| |listed or not. |
+----------------+-----------------------------------------------------------+
Concretely, it is possible to have the following cases:
* GET /templates?is_public=true. All public templates from all tenants will be
returned.
* GET /templates?is_public=false. All private templates from current tenant
will be returned.
* GET /templates. All templates from current tenant plus all public templates
from all tenants will be returned.
* **Content-Type**
application/json
*Response*
::
[{
"updated": "2015-01-26T09:12:53",
"name": "env_template_name1",
"created": "2015-01-26T09:12:51",
"tenant_id": "00000000000000000000000000000001",
"version": 0,
"is_public": true,
"id": "aa9033ca7ce245fca10e38e1c8c4bbf7",
}
{
"updated": "2015-01-26T09:12:55",
"name": "env_template_name2",
"created": "2015-01-26T09:12:51",
"tenant_id": "00000000000000000000000000000082",
"version": 0,
"is_public": true,
"id": "aa9033ca7ce245fca10easdfasdfadsfaasdf",
}]
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Environment Templates obtained successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to perform the operation |
+----------------+-----------------------------------------------------------+
**POST /templates/{env-temp-id}/clone**
*Request*
+----------+------------------------------+-----------------------------------+
| Method | URI | Description |
+==========+==============================+===================================+
| POST |/templates/{env-temp-id}/clone|It clones a template from one |
| | |tenant to another, changing its |
| | |name, its tenant-id and its public |
| | |availability if required. |
+----------+------------------------------+-----------------------------------+
*Parameters:*
* `env-temp-id` - environment template ID, required
*Example Payload*
::
{
'name': 'cloned_env_template_name'
}
*Content-Type*
application/json
*Response*
::
{
"updated": "2015-01-26T09:12:51",
"name": "cloned_env_template_name",
"created": "2015-01-26T09:12:51",
"tenant_id": "00000000000000000000000000000001",
"version": 0,
"is_public": false,
"id": "aa9033ca7ce245fca10e38e1c8c4bbf7",
}
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Environment Template cloned successfully |
+----------------+-----------------------------------------------------------+
| 401 | User is not authorized to perform the operation |
+----------------+-----------------------------------------------------------+
| 404 | The environment template does not exit |
+----------------+-----------------------------------------------------------+
| 409 | Conflict. The environment template name already exists |
+----------------+-----------------------------------------------------------+
Versioning impact
-------------------------
None
Other end user impact
---------------------
As well as a change in the API to include this new entity, the
python-muranoclient will be changed for including the environment template
clone.
* env-template-clone Clone the environment template.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
New views will be required for including the public template catalog and the
clone functionality.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
hmunfru
Other contributors:
jesuspg
Work Items
----------
1. Including the is_public in the model
#. Extension of the environment template API for the is_public parameter.
#. Extension of index operation and creation of clone functionality in API.
#. Adding functional tests.
Dependencies
============
Testing
=======
Unit and functional tests should be implemented.
Documentation Impact
====================
Environment template documentation should be included.
References
==========

272
specs/mitaka/regions-support.rst
View File

@ -1,272 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=============================
Support for OpenStack regions
=============================
Related blueprints:
https://blueprints.launchpad.net/murano/+spec/default-region-for-services
https://blueprints.launchpad.net/murano/+spec/assign-environment-to-region
Many if not most production grade OpenStack installations are comprised of
several so called OpenStack Regions - autonomous OpenStack clouds joined
together through shared Keystone identity server and managed through common
interface. However Murano is not region-aware and not always works correctly in
such setups.
Problem description
===================
There are several possible topologies for multi-region deployment:
#. There is a copy of all murano services at each region. Environments
consume resources in (or in other words get deployed to) the same regions
that murano-api/murano-engine run in. Because each murano-api instance is
going to use independent database each regions will have its own list of
environments, packages etc.
#. Single murano-engine is capable of deploying environments to different
regions. In this case environment gets deployed in one specific region but
not necessary the one engine is located in. The target region is chosen
according to attribute of the environment rather than location of the the
engine. Because in this case region of murano services is independent from
target environment region it is still possible that murano services are
present in more than one region up to the case where there is a copy in each
region and each one of them is capable of deploying applications to any of
the regions (but still each region has its own database and everything it
implies).
#. Single environment may contain applications and/or components that deploy
into different regions. Because environments cross region boundaries it is
no longer single heat stack per environment and applications, components and
resources need to be associated with regions and with stacks.
In the list above each subsequent item includes and extends the previous one
and thus can be implemented in turn.
This specification covers first two items because they can be implemented
without introducing major changes to applications design or existing Heat
API bindings.
Proposed change
===============
Only a few minor changes are needed in order to enable basic region support:
* Murano-engine should probe for correct region when querying endpoints from
keystone's service catalog.
* Add attribute (property) to the Environment class saying what is the
target region for the environment.
* Improve murano-dashboard with ability to select target environment.
Alternatives
------------
None
Data model impact
-----------------
Optionally target region name for the environment can be extracted from
object model and put into a separate column in environment/session tables.
This will allow to filter (group) environments that target different regions
in dashboard.
REST API impact
---------------
None
Versioning impact
-----------------
Because the changes are backward compatible a minor version component of
the core library need to be incremented (possibly by the end of Mitaka cycle).
Other end user impact
---------------------
For each OpenStack setup that has more than one region Horizon will
automatically presents user a drop-down that allow to switch between API
instances in different regions and thus see independent lists of environments,
packages etc. Nothing need to be done from murano side here.
In addition there should be a way to select target region for particular
environment independently of currently selected API region. However this
setting should remain optional and not affect existing user experience.
Murano CLI interface should not require any changes because region selection
for API is already supported using --os-region-name option or OS_REGION_NAME
environment variable. Target region name of environment is an ordinary
environment property as is set using all the same interface.
Deployer impact
---------------
Each region that has a copy of murano needs to have murano-api and
murano-engine as well as its own RabbitMQ for api to engine and engine to
agents communications (this can be either one shared or two independent
instances of RabbitMQ). Thus murano in different regions will have different
configuration files. In the same time the version of murano itself must be the
same in all regions. In addition service endpoint of each murano-api needs to
be registered in keystone.
In scenario where single murano instance can deploy to different regions it
might be that it is required to use different RabbitMQ instances in different
regions in order for engine to communicate with murano agents. Because
RabbitMQ instances are currently not registered in keystone service catalog
address and credentials for all RabbitMQs need to be put into murano config
file.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
* Additional control (drop box with region names) is added to environment
configuration form.
* Dashboard need to correctly handle situation when murano is present only
in some of available regions.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Stan Lagun <slagun>
Work Items
----------
Add setting that specifies default region name
``````````````````````````````````````````````
murano-engine needs to be aware what OpenStack region it belongs to.
When deploying environment that doesn't explicitly target some particular
environment (as it is now) this setting will control in what region should
a Heat stack be created. We assume that there is a Heat instance in each
region so we just need to use correct endpoint of the heat API.
There is already a setting `region_name_for_services` but it is located in
the wrong section ([murano] rather than [DEFAULT]).
Make ClientManager region aware
```````````````````````````````
There is a class in murano-engine that responsible for creation of specific
service clients. Typical workflow is to ask keystone for an endpoint of
desired type (using url_for() function) and then pass the endpoint to client's
`__init__` method. However in multi-region setups there can be several
endpoints for each service that differ by region name.
ClientManager's methods need to be extended with optional `region_name`
parameter to be used for url_for(). If no region name specified by the caller
then default region name must be used.
Two work items above alone is enough to enable first use case for regions.
Add region property to the Environment
``````````````````````````````````````
Add optional string property to the Environment class that will hold target
region name for the environment.
OpenStack API binding classes (like `HeatStack` and `NetExplorer`) that
internally request a client instance from ClientManager need to modified to
pass a region name of environment they are belong to (can be None)
Note, that murano-client must always use the default region endpoint because
it is used to update environment status and download packages and this is
always done in current region rather then in one we're deploying to.
Also optionally value from this property can be replicated to a dedicated
column in database so that it will be possible to filter on.
Extend [rabbitmq] configuration setting group
`````````````````````````````````````````````
Murano uses RabbitMQ to interact with guest-VM agents. But for multi-region
setup it is desirable to have dedicated RabbitMQ instance in each region.
Thus a single section with RabbitMQ parameters is not enough.
Instead it is proposed to have variable number of sections with a common
format [rabbitmq_RegionName] (i.e. [rabbitmq_RegionOne]).
When somebody needs RabbitMQ parameters it need to look in region specific
section and only if it is not present (or some particular setting is absent)
look in [rabbitmq] that now can be used to hold default values.
This will requre modifications to both Agent/AgentListener classes and
LinuxMuranoInstance/WindowsInstance core library classes.
Add UI control to environment configuration dialog
``````````````````````````````````````````````````
In order for introduced Environment property to get non-default value
a control need to be added to environment configuration form in dashboard.
It should be a drop-down with region names with possibility to leave it empty
to user server default.
A change of the region must also reset a network selection control because
the selected region is likely to have another set of networks.
Check that dashboard works when murano is missing in some regions
`````````````````````````````````````````````````````````````````
Murano services may be present just in one or in some of available regions.
Selection of region that doesn't run murano must not lead to errors.
Instead user must be presented with a message saying that it is impossible
to create environment in this region (or upload a package etc). However
this region can still be a target for environments in other regions.
Dependencies
============
None
Testing
=======
We should test application deployment with all possible combinations of
API region and environment region (which can be absent). In addition to manual
testing it would be good to improve automated CI tests to deploy applications
to different regions. However it will requires multi-region OpenStack setup for
tests.
Documentation Impact
====================
* Config settings that affect region selection need to be documented in
murano configuration guide.
* Changes made to UI need to be documented in user's guide.
References
==========
None

259
specs/mitaka/service-api-sdk.rst
View File

@ -1,259 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===============
Service API SDK
===============
https://blueprints.launchpad.net/murano/+spec/service-api-sdk
This specification defines the design of Murano Service API SDK - a high level
client library allowing cloud applications (both murano-deployed and external)
to access applications's Service API in consistent, convenient and secure way.
Problem description
===================
Service API is a way for murano applications to expose their configuration and
lifecycle & maintenance actions for other cloud applications, users and
administrators.
Since murano is designed to be a single integration point for applications in
the cloud, all the interactions with service APIs of deployed apps should be
done via Murano API by either of the two ways:
* Modifications of Object Model, i.e. changes of declarative definition of app
configuration;
* Calls of MuranoPL methods explicitly marked as externally executable actions.
Both these ways of interaction assume some deep and low-level work with murano
API: they require to fetch the object model from the environment, inspect it,
modify according to all the contracts and constraints, call appropriate methods
and so on. This does not bother end-users and administrators since the actual
API is hidden from them by various UI and CLI tools, but becomes a problem for
third-party software solutions since they have to interact with the API
directly and thus have to possess deep knowledge of murano internals.
If we want the cloud applications to be able to interact with each other via
murano we need to provide some higher level client library which will
encapsulate all the murano-related specifics and will provide the clients with
clean and streamlined software interface.
The primary consumers of this SDK library will be the developers of cloud
applications. This applications will integrate with SDK thus obtaining high
level access to their own (and their peers') service API exposed via murano.
Proposed change
===============
It is proposed to add a new set of python classes (distributed either as a
standalone pypi package or as a new module in python-muranoclient package)
which will provide a set of high level calls to murano API returning
dynamically constructed python objects representing appropriate entities of
object model. The attributes of these objects will match appropriate Input or
Output properties of their corresponding MuranoPL objects.
Consumer's code will be able to modify the values of Input (or InputOutput)
properties but only if the value being set matches the contracts of that
properties. Output properties are readonly, so attempts to modify the
corresponding python attributes will lead to exceptions.
The SDK will provide a context manager object creating a new session in the
given environment. Each property assignment being done within an appropriate
"with" code block will result in an API call immediately modifying the object
model of appropriate session. When the execution exits the "with" block the
session will be "committed", i.e. the changes will be copied to the main
environment object model. The SDK may be configured to automatically redeploy
the environment after commits.
The pythonic objects generated by the SDK will contain methods corresponding to
publicly-exposed MuranoPL methods ("actions"). Calling these pythonic methods
from the consumer code will execute appropriate actions. The parameters of the
called methods will be evaluated to match the contracts of appropriate MuranoPL
method arguments prior to calling the actual murano action API. These
dynamically generated methods will have names matching the names of appropriate
actions.
It will be possible to call these methods either synchronously or
asynchronously. In the former case the call will block the execution until the
action execution completes, so the method calls return final result, while the
actual execution will do polling on Action API under the hood. In the latter
case the action execution will be initiated, but the call will immediately
return a "Future" object which may be used later to retrieve the result (in a
single call if the action is already completed at that moment or by polling if
not).
The SDK will provide a factory class capable to create and configure the
instances of the client based on a set of configuration values. It should allow
passing these values both manually and reading them from a file in a specific
format. The latter will be used when the SDK access is provided to a virtual
machine deployed via murano: if the SDK access is granted to it the
configuration file will be delivered to the VM as part of the provisioning
process, so the SDK will be able to load it from the well-known location.
Deployment of SDK on the VMs may be done manually by the application developers
(e.g. they may include the dependency on appropriate python package in the
requirements.txt file of their python distribution), but it may be beneficial
to simplify this process by providing appropriate methods in the base class of
Standard Library: having a `deploySdk` method in the `LinuxMuranoInstance`
class will allow applications to easily deploy the SDK on its VMs with a single
call of a MuranoPL method. Such deployment will also sent a set of
configuration settings which will allow the SDK to properly connect to and
authenticate with murano API.
The SDK will include the authentication tooling: a set of utilities which will
allow applications to request access to murano API and the user to grant it.
This particular part is out of scope for this specification and should be
described separately.
Alternatives
------------
As an alternative to this SDK the client code could use regular Murano API
without any wrappers around it. This requires more efforts from application
developer and thus should be used only if the SDK does not provide the required
functionality.
Security Impact
---------------
Any consumers of Murano API should be properly authenticated to access its
resources. The regular users will use usual keystone-based authentication by
obtaining an access token using their username and password. However the
third-party applications which are the primary consumers of the SDK described
in this spec cannot be trusted with actual users credentials. Instead some role
delegation system should be used: based on trusts, OAuth delegation or
something else.
An authentication token of some kind should be delivered to VM and become part
of the SDK configuration file; this token will allow the SDK to authenticate in
murano API. This token will be scoped to
A separate spec is required to to describe the trust system, access scopes and
the changes which have to be introduced to support all of them in Murano.
Data model impact
-----------------
No model impact
REST API impact
---------------
To properly support transaction-like changes in Object Model the session API
needs to be improved to support session "commits" without the actual
deployments.
A new API call will be added:
* Complete Session
* Completes the session, thus merging its object model back to the
environment
* Method type: POST
* Normal http response code(s): 200
* Expected error http response code(s):
* 409 if the merge cannot be done due to a conflict, i.e. if the same parts
of the object model were modified by a another recently completed session
* 403 if the environment is being deployed
* URI: /v1/environments/%environment_id%/sessions/%session_id%/complete
* URI parameters:
* environment_id - id of the environment
* session_id - id of the session
Versioning impact
-----------------
Since the session completion operation adds a new call to murano API a minor
API version should be incremented
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
Application developers will need to be familiar with SDK, as well as with the
ways to authorize their VM-based apps to access it.
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ativelkov
Work Items
----------
* Introduce API changes
* Create the SDK classes in python-muranoclient or standalone library
* Add helper methods to automate SDK deployments on the VMs provisioned by
murano-deployed applications.
Dependencies
============
* Authentication tooling has to be implemented before this SDK is created
Testing
=======
* The internals of the SDK should be covered by unit and functional tests.
* All the API calls utilized by the SDK should be covered by the integration
tests.
Documentation Impact
====================
The SDK has to be clearly documented so the application developers know how to
use it.
References
==========
None

216
specs/mitaka/support-chef-berkshelf.rst
View File

@ -1,216 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=======================================
Support Berkshelf for Chef applications
=======================================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/support-chef-berkshelf
The goal of this specification is to make possible the use of Berkshelf
(http://berkshelf.com/) to manage Chef cookbooks dependencies when Chef is
used to configure a VM.
Problem description
===================
Currently, Chef is one of the types supported in the Execution plan template.
A Chef provider included in murano-agent gets the Chef cookbooks, generates
the node specifications file and executes chef-solo to configure the VM.
When a cookbook depends on other cookbooks, every transitive dependency must
be made available before executing Chef. That means that the dependencies
metadata for each cookbook must be recursively inspected by the creator of
the Murano package and added to the required files in the Execution plan
template.
Example from murano-apps::
Scripts:
executeRecipe:
Type: Chef
Version: 1.0.0
EntryPoint: git::default
Files:
- git: https://github.com/jssjr/git.git
- yum-epel: https://github.com/chef-cookbooks/yum-epel.git
- yum: https://github.com/chef-cookbooks/yum.git
- dmg: https://github.com/opscode-cookbooks/dmg
- openssl: https://github.com/opscode-cookbooks/openssl.git
- chef-sugar: https://github.com/sethvargo/chef-sugar.git
- chef_handler: https://github.com/opscode-cookbooks/chef_handler
- windows: https://github.com/opscode-cookbooks/windows
- build-essential: https://github.com/opscode-cookbooks/build-essential
Options:
captureStdout: true
captureStderr: true
This list of cookbooks is painful to maintain and error-prone. Furthermore,
a Chef cookbook can require a specific version or a range of versions for its
dependencies. Therefore, the master branch or a git repository does not contain
necessarily the right version of the cookbook. Versions resolution for the
whole dependency graph is not an easy task to execute manually. That is why
using a tool dedicated to this task is useful.
Proposed change
===============
It is proposed to add two new possible options to the Execution plan template.
These options will be available only with the Chef executor (options are
executor-dependent):
* ``useBerkshelf`` (optional, default ``false``)
* ``berksfilePath`` (optional, used only if ``useBerkshelf = true``)
If ``useBerkshelf`` is not set or set to ``false``, the Chef executor will
work as now.
If ``useBerkshelf`` is set to ``true``, the Chef executor will
expect Berkshelf to be present on the VM, and will use it to manage the
cookbooks dependencies.
In that case, the Chef executor will use Berkshelf and the *Berksfile* found on
``berksfilePath`` to *vendor* the package and its dependencies.
Dependency and version resolution is done by Berkshelf using the Berksfile.
The Chef executor will then specify to chef-solo the location of the cookbooks.
If ``berksfilePath`` is not provided, its default value is
``<cookbookName>/Berksfile``, where ``<cookbookName>`` is taken from the
``EntryPoint`` value. For example, if ``EntryPoint = example::default``,
default value for ``berksfilePath`` is ``example/Berksfile``.
A Murano package creator wanting to use Chef with Berkshelf will have to
provide a link to the URL of a repository containing the main cookbook.
Example::
|__ Classes
│   |__ Example.yaml
|__ logo.png
|__ manifest.yaml
|__ Resources
│   |__ DeployExample.template
|__ UI
|__ ui.yaml
With ``DeployExample.template``::
FormatVersion: 2.1.0
Version: 1.0.0
Name: Deploy example
Body: |
return executeRecipe(args).stdout
Scripts:
executeRecipe:
Type: Chef
Version: 1.0.0
EntryPoint: example::default
Files:
- example: https://github.com/cookbook/example.git
Options:
captureStdout: true
captureStderr: true
useBerkshelf: true
In that example, the *berksfile* is found on ``example/Berksfile``, in the
root directory of the git repository.
Alternatives
------------
Librarian-Chef is another tool to manage cookbooks. But Berkshelf is better
integrated in the Chef ecosystem than Librarian-Chef (Berkshelf is actually
part of ChefDK). Furthermore, Librarian-Chef requires the creation of a
``Cheffile``, whereas Berkshelf uses the ``Berksfile`` already included in
every cookbook.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
Minor version of ``FormatVersion`` for the Execution plan template should be
incremented (new feature, backwards-compatible). New ``FormatVersion``
should be ``2.2.0``.
Other end user impact
---------------------
None
Deployer impact
---------------
Berkshelf need to be installed in the VMs to be configured.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
o-lemasle (Olivier Lemasle <olivier.lemasle@apalia.net>)
Work Items
----------
#. Add Berkshelf support in murano-agent
#. Create DIB elements to include Berkshelf in image
#. Create an example application using Chef with Berkshelf
Dependencies
============
None
Testing
=======
* Unit tests on murano-agent
* Functional tests: deploy an application with Chef template and Chef
cookbooks dependencies in Murano integration tests
(``MuranoDeploymentTest``)
Documentation Impact
====================
Information about how to create Execution plan templates with Chef and
Berkshelf will have to be documented.
References
==========
* http://berkshelf.com/

201
specs/mitaka/support-tosca-cloudify-format.rst
View File

@ -1,201 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===================================================
Support TOSCA-Cloudify definitions for applications
===================================================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/support-tosca-cloudify-format
TOSCA is a standard developed under OASIS foundation. It is aimed to cover
definition of complex enterprise applications which might consist of different
loosely coupled components. Components can be tied by using requirements and
capabilities which are part of TOSCA standard. As TOSCA covers most of the
Applications aspects (excluding application compilation and build) it should
be straightforward to support TOSCA format in Murano. TOSCA is adopted by
enterprises so it will allow the OpenStack ecosystem to integrate with
enterprise IT applications.
Cloudify is an open source TOSCA-based cloud orchestration software platform
written in Python, and YAML created by GigaSpaces Technologies.
It is licensed under the Apache License Version 2.0
With Murano-Cloudify TOSCA-based integration it will be possible to package
Cloudify-compatible TOSCA applications as Murano packages and deploy them via
Cloudify not necessary on OpenStack but on all cloud platforms currently
supported by Cloudify.
Problem description
===================
Murano currently does not support TOSCA and TOSCA-based applications. With
the growing popularity of TOSCA for describing application orchestrations
it would be nice if murano supports TOSCA-based orchestration templates.
Murano has accepted HOT-translator based TOSCA support but that is not the
only possible TOSCA implementation possible. Cloudify brings alternate
TOSCA solution that is not limited to Heat and OpenStack in general.
One standard for defining TOSCA-based application packages is CSAR
(http://tinyurl.com/tosca-yaml-csar). It is a compressed file that includes the
main application definition template called "blueprint" in Cloudify along with
the supporting templates and files that are referenced in the main template.
This specification addresses adding support for Cloudify blueprints
application packages in Murano application catalog with ability to deploy them
via Cloudify.
Proposed change
===============
Make it possible for Murano packages to ship Cloudify blueprints.
Cloudify blueprints will be stored in the Resource folder of the package.
The manifest will be in the package root folder.
The Cloudify code will be converted to Murano PL code on the fly.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
There should be no user impact. Users who are creating HOT-based packages would
do it the same way as before. Users who are creating TOSCA-based packages would
do it in a similar way. Application packages can be added to environments for
deployment the same way as before too.
There should be also a utility to help users package CSARs into Murano package
but it is out of the scope of this spec.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
Murano would need to represent TOSCA-based components using a specific
logo, to distinguish them from HOT-based components. This applies to other
specification formats that may come on board in the future.
Implementation
==============
Assignee(s)
-----------
Trammell <trammell@gigaspaces.comx>
Work Items
----------
#. Define new package format Cloudify.TOSCA/1.0 and create a package-type
class that will process packages with such format string.
#. Cloudify Blueprint directory that contains a blueprint in the format
filename.yaml. The name of the file needs to be declared in the
`manifest.yaml` file.
#. From main blueprint yaml file extract information about application
inputs and outputs.
#. Generate UI form definition by generating 1 input field per each TOSCA
input and also generate Application section of UI form that will map
those fields to MuranoPL application class properties.
#. From TOSCA inputs/outputs and manifest data generate MuranoPL class
that will represent TOSCA application. There should be a MuranoPL
property for each TOSCA input and output and usual application methods
as in other Murano applications.
#. In order to avoid generation of complex MuranoPL code create base class
for Cloudify applications that will implement deploy, ``destroy`` and other
common methods. Generated application will inherit from this base class
and implement only application-specific method that will be used by the
base class to get required information (like input values, entry point
name, etc.). Bacause base class is going to be regular MuranoPL class
it will form a separate MuranoPL package that need to be present in
catalog as well. Cloudify package processing code will need to tell
Murano Engine that there is a dependency to that package without
explicitly specifying that in manifest file as normaly done in MuranoPL
packages.
#. For Cloudify application to be able to talk to Cloudify manager and
do the actual provisioning/deployment MuranoPL bindings to Cloudify
rest-client need to be developed. It should provide all required methods
to upload the blueprint, create deployment and execute TOSCA workflows.
#. In base CloudifyApplication class implement common application workflows:
for installation:
#. Provide name of the blueprint file in the Cloudify Blueprint directory.
Cloudify Rest Client will tar that directory and upload the payload
to the Cloudify Manager.
#. Create blueprint deployment (Cloudify term similar to Stack in Heat or
Environment in Murano). Generated application object ID can be used
as a deployment ID
#. Invoke "install" workflow on created deployment. Watch its progress.
for uninstallation (.destroy method):
#. Invoke "uninstall" workflow on previously created deployment.
#. Delete deployment.
Dependencies
============
Applications should be compatible with Cloudify 3.1 (latest GA version - 1)
Testing
=======
#. Follow these instructions to create a Cloudify Manager in a vagrant
box on the same machine as your development environment. From there
#. Add the manager IP to your etc/murano.conf file and install the
plugin.
#. Upload the Cloudify Application Package
(contrib/plugins/cloudify_application).
#. Upload the Nodecellar Example Deployment.
(contrib/plugins/cloudify-nodecellar-example-murano).
Documentation Impact
====================
How TOSCA packages can be added to catalog and deployed should be
documented, perhaps with some examples.
References
==========
* `Cloudify <http://getcloudify.org>`_
* `Tosca-Parser on GitHub <https://github.com/openstack/tosca-parser>`_

406
specs/mitaka/support-tosca-format.rst
View File

@ -1,406 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Support TOSCA definitions for applications
==========================================
URL of launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/support-tosca-format
TOSCA is a standard developed under OASIS foundation. It is aimed to cover
definition of complex enterprise applications which might consist of different
loosely coupled components. Components can be tied by using requirements and
capabilities which are part of TOSCA standard. As TOSCA covers most of the
Applications aspects (excluding application compilation and build) it should
be straightforward to support TOSCA format in murano. TOSCA is adopted by
enterprises so it will allow the openstack ecosystem to integrate with
enterprise IT applications.
The suggested approach for adding TOSCA support to murano is to leverage
the heat-translator project, which is an openstack project. Heat-translator is
a command line tool that takes non-heat templates (e.g., TOSCA-based templates)
as an input and produces a heat orchestration template (HOT) which can then be
deployed by heat. The heat-translator functionality is imported as a library.
The benefit of using heat-translator is that it is not TOSCA specific, and
designed to be easily extensible to support any other specification language.
And this flexibility will be transferred to murano as well.
Problem description
===================
Murano currently does not support TOSCA and TOSCA-based applications. With
the growing popularity of TOSCA for describing application orchestrations
it would be nice if murano supports TOSCA-based orchestration templates.
One standard for defining TOSCA-based application packages is CSAR
(http://tinyurl.com/tosca-yaml-csar). It is a compressed file that includes the
main application definition template along with the supporting templates and
files that are referenced in the main template. There is also a required
TOSCA.meta file in the CSAR that provides metadata about the package.
This specification addresses adding support for CSAR-based application packages
in murano application catalog. It can simply be expanded to other formats and
standards that are, or will be, supported by heat-translator.
Proposed change
===============
In order to support the TOSCA CSAR package format there are a couple of use
cases that need to be implemented in murano:
*package-create / package-import*
These commands are currently used to create application definition archives
from HOT templates (diagram below).
::
+------------------+
| Murano |
+--------+---------+
|
package-create(HOT) |
+---------------------> +++
| |
<---------------------+ +++
application definition |
archive (HOT ADA) |
|
|
package-import (HOT ADA) |
+---------------------> +++
| |
| | +-----+ add component
| | | to catalog
| | <-----+ along with
| | its HOT
<---------------------+ +++ template
OK |
|
+
Murano should be able to create similar application definition archives from
templates specified in TOSCA or any other language. Diagram below shows how
this can be achieved for TOSCA CSAR packages.
Note that the TOSCA parsing functionality has been taken out of heat-translator
and put into its stand-alone own OpenStack project, called tosca-parser.
Heat-translator relies on tosca-parser in translating TOSCA specifications.
Both projects are released as PyPI packages.
::
+------------------+ +------------------+
| Murano | | Tosca Parser |
+--------+---------+ +---------+--------+
| |
package-create(CSAR)(1) | |
+---------------------> +++ |
| | validate(CSAR) |
| | +--------------------> +++
| | | |
| | <--------------------+ +++
| | OK |
| | |
| | +-----+ extract |
| | | metadata |
| | <-----+ from CSAR |
| | |
<---------------------+ +++ |
application definition | |
archive (CSAR ADA) | |
| |
| |
package-import(CSAR ADA) | |
+---------------------> +++ |
| | validate(CSAR)(2) |
| | +--------------------> +++
| | | |
| | <--------------------+ +++
| | OK |
| | |
| | +-----+ add component |
| | | to catalog |
| | <-----+ along with |
<---------------------+ +++ its CSAR |
OK | package |
| |
+ +
(1) The package format is provided to the package-create command so murano
knows it has to process a TOSCA CSAR file. This can be done using a 'format'
parameter. Also, target platforms are specified using another parameter so
murano can build a proper UI field accordingly. To guarantee backward
compatibility if no format input is provided 'HOT' is assumed as default.
(2) Re-validation is required to cover cases where the application archive
is manually created or modified.
*Deploying an environment*
For HOT-based packages murano deploys one heat stack per component. Even if
two components are in the same environment they are deployed as their own
stack and independent of each other (diagram below)
::
+----------------+ +----------------+
| Murano | | Heat |
+--------+-------+ +--------+-------+
deploy env | |
+-----------------------------------> +++ |
| | |
+-------+------------------------------------------------------------------------+
| loop | | | | |
+-------+ +------+---------------------------------------------------------+ |
|[for each | opt | | | | | |
|component +------+ | | deploy component's HOT | | |
|in env] |[if component | | +--------------------------------> +++ | |
| |is not already | | | | | |
| |deployed] | | <--------------------------------+ +++ | |
| | | | deployed stack info | | |
| | | | | | |
| +----------------------------------------------------------------+ |
| | | | |
+--------------------------------------------------------------------------------+
| | |
+-------+------------------------------------------------------------------------+
| loop | | | | |
+-------+ | | remove component's stack | |
|[for each previously | | +--------------------------------> +++ |
|deployed component | | | | |
|removed from env] | | <--------------------------------+ +++ |
| | | OK | |
| | | | |
+--------------------------------------------------------------------------------+
| | |
<-----------------------------------+ +++ |
OK + +
A similar behavior can be achieved by introducing TOSCA or other non-HOT
templates. A TOSCA-based package is stored with its TOSCA template(s) (and not
the corresponding HOT template). It may or may not need translation depending
on the platform it is deployed to. Murano passes the CSAR file to
heat-translator. Heat-translator translates the CSAR file if translation is
required for deployment, and then deploys it to the target platform.
Similar to how it is done for HOT templates, proper UI forms will be created
using MuranoPL so users can specify input parameters for the TOSCA template.
The CSAR archive will appear inside the murano package archive in the same
compressed format. This is because murano does not need to process the
internals of the CSAR archive, and the vision is that any piece of information
required from the CSAR will be acquired through tosca-parser. MuranoPL cannot
access the root of the murano archive, and therefore one of the following
options will be implemented:
1. Similar to how attached files to a HOT template are handled put the CSAR
into Resources subfolder at package creation time.
#. Put the CSAR in the root folder and move it to Resources when processing the
unzipped package.
::
+--------------+ +-----------------+ +------------------+
| Murano | | Heat Translator | | Target Platform |
+--------+-----+ +---------+-------+ +---------+--------+
deploy env | | |
+-------------------------> +++ | |
| | | |
+-------+---------------------------------------------------------------------------------------+
| loop | | | | | |
+-------+ +------+-------------------------------------------------------------------------+ |
|[for each | opt | | | | | | |
|CSAR +------+ | | deploy CSAR (platform) (1) | | | |
|component |[if component | | +--------------------------> +-+ | | |
|in env] |is not already | | | | +---+ translate if | | |
| |deployed] | | | | | deployment | | |
| | | | | | <---+ requires | | |
| | | | | | translation | | |
| | | | | | | | |
| | | | | | deploy | | |
| | | | | | +-------------------> +-+ | |
| | | | | | | | | |
| | | | | | <-------------------+ +-+ | |
| | | | | | deployment info | | |
| | | | <--------------------------+ +-+ | | |
| | | | deployment confirmation | | | |
| | | | | | | |
| +--------------------------------------------------------------------------------+ |
| | | | | |
+--------------------------------------------------------------------------------------------- -+
| | | |
| | | |
+-------+---------------------------------------------------------------------------------------+
| loop | | | | | |
+-------+ | | remove deployed CSAR | | |
|[for each previously | | +--------------------------> +++ remove | |
|deployed CSAR | | | | +-------------------> +++ |
|removed from env] | | | | | | |
| | | | | <-------------------+ +++ |
| | | | | OK | |
| | | <--------------------------+ +++ | |
| | | removal confirmation | | |
| | | | | |
+-----------------------------------------------------------------------------------------------+
| | | |
<-------------------------+ +-+ | |
OK + + +
(1) Necessary bindings for the integration of murano and heat-translator will
be implemented similar to existing bindings for HOT.
Alternatives
------------
* An alternative to explicitly providing the format for package-create is
enhancing package-create so it can auto-detect the format of the input
template and act accordingly. This enhancement can be implemented via a
separate blueprint.
* If by the time the deployment functionality is implemented heat-translator
does not yet support deployment, murano's deployment through heat will be
used.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
This should not have any negative impact on versioning. Older versions of
murano would still not support TOSCA-based templates. And the new version
would work with HOT-based templates the same way as before.
Other end user impact
---------------------
There should be no user impact. Users who are creating HOT-based packages would
do it the same way as before. Users who are creating TOSCA-based packages would
do it in a similar way. Application packages can be added to environments for
deployment the same way as before too.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
Murano would need to represent TOSCA CSAR-based components using a specific
logo, to distinguish them from HOT-based components. This applies to other
specification formats that may come on board in the future.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
vahidhashemian (I would be happy to take the lead on this, but I am hoping
I could leverage the existing knowledge on murano development by having a
co-contributor who knows murano code well).
Other contributors:
TBD
Work Items
----------
In order to support the above use cases murano needs to add functionality for
the pieces where it communicates with heat-translator. This is a break-down of
work items (each work item may be broken further into several patches for
implementation):
1. Enhance package-create command to accept CSAR format, validate the CSAR
package by calling tosca-parser, extracting metadata that comes as part of
CSAR, and eventually building the application definition archive for the
given CSAR (that includes the CSAR file too).
#. Enhance package-import command to accept CSAR-based application definition
archives, re-validate them using tosca-parser, and import them to murano
application catalog.
#. Enhance environment definition by creating a UI form for CSAR-based
applications, asking user for their input parameters and target platform
when adding them to an environment, and storing user-provided inputs along
with the added CSAR-based application.
#. Enhance environment deployment and send CSAR-based components to
heat-translator for, potential translation, and deployment to the user
specified target platform. Also, add support for removal of a deployed CSAR
component when it is removed from a re-deployed environment.
Dependencies
============
Aside from what needs to be implemented inside murano project, heat-translator
would need to implement a couple of new functionalities:
* Tosca-parser currently does not support full validation of TOSCA templates.
Workaround is to call tosca-parser and report errors, if any. To do so, an
option has to be added so it does not expect input parameters (and just
translates to HOT instead of translation and deployment) in case the TOSCA
template requires inputs (`related bug
<https://bugs.launchpad.net/heat-translator/+bug/1464024>`_).
* Tosca-parser needs to fully support translation of CSAR packages to HOT.
* Heat-translator needs to enhance its API and add a 'deploy' functionally
using which it would deploy the given application specification to the
platform of choice.
Testing
=======
Implementation work items above requires a number of overall unit tests each of
which may be broken into smaller unit tests for implementation:
1. Verify package-create for TOSCA CSAR files.
#. Verify package-import for TOSCA CSAR packages.
#. Verify UI form creation for TOSCA CSAR packages.
#. Verify deployment and removal of TOSCA CSAR packages.
Documentation Impact
====================
How TOSCA CSAR packages can be added to catalog and deployed should be
documented, perhaps with some examples.
References
==========
* `Mailing List Discussion
<http://lists.openstack.org/pipermail/openstack-dev/2015-June/065424.html>`_
* `TOSCA YAML CSAR Format <http://tinyurl.com/tosca-yaml-csar>`_
* `Heat-Translator on GitHub <https://github.com/openstack/heat-translator>`_
* `Tosca-Parser on GitHub <https://github.com/openstack/tosca-parser>`_

204
specs/murano-apps/clearwater.rst
View File

@ -1,204 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=============================
Clearwater Murano application
=============================
https://blueprints.launchpad.net/murano-apps/+spec/clearwater-vims-implementation
Problem description
===================
It must be possible to deploy and manage lifecycle of Clearwater vIMS.
Clearwater vIMS consists of several components:
* Bono (Edge Proxy)
* Sprout (SIP Router)
* Homestead (HSS Cache)
* Homer (XDMS storage)
* Ellis (provisioning portal)
* Ralf (CTF)
Additionally following DNS server application should be written:
* BIND
Clearwater deployment diagram
::
+----------+
| |
| Ellis |
HTTP | |
+------+ +-----+
| | | | HTTP
| +----------+ |
| |
+-----+-----+ +-----+-----+ +------------------+
| | | | | |
| Homer | | Homestead | |Client SIP software
| | XCAP SIP | | | |
| +--+ +--+ | +---------------+--+
| | | | | | ^
+-----------+ | | +-----------+ |
| | |
| | |
+------------+ +-+-----+----+ +------------+ |
| | HTTP | | SIP | | SIP |
| Ralf +---------+ Sprout +----------+ Bono +---------+
| | | | | |
| | | | | |
| | | | | |
+------+-----+ +------------+ +---+--------+
| |
| HTTP |
+-------------------------------------------+
More details about Clearwater components may be found at http://www.projectclearwater.org
Proposed changes
================
Separate application should be written for each of Clearwater components.
One more application should be written for aggregation of all components
(Clearwater main application).
Main application input parameters:
* OS image for deploy
* user keypair name
* zone name
* DNSSEC key
This parameters should be passed to child components.
Components input parameters:
* Flavour for instances
* Count of instances
Main application workflow
#. Instantiate child applications
#. Create common networks and security group
#. Call child applications `deploy()` method
Components have to be deployed in following order:
#. DNS application
#. One application with initial
`etcd <https://coreos.com/etcd>`_ cluster node (Ellis preferred)
#. Rest of applications
Components common workflow
#. Find main application for common properties retrieving
#. Create security group for application
#. Create instances
#. Create deploy scripts from templates
#. Run deploy scripts
For components (except DNS server) following actions must be provided:
* Scale Up - add additional node to cluster
* Scale Down - remove node from cluster
Both actions can be implemented in two steps:
* add/remove node to/from component object model
* update appropriate DNS records
Alternatives
------------
* `OpenStack Heat templates for Clearwater deployments <https://github.com/Metaswitch/clearwater-heat>`_
* `Cloudify TOSCA application <https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater>`_
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Konstantin <ksnihyr@mirantis.com>
Work Items
----------
#. Create Clearwater application prototype
#. Extend Clearwater application functionality using current Core Library
#. Add Scalable Application Framework support
Dependencies
============
#. `Scalable Application Framework <https://blueprints.launchpad.net/murano/+spec/scalable-application-framework>`_
Testing
=======
#. Clearwater live tests can be used for testing installation.
Documentation Impact
====================
None
References
==========
* `Project Clearwater Home <http://www.projectclearwater.org>`_
* `Clearwater documentation <http://clearwater.readthedocs.io/en/stable/>`_
* `Clearwater heat template <https://github.com/skolekonov/clearwater-heat/tree/mitaka>`_
* `Clearwater live tests <https://github.com/Metaswitch/clearwater-live-test>`_

3
specs/newton/approved/placeholder.rst
View File

@ -1,3 +0,0 @@
Placeholder for implemented newton specs
========================================

650
specs/newton/implemented/application-development-framework.rst
View File

@ -1,650 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=================================
Application Development Framework
=================================
https://blueprints.launchpad.net/murano/+spec/application-development-framework
Build a library of MuranoPL classes to define basic building blocks for the
applications utilizing standard patterns for scaling, load balancing, healing
etc.
Problem description
===================
Currently Murano Applications are not opinionated about the workflow they have
to follow to be scalable, highly available, (self) healable and so on. The only
interface which is declared for a standard murano application defines a single
method called “deploy” which has to implement the complete provisioning
procedure of the app. This approach is cumbersome, requires lots of coding
while most of the code is a boilerplate which could be reused by most the
applications.
Also, the “deploy” workflow is just a beginning of applications’ life in the
cloud: Murano app should define the lifecycle management of the app as well: to
handle the scaling, healing and other special behaviors. Similarly to the
deployment behaviors these kinds of workflow may have some generic steps which
should be made on most of the applications and should not require manual
copying.
It would be beneficial for the developers of Murano Applications to have a set
of base classes defining the standard building blocks for the typical
applications and their life cycle management operations.
So Murano should allow Application Developers to focus on their
application-specific tasks only (e.g. “how to install the software on the
VMs”, “how to configure it to interact with other apps” etc) without the real
need to dive into resource orchestration, server farm configuration and so on.
The application developers’ experience has to be as lightweight as possible:
ideally they should be able to focus on the software configuration tools
(scripts, puppets etc) and completely ignore the MuranoPL if they do not need
to define any custom workflow logic.
Proposed change
===============
It is proposed to implement a set of base classes for the applications and
their components.
Library structure
-----------------
Scaling primitives
~~~~~~~~~~~~~~~~~~
To properly handle various scaling scenarios it is required to have a set of
classes which will be able to group similar resources together, produce new
copies of the same resources or release the existing ones on request.
The following hierarchy of classes is proposed to define this functionality on
the generic level as well as to implement it for the case of the Servers/VMs:
::
+-------+
| +-------+
| | +--------+ +------------------+ +-----------------+
| | | | | | | |
+-+ | Object <--------+ ReplicationGroup +--------> ReplicaProvider |
+-+ | | | | |
+--------+ +---+--------------+ +-+--------+------+
^ ^ ^
| | |
| +------------------+-----+ |
| | | |
+-------+ | | CloneReplicaProvider | |
| +-------+ | | | |
| | +----------+ | +------------------------+ |
| | | | | |
+-+ | Instance | | |
+-+ | | |
+----+-----+ | |
| | |
+-----+-------+ | |
| | | |
| ServerGroup | | +---------------+--+
| | | | Template |
+-----^-------+ +---+----------+ | Server +--+
| | Server +-------> Provider | |
+------------+ Replication | +-----+------------+ +---+
| Group | | | |
+--------------+ +---+---other---+ |
| |
+---------------+
**ReplicationGroup**
A base class which does the object replication. It holds the collection of
objects generated in runtime in one of its output properties and contains
a reference to a ``ReplicaProvider`` object which is used to dynamically
generate the objects in runtime.
Input properties of this class include the ``minItems`` and ``maxItems``
allowing to limit the number of objects it holds in its collection.
An input-output property ``numItems`` allows to declaratively change the
set of objects in the collection by setting its size.
The ``deploy`` method of this class will be used to apply the replica
settings: it will drop the objects from the collection if their number
exceeds the specified by the ``numItems`` or generate some new if there
are not enough of them.
**ReplicaProvider**
A class to generate the objects for a ``ReplicationGroup``. The base one
is abstract, its inheritors should implement the abstract
``createReplica`` method to create the actual object. The method may
accept some index parameter to properly parametrize the newly created copy.
The concrete implementations of this class should define all the input
properties needed to create new instances of object. Thus the provider
actually acts as a template of the object it generates.
**CloneReplicaProvider**
An implementation of ``ReplicaProvider`` capable to create replicas by
cloning some user-provided object, making use of the ``template()``
contract.
**ServerGroup**
A class that provides static methods for deployment and releasing
resources on the group of instances.
**ServerReplicationGroup**
A subclass of the ``ReplicationGroup`` class and the ``ServerGroup``
class to replicate the ``Instance`` objects it holds.
The ``deploy`` method of this group not only generates new instances of
servers but also deploys them if needed.
**TemplateServerProvider**
A subclass of ``CloneReplicaProvider`` which is used to produce the objects
of ``Instance`` class by cloning them with subsequent parameterization of
the hostnames. May be passed as ``provider`` property to objects of the
``ServerReplicationGroup`` class.
**other replica providers**
Other subclasses of ``ReplicaProvider`` may be created to produce different
objects of ``Instance`` class and its subclasses depending on particular
application needs.
Software Components
~~~~~~~~~~~~~~~~~~~
The class to handle the lifecycle of the application is the
``SoftwareComponent`` class which is a subclass of ``Installable`` and
``Configurable``:
::
+-----------+-+ +-+------------+
| | | |
| Installable | | Configurable |
| | | |
+-----------+-+ +-+------------+
^ ^
| |
| |
+-+---------------+-+
| |
| SoftwareComponent |
| |
+-------------------+
The hierarchy of the ``SoftwareComponent`` classes should be used to define the
workflows of different application lifecycles. The general idea is to have the
generic logic in the methods of the base classes and let the derived classes
implement the handlers for the custom logic. The model is event-driven: the
workflow consists of the multiple steps, and most of the steps invoke
appropriate `on%StepName%` methods intended to provide application-specific
logic.
It is proposed to split 'internal' step logic and their 'public' handlers
into separate methods. Technically this is not necessary since the subclass may
always call `super()` to invoke the base logic, but the developers tend to
forget to invoke these super-implementations – so having the logic split into
two parts should improve the developers' experience and simplify the code of
derived classes.
The standard workflows (such as Installation and Configuration) will be defined
by the ``Installable`` and ``Configurable`` classes. The main implementation -
``SoftwareComponent`` will inherit both these classes and will define its
deployment workflow as a sequence of Installation and Configuration flows.
Other future implementations may add new workflow interfaces and mix them in
to change the deployment workflow or add new actions.
Installation workflow consists of the following methods:
::
+----------------------------------------------------------------------------------------------------------------------+
| INSTALL |
| |
| +------------------------------+ +---------------+ |
| +------------------------------+ | +---------------+ | |
| +------------------------------+ | | +---------------+ +---------------+ | | +----------------------+ |
| | | | | | | | | | | | | |
| | checkServerIsInstalled | +-+ +----> beforeInstall +----> installServer | +-+ +----> completeInstallation | |
| | +-+ | | | +-+ | | |
| +------------------------------+ +------+--------+ +------+--------+ +-----------+----------+ |
| | | | |
+----------------------------------------------------------------------------------------------------------------------+
| | |
| | |
| | |
v v v
onBeforeInstall onInstallServer onCompleteInstallation
**install**
* **Arguments:** ``serverGroup``
* **Description:**
Entry point of the installation workflow.
Iterates through all the servers of the passed ServerGroup and calls the
``checkServerIsInstalled`` method for each of them. If at least one
of the calls has returned `false` calls a ``beforeInstall`` method. Then,
for each server which returned `false` as the result of the
``checkServerIsInstalled`` calls the ``installServer`` method to do
the actual software installation.
After the installation has been completed on all the servers and if at
least one of the previous calls of ``checkServerIsInstalled``
returned `false` the method runs the ``completeInstallation`` method.
If all the calls to ``checkServerIsInstalled`` returned `true`
this method concludes without calling any others.
**checkServerNeedsInstallation**
* **Arguments:** ``server``
* **Description:** checks if the given server requires a (re)deployment of
the software component. By default checks for the value of the attribute
`installed` of the instance.
May be overridden by subclasses to provide some better logic (e.g. the
app developer may provide code to check if the given software is
pre-installed on the image which was provisioned on the VM)
**beforeInstall**
* **Arguments:** ``servers``, ``serverGroup``
* **Description:**
Reports the beginning of installation process and calls the public event
handler ``onBeforeInstall``.
**onBeforeInstall**
* **Arguments:** ``servers``, ``serverGroup``
* **Description:** Public handler of the `beforeInstall` event. Empty in
the base class, may be overridden in subclasses if some custom pre-install
logic needs to be executed.
**installServer**
* **Arguments:** ``server``, ``serverGroup``
* **Description:** Does the actual software deployment on a given server by
calling an ``onInstallServer`` public event handler. If the installation
completes successfully sets the `installed` attribute of the server to
`true`, reports successful installation and returns `null`. If an
exception encountered during the invocation of ``onInstallServer``, the
method handles that exception, reports a warning and returns the server.
The return value of the method indicates to the ``install`` method how
many failures encountered in total during the installation and with what
servers.
**onInstallServer**
* **Arguments:** ``server``, ``serverGroup``
* **Description:** an event-handler method which is called by the
``installServer`` method when the actual software deployment is needed.
Is empty in the base class. The implementations should override it with
custom logic to deploy the actual software bits.
**completeInstallation**
* **Arguments:** ``servers``, ``serverGroup``, ``failedServers``
* **Description:** is executed after all the ``installServer`` methods were
called. Checks for the number of errors reported during the installation:
if it is greater than some pre-configurable threshold an exception is
risen to interrupt the deployment workflow. Otherwise the method calls an
``onCompleteInstallation`` event handler and then reports a successful
completion of the installation workflow.
**onCompleteInstallation**
* **Arguments:** ``servers``, ``serverGroup``, ``failedServers``
* **Description:** an event-handler method which is called by the
``completeInstallation`` method when the component installation is about
to be completed.
Default implementation is empty. Inheritors may implement this method to
add some final handling, reporting etc.
Configuration workflow consists of the following methods:
::
+----------------------------------------------------------------------------------------------------------------------+
| CONFIGURATION |
| +-----------------+ |
| | | |
| | +---------------+ +-----------------+ |
| | +---------------+ | +-----------------+ | |
| +------------v--+ +---------------+ | | +--------------+ +-----------------+ | | +-----------------------+ |
| | | | | | | | | | | | | | | |
| | checkCluster\ +---> checkServer\ | +-+---> preConfigure +---> configureServer | +-+---> completeConfiguration | |
| | IsConfigured | | IsConfigured +-+ | | | +-+ | | |
| +------------+--+ +---------------+ +------+-------+ +--------+--------+ +-----------+-----------+ |
| | | | | |
| | | | | |
| +----------v----------+ | | | |
| | | | | | |
| | getConfigurationKey | | | | |
| | | | | | |
| +---------------------+ | | | |
| | | | |
+----------------------------------------------------------------------------------------------------------------------+
| | |
| | |
v v v
onPreConfigure onConfigureServer onCompleteConfiguration
**configure**
* **Arguments:** ``serverGroup``
* **Description:**
Entry point of the configuration workflow.
Calls a ``checkClusterIsConfigured`` method. If the call returns `true`,
workflow exits without any further action. Otherwise for each server in
the ``serverGroup`` it calls ``checkServerIsConfigured`` method and gets
the list of servers that need reconfiguration. The ``preConfigure``
method is called with that list. At the end calls the
``completeConfiguration`` method.
**checkClusterIsConfigured**
* **Arguments:** ``serverGroup``
* **Description:**
Has to return `true` if the configuration (i.e. the values of input
properties) of the component has not been changed since it was last
deployed on the given server group. Default implementation calls the
``getConfigurationKey`` method and compares the returned result with a
value of `configuration` attribute of ``serverGroup``. If the results
match returns `true` otherwise `false`.
**getConfigurationKey**
* **Arguments:** None
* **Description:**
Should return some values describing the configuration state of the
component. This state is used to track the changes of the configuration
by the ``checkClusterIsConfigured`` and ``checkServerIsConfigured``
methods.
Default implementation returns a synthetic value which gets updated on
every environment redeployment. Thus the subsequent calls of the
``configure`` method on the same server group during the same deployment
will not cause the reconfiguration, while the calls on the next
deployment will reapply the configuration again.
The inheritors may redefine this to include the actual values of the
configuration properties, so the configuration is reapplied only if the
appropriate input properties are changed.
**checkServerIsConfigured**
* **Arguments:** ``server``, ``serverGroup``
* **Description:**
It is called to check if the particular server of the server group has
to be reconfigured thus providing more precise control compared to
cluster-wide ``checkClusterIsConfigured``.
Default implementation calls the ``getConfigurationKey`` method and
compares the returned result with a value of `configuration` attribute
of the server. If the results match returns `true` otherwise `false`.
This method gets called only if the ``checkClusterIsConfigured`` method
returned `false` for the whole server group.
**preConfigure**
* **Arguments:** ``servers``, ``serverGroup``
* **Description:**
Reports the beginning of configuration process and calls the public
event handler ``onPreConfigure``. This method is called once per the
server group and only if the changes in configuration are detected.
**onPreConfigure**
* **Arguments:** ``servers``, ``serverGroup``
* **Description:**
Public event-handler which is called by the ``preConfigure`` method
when the (re)configuration of the component is required.
Default implementation is empty. Inheritors may implement this method to
set various kinds of cluster-wide states or output properties which may
be of use at later stages of the workflow.
**configureServer**
* **Arguments:** ``server``, ``serverGroup``
* **Description:**
Does the actual software configuration on a given server by calling the
``onConfigureServer`` public event handler. If the configuration
completes successfully calls the ``getConfigurationKey`` method and sets
the `configuration` attribute of the server to resulting value thus
saving the configuration applied to a given server. Returns `null` to
indicate successful configuration.
If an exception encountered during the invocation of
``onConfigureServer``, the method will handle that exception, report a
warning and return the current server to signal its failure to the
``configure`` method.
**onConfigureServer**
* **Arguments:** ``server``, ``serverGroup``
* **Description:**
An event-handler method which is called by the ``configureServer``
method when the actual software configuration is needed. It is empty in
the base class. The implementations should override it with custom logic
to apply the actual software configuration on a given server.
**completeConfiguration**
* **Arguments:** ``servers``, ``serverGroup``, ``failedServers``
* **Description:**
It is executed after all the ``configureServer`` methods were called.
Checks for the number of errors reported during the configuration: if it
is greater than set by some pre-configured threshold, an exception is
risen to interrupt the deployment workflow. Otherwise the method calls
an ``onCompleteConfiguration`` event handler, calls the
``getConfigurationKey`` method and sets the `configuration` attribute of
the server group to resulting value and then reports successful
completion of the configuration workflow.
**onCompleteConfiguration**
* **Arguments:** ``servers``, ``serverGroup``, ``failedServers``
* **Description:**
The event-handler method which is called by the ``completeConfiguration``
method when the component configuration is finished at all the servers.
Default implementation is empty. Inheritors may implement this method to
add some final handling, reporting etc.
Uninstallation workflow consists of the following methods:
::
+-----------------------------------------------------------------------------------+
| UNINSTALL |
| |
| +----------------+ |
| +-----------------+ | |
| +-----------------+ +-----------------+ | | +------------------------+ |
| | | | | | | | | |
| | beforeUninstall +------> uninstallServer | +-+------> completeUninstallation | |
| | | | +-+ | | |
| +-------+---------+ +--------+--------+ +-----------+------------+ |
| | | | |
| | | | |
+-----------------------------------------------------------------------------------+
| | |
v v v
onBeforeUninstall onUninstallServer onCompleteUninstallation
**uninstall**
* **Arguments:** ``ServerGroup``
* **Description:**
Entry point of the uninstallation workflow.
Iterates through all the servers of the passed ServerGroup, for each of
them checks the presence of the `installed_at_%serverId%` attribute.
If at least one attribute is present calls a ``beforeUninstall`` method
once, and then calls an ``uninstallServer`` method for each server which
has the attribute. If at least one method was called, calls an
``afterUninstall`` method at the end.
**beforeUninstall**
* **Arguments:** ``ServerGroup``
* **Description:** reports the beginning of uninstalling process and then
calls an ``onBeforeUninstall`` public event handler.
**onBeforeUninstall**
* **Arguments:** ``ServerGroup``
* **Description:** Public handler of the `beforeUninstall` event. Empty in
the base class, may be overridden in subclasses if some custom pre
uninstall logic needs to be executed.
**uninstallServer**
* **Arguments:** ``Server``
* **Description:** does the actual software removal on a given server by
calling an ``onUninstallServer`` public event handler. If the removal
completes successfully clear the `installed_at_%serverId%` attribute of
the component's attribute storage to indicate that the software component
is no longer installed on that particular machine.
If an exception was encountered during the invocation of
``onUninstallServer`` the method will handle that exception, report a
warning and increment the error counter for the particular deployment.
**onUninstallServer**
* **Arguments:** ``Server``
* **Description:** an event-handler method which is called by the
``uninstallServer`` method when the actual software removal is needed.
Is empty in the base class. The implementations should override it with
custom logic to uninstall the actual software bits.
**completeUninstallation**
* **Arguments:** ``ServerGroup``
* **Description:** is executed after all the ``uninstallServer`` methods
were called. Checks for the number of errors reported during the
uninstalling: if it is greater than some pre-configurable threshold an
exception is risen to interrupt the uninstalling workflow. Otherwise the
method calls an ``onCompleteUninstallation`` event handler and then
reports a successful completion of the uninstalling workflow.
**onCompleteUninstallation**
* **Arguments:** ``ServerGroup``
* **Description:** an event-handler method which is called by the
``completeUninstallation`` method when the component removal is done on
all the servers.
Default implementation is empty. Inheritors may implement this method to
add some final handling, reporting etc.
Alternatives
------------
The only alternative is to let application developers to write their own code
for these common tasks. We don't completely drop this alternative, since the
developers are not forced to use the framework and may still continue having
the applications which do not inherit its base classes.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
The first implementation of this spec will utilize the existing version of the
core library. Subsequent ones - redefining the hierarchy of base resource
classes - will need to increment the major version of the core library.
Other end user impact
---------------------
End users should not notice the difference between apps written using the
proposed framework and regular ones.
Deployer impact
---------------
The Framework's library package should be deployed in the target catalog for
other applications to use it.
Developer impact
----------------
This is all about simplifying the life of application developer. Developers
will have to learn the classes and patterns to utilize the benefits of the
framework.
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ativelkov
Other contributors:
tbd
Work Items
----------
* Implement the base ``Server`` class and a stub for its core-library-compatible
inheritor.
* Implement scalable primitives (ReplicationGroup, ReplicaProvider and their
inheritors)
* Implement a Event/Notification layer in MuranoPL
* Implement ``SoftwareComponent`` class with its standard
Install/Configure/Uninstall workflows.
* Implement base application classes binding ``SoftwareComponent`` objects to
``ReplicationGroup`` objects producing Servers.
Dependencies
============
None
Testing
=======
All the new MuranoPL classes should be covered by test-runner based tests.
Documentation Impact
====================
The framework should be well documented so the package developers have a
reliable and up-to-date source of information.
References
==========
None

534
specs/newton/implemented/dependency-driven-resource-deallocation.rst
View File

@ -1,534 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==================================================
Dependency-driven multi-step resource deallocation
==================================================
https://blueprints.launchpad.net/murano/+spec/dependency-driven-resource-deallocation
Murano components may allocate various kinds of resources: virtual machines,
networks, volumes etc. When these components get removed from the deployment
appropriate resources have to be deallocated. Current implementation of this
process has some significant limitations and flaws.
This specification aims to address these issues and provide a design for the
better resource deallocation / garbage collection system for MuranoPL.
Problem description
===================
In Murano the deallocation of resources is managed by a garbage collection
system (GC). Its present implementation is based on the execution of special
methods called ``.destroy()`` which may be defined in each MuranoPL class and
are intended to contain the custom code to deallocate resources allocated by
the objects of those classes.
These methods get executed when their objects leave object graph, however the
exact order of these executions is currently undefined.
There are two different scenarios when the objects may leave the object graph
thus causing their ``.destroy`` methods to be called:
* "Offline" changes of the Object Graph, i.e. the changes introduced in the
serialized version of the object graph via the API. These changes are
detected by comparing the incoming object graph (the one passed from the
API for deployment) with the "snapshot" of current environment made after
the previous deployment has been completed. If some object exists in the
"snapshot" but is missing in the input graph it is considered to be
removed. Such objects are deserialized from the snapshot and their
``.destroy`` methods are called in the order from deepest nested objects
towards topmost ones.
* "Runtime" changes. Some objects may be removed from the object graph during
deployment: they may be unreferenced or assigned to Runtime properties or
local variable only. As a result after the deployment completes these
objects are not serialized neither into the output version of the object
graph nor into its "snapshot". The next deployments will know nothing about
their existence so the objects will be lost forever. To recover the
resources allocated by such unreferenced objects murano analyzes its
ObjectStore after the execution is complete. Each object which is present in
the store but is not present in the output version of the object graph is
considered to be "orphan" and thus its ``.destroy`` method is called. The
order of these calls is currently undefined: the objects get destroyed based
on their position in the object store, which is hardly predictable.
Such a design is insufficient for production grade applications which
often require the following scenarios:
* If some object is going to be deleted another object (either owning or just
referencing it) may need to execute some actions before or after the object
is deleted.
* When a group of nested or interconnected objects is about to be deleted the
order in which their destructors should be executed may be different in
different cases.
* Sometimes the actions being executed during the destruction of an object may
depend on the fact whether some other object is about to be deleted or not.
*Example*
*Consider an Application which consists of a Network component and several
VM components. All the components are owned by the Application but there
are no ownership relationships between them. When the Application is going
to be deleted (i.e. its whole subgraph leaves the environment) all of its
components are about to be removed as well, and so their ``.destroy``
methods will be called. Since Network does not own the VM's the order
of these calls is undefined. Due to various implementation details it may
be impossible to remove the Network before the VMs which are connected to
it (e.g. in case when the VM has a mandatory requirement to be always
connected to at least one network). In this case the ``.destroy`` of a
Network component should be always called after all the VMs have been
destroyed.*
Another issue is that murano uses ``Objects`` and ``ObjectsCopy`` objects
to transfer data between deployments. When destruction dependencies are
implemented, the handler will make changes (if any) to objects in
``ObjectsCopy``. Therefore, these changes are not applied during the next
deployment and this should be addressed.
Also, sometimes it can be useful to deallocate resources used by the
unreferenced objects even before the end of deployment on demand from the
application code.
Proposed change
===============
Several improvements have to be made in Murano engine to address the problems
described above.
General concepts
----------------
Destruction dependencies
^^^^^^^^^^^^^^^^^^^^^^^^
There should be a way to establish directional `destruction dependencies`
between Murano objects. If object `Foo` establishes such a dependency on the
object `Bar` then:
* `Foo` will be notified when `Bar` is about to be destroyed. These
notifications are covered in details in "Multi-step destruction" section
below.
* If both `Foo` and `Bar` are going to be destroyed in the same garbage
collection execution, `Bar` will be destroyed before Foo.
These dependencies are not related to object ownership relationship or
property-based cross-references: the owner may have a destruction dependency on
its nested object or vise versa; the objects referencing each other may have
some destruction dependency established. Even entirely unrelated objects may
have a destruction dependency between them.
Since the destruction dependencies are directional there is a theoretical
possibility of a circular dependency to exist. In case if two or more objects
form such a circle they will still be notified about pending destruction of
their dependencies, however the order of this notifications - and the
destruction itself - is undefined in this case.
Destruction dependencies are going to be used during all kinds of garbage
collection: pre-deployment ("offline"), on-demand (during deployment) and
post-deployment.
Multi-step destruction
^^^^^^^^^^^^^^^^^^^^^^
Instead of just iterating through all the objects going to be destructed and
calling their ``.destroy`` methods Murano should perform a multi-step garbage
collection according to the following algorithm:
1. Detect all the objects going to be destroyed. It can be done by using
Python GC to track and collect objects, as described in the
*Garbage collection executions* section.
2. Sort the list of detected objects using the following comparator: for
any two objects A and B in the list:
.. code::
IF (A owns B) THEN A>B
ELSE IF (A has-a-destruction-dependency-on B)
AND (NOT B has-a-destruction-dependency-on A)
THEN A>B
ELSE A==B
where `has-a-destruction-dependency-on` means that the left operand object
has a destruction dependency (probably transitive) on right operand object,
`owns` means that the left operand object owns (probably transitively) the
right operand object.
The objects which are considered to be equal by the algorithm above can be
destroyed in parallel.
The result of the sorting is the dictionary with indexes as keys and
lists of objects with equal destruction priority as values.
3. Sort the keys of the dictionary in the reversed order of destruction
priority and for each key start parallel notification about scheduled
destruction of the objects in the corresponding group. During
notification, handlers of the objects that have a destruction dependency on
some "sentenced" object will be invoked.
4. Sort the keys of the dictionary in the direct order of destruction
priority and for each index in the dictionary start parallel destruction of
all objects in the corresponding group. Destruction of individual object
means calling the ``.destroy()`` method of the object if present and
changing the object's status to "Destroyed" (see below).
As an environment does not have owners, it will always be in the last
group of destruction. There is no guarantee that some other objects (for
example, heat stack) will be alive at the time of its destruction. Thus
``io.murano.Environment`` class should not have the ``.destroy()`` method.
Destroyed objects
^^^^^^^^^^^^^^^^^
When an object is being processed by a garbage collector, it means that there
are no live references to it from the objects of the environment. However there
may be cases when the code which handles either the pre-destroy notification
(step 4 above) or the actual ``.destroy`` method re-establishes the references
to the object being destructed, and thus the object remains in the object graph
after the GC is completed. Since the resources may be deallocated at this time
the regular usage of the object is not possible, however if it is assigned to
a property of some another object in the graph it may not always be possible to
just nullify that property since it may cause a contract violation.
To resolve such collisions it is proposed to explicitly mark such destroyed
objects as "destroyed". It means setting object's ``destroyed`` attribute to
``True`` and removing the self-reference from it.
MuranoPL executor will not allow to execute any methods on such objects,
however their properties remain accessible (i.e. readable) so
any runtime information associated with them may be recovered. Destroyed
objects will be serialized with the rest of object graph but the
json-representation of the object will have a special flag in their class
header (the "?" section) to indicate their special status. When deserialized
from json such objects will retain their "destroyed" status, so the method
execution will still be impossible even in subsequent deployments.
When "destroyed" objects are unreferenced from the object graph, their
properties get nullified and they get destroyed automatically by Python GC.
Garbage collection executions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The multi-step object destruction described above should take place in three
different scenarios:
1. *(currently existing)* Before the deployment, destroys objects which were
present in the object graph after the previous deployment was finished but
were not found in the incoming object graph of a new deployment, i.e. the
ones explicitly removed using the API.
2. *(currently existing)* After the deployment, destroys the objects existing
in the Object Store but not being a part of the **persistable** object graph
of current environment, i.e. having no references to them from the
persistable (In, Out, InOut) properties of the environment or its transitive
children).
3. *(proposed)* During the deployment, explicitly initiated from MuranoPL code.
Destroys objects which are not part of the **complete** object graph, i.e.
having no references to them from any properties of the environment
(including runtime and private properties) AND not being referenced by local
variables in any frame of all the green threads of current deployment.
To implement scenario 3, a new algorithm is needed. As mentioned in the
*Multi-step destruction* section, Python garbage collector can be used
for that. MuranoPL can make use of the way Python's ``gc.collect`` works.
Python library ``gc`` allows running on-demand garbage collection through the
``gc.collect()`` method and provides access to unreachable objects that the
collector found but cannot free through collection ``gc.garbage`` [2].
To make use of this, there should be an ability to:
* Make object store have weak links only and prevent hard links in other DSL
places so that only links between objects remain.
* Prevent murano objects that should have been destroyed by Python GC from
being destroyed.
* Get the list of such objects and destroy them in correct order and notify
subscribers about destruction.
The prevention can be done by adding ``__del__()`` magic method to the
MuranoObject class and creating cyclic reference in the object to itself.
When ``gc.collect()`` is done, all unreachable objects can be examined and
murano objects owned by current executor can be distinguished among them.
The difference in Python 3.4 and higher is that objects with a ``__del__()``
method don't end up in ``gc.garbage`` anymore and this list should be empty
most of the time [3]. So logic of adding object to GC candidates can be added
directly to ``__del__()``.
It means that in Python versions 3.4 and higher, murano objects will be added
to planned destruction from ``__del__()`` call caused by ``gc.collect()``,
and in versions prior to 3.4, presence of ``__del__()`` along with cyclic
reference to itself will provide adding the object to ``gc.garbage`` list,
and it can be added to candidates for destruction from there.
This logic can be used for garbage collection in all three scenarios
mentioned above.
The resulting list of GC candidates is then destroyed as described in the
*Multi-step destruction* section above.
With this approach, the comparison of Objects and ObjectsCopy is not needed
anymore. Garbage collector works with the same objects on each deployment,
so all changes are saved properly.
Code changes
------------
GC class
^^^^^^^^
A new python-backed Murano class called ``GC`` should be added to the core
library. It should have the following static methods:
* ``collect()`` - initiates garbage collection of unreferenced objects of
current deployment (see p.3 in "Garbage collection executions" section
above).
* ``isDestroyed(object)`` - checks if the ``object`` was already destroyed
during some GC session and thus its methods cannot be called.
* ``isDoomed(object)`` - can be used within the ``.destroy()`` method to
test if another object is also going to be destroyed.
* ``subscribeDestruction(publisher, subscriber, handler=null)`` - establishes
a destruction dependency from the ``subscriber`` to the object passed as
``publisher``. Method may be called several times, in this case only a single
destruction dependency will be established, however the same amount of calls
of ``unsubscribeDestruction`` will be required to remove it.
``handler`` argument is optional. If passed it should be the name of an
instance method defined by the caller class to handle notification of
``publisher``'s destruction (see "Multi-step destruction" section above: this
handler is executed for p. 3.1)
The following arguments will be passed to the handler method:
* ``object`` - a target object which is going to be destroyed. It is not
recommended to persist the reference to this object anywhere. This will not
prevent the object from being garbage collected but the object will be
moved to the "destroyed" state which is almost always bad. The option to do
so is considered to be advanced feature which should not be done unless it
is absolutely necessary.
* ``unsubscribeDestruction(publisher, subscriber, handler=null)`` - removes
the destruction dependency from the ``subscriber`` to the object passed as
``publisher``. Method may be called several times without any side-effects.
If ``subscribeDestruction`` was called more than once the same (or more)
amount of calls to ``unsubscribeDestruction`` is needed to remove the
dependency.
``handler`` argument is optional and must correspond the handler passed
during subscription if it was provided back then.
Alternatives
------------
Application developers may try to implement their own event-based notification
logic to notify about pending and completed object destructions. However it
will solve only part of the problem: notifications will work properly, but they
will not affect the order in which the objects are destroyed, so the workflows
will be too complicated. Also this alternative will not have the advanced
features proposed in this spec, such as ability to check if some object is
going to be destroyed.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
The proposed change is completely backwards compatible: without explicit
destruction dependencies objects will be collected based on their ownership
relationships, i.e. as it is done in the current implementation.
The packages containing classes which explicitly call the methods of ``GC``
should have package format of at least 1.4 to prevent their execution on older
versions of Murano which do not have this feature.
Other end user impact
---------------------
None
Deployer impact
---------------
None
Developer impact
----------------
Developers will get the new MuranoPL-based API to manage resource deallocation
lifecycle. If they do not want to use it they don't need to do anything.
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ativelkov
Other contributors:
Stan Lagun <istalker2>
starodubcevna
Work Items
----------
* Implement a system to define and use destruction dependencies in runtime.
* Introduce changes to MuranoObject class to keep track of "destroyed"
object status.
* Modify the serializer / deserializer to properly persist the value of the
"destroyed object" flag.
* Implement collecting unreferenced murano objects utilizing Python ``gc``
library
* Implement sorting algorithms to arrange objects-to-be-destroyed based on
criteria defined in p.2 of "Multi-step destruction" section above.
* Implement multi-step destruction workflow.
* Implement ``GC`` class to bind all the above.
* Create test-runner-based tests to cover all the test scenarios.
* Document the new features.
Dependencies
============
The development of this feature will enable Application Development Framework
[1] to address resource deallocation problems during application uninstall.
Testing
=======
Tests should be written for test-runner to cover various scenarios of resource
deallocation.
Runtime garbage collection
--------------------------
There should be test cases covering that:
* objects assigned to persistent (Input, Output, InputOutput) properties (both
locally-declared and inherited) of objects reachable from the current roots
are NOT garbage collected;
* objects assigned to transient (Runtime and undeclared) properties (both
locally-declared and inherited) of objects reachable from the current roots
are NOT garbage collected; target properties should be both locally-declared
and inherited;
* objects assigned to static properties of various classes are NOT garbage
collected;
* objects passed to python-backed objects and unreferenced in MuranoPL are NOT
garbage collected unless their MuranoObjectInterface proxies are unreferenced
/ GC'ed in python;
* objects assigned to local variables of the current execution frame (i.e.
variables of the current method and all the caller methods in call stack)
including method arguments are NOT garbage collected;
* single unreferenced objects ARE garbage collected;
* graphs of interconnected objects having no references from non-collected
objects ARE garbage collected;
* objects passed to python-backed objects and unreferenced in both MuranoPL and
python ARE garbage collected;
* garbage collector correctly processes stack-frame objects from green-threads
other than the one it is executed from
Destruction dependency resolution order
---------------------------------------
There should be test cases covering that:
* if some child object has a destruction dependency on its parent, the parent
gets destroyed before the child;
* if some parent object has a destruction dependency on its child, the child
gets destroyed before the parent;
* if some objects not being the part of some ownership hierarchy have some
destruction dependency, the dependency-object is destroyed before the
dependent one;
* if some objects have circular destruction dependency they are all destroyed
(the order is not enforced by the test);
Destruction events
------------------
Given the base scenario of object A having a destruction dependency on object B
and B being GC'ed, there should be tests covering that:
* the right order of events occurs (B scheduled for destruction -> A is
notified about planned B's destruction -> B's ``.destroy()`` method is
called -> B gets destroyed);
* A may prevent B's destruction by establishing a reference on B in the
handler;
* A may establish more than 1 destruction dependency on B and still be
notified just once;
* A may remove the destruction dependency and not get notified on B's
destruction;
* If A established N destruction dependencies and then removed them M times,
(N>M) then notifications are still delivered;
* If A established N destruction dependencies and then removed them M times,
(N<=M) then notifications are not delivered;
* B may establish a destruction dependency on itself thus subscribing to
appropriate notifications;
* ``isDoomed`` and ``isDestroyed`` methods return appropriate values when
called by A for B in appropriate event handlers.
Documentation Impact
====================
Developers documentation should be updated to describe the new ``GC`` class and
its methods, as well as the design guidelines for application developers to
follow to utilize the new capability.
References
==========
[1] https://github.com/openstack/murano-specs/blob/master/specs/newton/approved/application-development-framework.rst
[2] https://docs.python.org/2/library/gc.html
[3] https://docs.python.org/3/library/gc.html

521
specs/newton/implemented/muranopl_forms.rst
View File

@ -1,521 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==============
MuranoPL forms
==============
https://blueprints.launchpad.net/murano/+spec/muranopl-forms
Murano has a YAML-based DSL to describe the UI form required by the
application. However it is completely independent from the application itself.
So the changes made to the application properties doesn't affect the form.
This specification is aimed to make UI definition be driven by the application
code and metadata rather than by a separate language.
Problem description
===================
Current UI definition format and the way Murano handles it comes with a bunch
of problems that are there by design and cannot be solved without making some
significant changes. Most notable problems are:
* It is monolithic. UI form defines all the inputs required not just by the
application but for all its object model subtree except for the other
application references. Thus it is impossible to have UI forms for
individual components. Neither it is possible for a user to choose the
desired implementation of those components without turning them into
applications (and putting in a separate packages).
* It contains yet another yaql-based DSL that defines the mapping between
input fields and application (and inner component) properties. However
this mapping is not bound to the property contracts. So any change made to
the properties or their contracts will not affect the form but only its
validation that happens upon deployment after the form is no longer visible
to the user.
* Because the input fields do not directly correlate to application properties
it is not possible to get the reverse mappings from object model to the
input fields. Because of that once the form is closed it is impossible to
edit entered values because they already transformed to some other structure
that corresponds to application's object model and we don't have any UI
definition for that structure.
* It also means that each property needs to be described in several places in
different syntax thus duplicating the work. It is also easy to fail or get
out of sync with the code if one changes the contract but forgets to update
the UI definition or vice versa.
* It is one more DSL to learn. One more DSL invented in Murano and not covered
by any standard.
* Its object model template part (yaql mappings) use functions that are not
available in MuranoPL and vice versa. And its syntax has its own number of
problems like inability to define an object and reference it from several
places.
* It requires the client be aware of several OpenStack entities like images,
flavors etc. For example murano-dashboard talks to nova to get the list of
available flavors. This brings two additional problems:
a. It cannot be used for application that is not targeting OpenStack or
targeting OpenStack cloud other than the one used to run the dashboard.
b. It is not extensible. Each new selector requires special type in UI
definition that must be know to the client.
* Applications cannot dynamically control content of the form. For example it
is not possible to populate values for a drop-down list if those values are
not constants that are known in advance.
* Neither murano-dashboard nor the UI definition syntax support non-scalar
properties (dictionaries, lists etc.). Thus it is easy to come up with a
class design that is very useful but yet cannot be shown in UI.
* There can be only one UI definition form per package.
Proposed change
===============
Solution overview
-----------------
Proposed solution is to implement two major paradigm changes:
* Switch from the current UI definition format to an auto generated form
definitions that are not supposed to be written manually. Form generator is
going to accept MuranoPL class as an input and produce json-schema based
output that has all the required information to both present the form to a
user and do a client-side validation of the input.
* Make that UI schemas be per-class. There will be a new API call to request
UI definition for particular class or even particular method of that class
rather than for a package.
As a result the UI workflow to add new application is going to be like this:
#. User wants to add application x.y.z identified by the application package
FQN.
#. Client asks for UI definition for that package using existing API call
(the one that returns UI definition in existing "old" format).
If the call is successful and UI definition was obtained then the rest of
workflow is remained as it is now.
#. Otherwise the client (dashboard) requests UI schema for the class x.y.z
using new API call.
#. API sends murano-engine request to generate UI schema for the class x.y.z.
#. Engine loads class definition from appropriate package including attached
meta-values.
#. Taking class property declarations (contracts) and attached meta-values
as an input, engine generates json-schema that has everything needed to
present the UI form and do most of the client-side validation.
#. The same is done for all model builder methods (see below) using the same
algorithm but using their arguments instead of class properties as an input.
#. List of schemas is sent back to the API and, in turn, returned to the
caller (client).
#. The client decides which of the schemas to use (usually by asking the user).
It can always go with default schema for the class or take advantage of
one of the schemas provided by model builder methods.
#. Using the extended json schema client generates UI form. The form is going
to have 1-1 mapping between input controls and class properties or method
arguments.
#. After the form was filled it is validated on the client side using given
json schema (and its optional murano-specific extensions) and then submitted
to the API.
#. If the client opted to go with one of the model builder schemas the form
output is sent to the engine in attempt to call the model builder method
using the form values as an argument values. Model builder will then
return modified object model that is sent back to the client (via API).
Then the client can continue with the form edit using default schema or
apply additional model builder method.
#. Otherwise the constructed object model is inserted into environment
definition into API's database.
To change the application settings later, the same workflow is applied. The
only difference is that it doesn't try to use the old UI definition approach
but instead immediately requests new UI schema for the type specified in the
object model.
Schema generation
-----------------
Schema generation is a special service provided by the API (through the RPC
call to murano-engine) that takes a class FQN (including version or version
spec) and optional method name and produces json-schema compliant JSON. The
schema may have extra attributes for information that cannot be expressed by
standard json-schema attributes alone (for example field order).
If no method was provided the generation code takes class declaration and tries
to produce json-schema records by running YAQL expression contracts for its
properties in a special yaql context where all contract functions are
redefined to produce json-schema rather than validate the input. Thus it
creates a different implementation of the same contract syntax. In addition
the generation code may take into account number of well-known Meta classes
from the core library (to be added) and alter the schema if corresponding Meta
is attached to the properties. Meta values can specify things that cannot be
taken from contracts alone. For example it can be a property description text.
For the check() contract that accepts arbitrary yaql expression the analysis
of the expression AST is performed:
#. If the expression matches EXPR1 and EXPR2 and ... and EXPRN pattern it is
split into several validations (as if it was written as
``check(EXPR1).check(EXPR2)...check(EXPRN))``.
#. Each of (sub)expressions is checked against supported patterns that can
be translated to json-schema:
* comparison of len($) for a string len
* regex match function
* number comparison
* `in` operator that can be converted to enum
All expressions that cannot be translated with this algorithm are ignored and
the value will not be validated on the client side.
If the property/argument has a `Default` specifier it is translated to
`default` schema property.
For the class() the generated schema type is going to be `muranoObject`
with the following attributes:
* `muranoType` - FQN of the base class;
* `version` - version or version-spec that should be used to obtain
`muranoType` (as seen in the manifest);
* `owned` which can be `true`, `false` or null (or missing which means the
same as null). `true` means that the object must be owned by the parent
(thus ID of existing object in object model cannot be provided here),
`false` means that only existing object can be referenced and `null`
means that both options are valid.
Custom json-schema type is needed in order to retain reference semantics that
is it is an object which real type needs to be looked up in the catalog rather
than some plain dictionary or string. Client must understand `muranoObject`
type and know how to get list of valid type inheritors.
When `check()` contract is used to validate MuranoObject value (i.e. the
result of `class()` contract) it may put some constraints on that object's
properties or properties of some inner objects. In this case the schema
generator can emit additional `context` attribute to the property schema.
`context` is set to json-schema for the nested object (or its children).
Upon the input of a referenced object the client should check it against
all the `context` schemas up in the object model tree.
By default the generation algorithm produces the schema for the class and
for each model builder method available.
If the method name was provided to the engine command the same algorithm is
applied to that method only and its arguments are used where the class
properties would be used otherwise. In this case methods can be any methods
that can be invoked by the API which currently are actions and model builder
methods.
Model builders
--------------
Model builders are special MuranoPL methods that take a class definition
(in an object model format dictionary form) and number of optional arguments
and return modified object model.
Model builders are used to simplify object model generation using the template
obtained from its parameters. When generating json schema from such methods
their first parameter (which is current object model) is skipped.
In order for a method to be considered a model builder it must have the
following properties:
#. It must be static (`Usage: Static`)
#. In must have the public scope (`Scope: Public`. I.e. it must be a static
action. See https://blueprints.launchpad.net/murano/+spec/static-actions for
more information on static actions)
#. It must have `io.murano.metadata.ModelBuilder` `Meta` applied to it.
This is a marker class that is going to be introduced to the core library
to distinguish model builders from other static actions.
Caller uses static actions API to invoke the builder and obtain a generated
object model snippet.
UI hints
--------
In addition to the information that can be obtained from contracts some
additional information is needed to produce correct representation for
the property or argument. This information is provided by meta-classes
that need to be introduced to the core library:
`io.murano.metadata.Title`: title of an entity. Can be applied to anything.
The value is in `text` property of a meta-class. Upon schema generation
it is translated to `title` schema key. If no meta is attached then the
property/argument name is used as a title.
`io.murano.metadata.Description`: description of an entity. Can be applied to
anything. The value is in `text` property of the meta-class. Upon schema
generation it is translated to `description` schema key.
`io.murano.metadata.HelpText`: help text of an entity. Can be applied to
anything. The value is in `text` property of the meta-class. Upon schema
generation it is translated to `helpText` schema key.
`io.murano.metadata.forms.Hidden`: marks property or argument to be invisible.
Upon schema generation `"visible": false` is produced.
`io.murano.metadata.Position`: position of the property/argument within the
form. It has two properties:
* `index`: integer by which all of the fields are sorted before rendering.
it doesn't have to be consecutive. If the inherited field has the same
index as the field from the generated class then inherited one goes first.
For this matter property indexes might be re-enumerated upon schema
generation to the consecutive unique indexes. This property is translated
to `formIndex` schema key. If no position specified then the field will be
placed in the list of unordered fields (probably sorted by their title).
* `section': section name for the field. If not provided then it will be
automatically placed in default section for all such fields.
Section name is represented as `formSection` key in the schema of each
field. Additional attributes for the section with that name can be found
in the root schema for the type/method.
`io.murano.metadata.forms.Section`: specifies form sections for the class.
Can be multiple times applied either to the class or to the method.
It has the following properties:
* `name`: name of the section that is going to be used in
`io.murano.metadata.Position` instances.
* `title`: title of the section. In no title provided it is assumed to be
equal to the section mame.
* `index`: index of the section in a section list (e.g. tab number).
Similar to `Position` indexes those numbers doesn't have to be consecutive
and only used to sort sections within the form.
Child classes may redefine sections inherited from their parent classes by
re-declaring section with the same name.
Sections are translated to the
::
"formSections": {
"mySectionName1": {
"title": "text1",
"index": 0
},
"mySectionName2": {
"title": "text2",
"index": 1
}
}
entries in the root schema of the type or method.
Alternatives
------------
Instead of switching to json-schema we could generate UI definition in existing
(or improved) UI definition format.
Data model impact
-----------------
None
REST API impact
---------------
**GET /schemas**
Execute static MuranoPL method. Method must have a Public scope.
*Request*
+---------+--------------------------------+-------------------------------------+
| Method | URI | Description |
+=========+================================+=====================================+
| GET | /schemas/{className} | Obtain json-schema for class |
+---------+--------------------------------+-------------------------------------+
| GET | /schemas/{className}/{methods} | Obtain json-schema for class method |
+---------+--------------------------------+-------------------------------------+
Parameters:
* `className`: name of the class
* `classVersion`: version or version spec for the class. Optional. If not
provided then '=0' is assumed
* `packageName`: optional FQN of the package. If provided the class will only
looked up there instead of full catalog.
* `methods`: model builder method name or list of names which schemas are
requested. Empty string indicates schema of the class rather than of one of
its methods. If the parameter is absent then all the schemas (both class and
model builders) are returned.
*Response*
::
{
"": {
# json-schema for the class
},
"myModelBuilder1": {
# json-schema for the myModelBuilder1 method
}
}
HTTP codes:
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Schema was generated successfully |
+----------------+-----------------------------------------------------------+
| 400 | Bad request. |
+----------------+-----------------------------------------------------------+
| 401 | User is not allowed to access the schema |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified class or doesn't exist |
+----------------+-----------------------------------------------------------+
Versioning impact
-----------------
If we introduce more capabilities to the contracts then a new FormatVersion
should be introduced.
New murano-dashboard/python-client could still talk to an older API service
that lacks new API call and uses old UI definition alone in this case.
New approach is backward compatible so existing applications will still work.
Other end user impact
---------------------
A new python-muranoclient with a method to obtain json-schema for the class
will be required in order to take advantage on the MuranoPL forms.
Deployer impact
---------------
Maximum number of class implementations need to be specified in murano.conf
file. However it is going to have a reasonable default value.
Developer impact
----------------
In order to have rich GUI, an application developer will have to decorate all
his properties with lots of Meta values. Otherwise if no UI definition file
was provided the UI form may present input fields in a random order with a
labels set to a property name which is not very user friendly.
We should design a way to extract all visual hints into a separate per-class
file to separate them from the application code. This is a subject for another
spec.
Murano-dashboard / Horizon impact
---------------------------------
murano-dashboard should present user with the form constructed from a json
schema. The schema would contain all the required visual hints in the extra
attributes (not defined by the json-schema standard).
The dashboard may either construct the form on its own or use 3rd party
libraries that are capable to generate UI from the schema. In the later case
dashboard might become responsible for generating form definition - a structure
describing visual aspects of the form that is provided in addition to the
schema. Such structure might be produced by extracting required information
from extra attributes of the schema. The dashboard might also split it into
several forms/schemas in order to have a wizard UI rather than a single form.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Stan Lagun <istalker2>
Work Items
----------
* Create new API method;
* Write python-muranoclient method for new API call;
* Implement RPC method in murano-engine that will do schema generation;
* Write json-schema generator for the class/method;
* Define all the mentioned meta-classes and enhance schema generator to make
use of them.
Dependencies
============
* https://blueprints.launchpad.net/murano/+spec/static-actions
Testing
=======
All the path from the MuranoPL code to the rendered UI form can be tested
by the unit tests. Transformation from MuranoPL to json-schema can be tested
independently from the one from json-schema to the form definition or even
HTML layout.
Documentation Impact
====================
The following need to be documented:
* The new UI workflow.
* json-schema specification link along with description of extra fields added
by Murano;
* All changes made to the contracts;
* Standard Meta classes that can be used for visual hints in MuranoPL;
* Model builder methods documentation;
* Developers guide.
References
==========
* http://json-schema.org
* https://blueprints.launchpad.net/murano/+spec/static-actions

228
specs/newton/implemented/plugin-fqn-rename.rst
View File

@ -1,228 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=============================================
Change the naming scheme for MuranoPL plugins
=============================================
https://blueprints.launchpad.net/murano/+spec/plugin-fqn-rename
Current naming scheme for classes defined in MuranoPL extension plugins is not
consistent with the naming scheme for regular MuranoPL classes and packages.
This spec addresses this by changing the scheme and proposing a fallback logic
which will provide a backward compatibility for the existing MuranoPL
applications targeting old plugins.
Problem description
===================
MuranoPL classes and packages should have globally unique FQNs to prevent
potential name collisions when environment combines packages provided by
different developers. Recommended approach here is to start the name of a
package with a reversed Internet domain name given that the domain is
controlled by the developer of the package or class (such approach is inspired
by package naming rules of Java, see [1] for more details)
The examples of such class names may include:
* ``com.example.Foo`` - for demo applications and packages
* ``org.openstack.projectName.Foo`` - for applications and packages developed
and maintained by the teams of official Openstack projects
* ``com.companyname.Foo`` - for applications and packages developed and
maintained by a third party controlling the "companyname.com" domain name
* ``io.murano.Foo`` - for applications and packages developed and maintained
by the core murano team as part of the murano project. So, ``io.murano`` is
a preferred alias for longer ``org.openstack.murano`` FQN prefix.
However, currently this approach is not working for the classes defined by
MuranoPL extension plugins, since all the names of these classes have to start
with ``io.murano.extension`` prefix, as this is the requirement of the current
plugin management system. This breaks the naming convention and introduces the
possibility of name collision for plugin-defined classes.
Proposed change
===============
Current plugin system is based on stevedore library [2]. So we are utilizing
the concepts of `namespace` and `entry point name`. The namespace value
identifies the consumer of the plugin, the MuranoPL class plugins are
identified by the ``io.murano.extension`` stevedore namespace. Entry-point
names are the suffixes of the declared class' FQNs: the plugin loader forms the
actual FQN of the class by concatenating the stevedore namespace with entry
point name. For example, Murano's demo plugin [3] (providing Glance API
connectivity) declares the following stevedore-based extension:
::
[entry_points]
io.murano.extensions =
mirantis.example.Glance = murano_exampleplugin:GlanceClient
The developer's provided entry-point name is ``mirantis.example.Glance`` here,
and the current plugin loader will concatenate it with the namespace name thus
resulting in class ``io.murano.extensions.mirantis.example.Glance`` being
loaded by the class loader.
It is proposed to get rid of the concatenation: the stevedore's namespace name
should be used **only** to identify the type of the plugin, while the FQNs of
its classes should be defined only by the names of the entry points. So, when
this change is implemented the example above will load a class with an FQN
``mirantis.example.Glance``, without the ``io.murano.extensions`` prefix.
After this change is implemented the developers of the plugin will be able to
follow the regular naming rules of Murano. So, our example plugin will have to
be renamed to ``com.example.mirantis.Glance`` etc - and it will not be part of
the ``io.murano`` space, as this plugin is just an example and is not supposed
to be part of the core murano.
Backwards compatibility
-----------------------
The proposed change will effectively rename all the classes defined in the
existing plugins by removing the ``io.murano.extensions`` prefix from their
names. These may (and will) break all the MuranoPL code which relies on these
plugins.
To prevent this there should be a fallback logic implemented for the
class loader. When a class being loaded by name is not found and the name
starts with the ``io.murano.extensions`` prefix, there should be a second
attempt to load it by a shortened name without a prefix. If such class is found
and is loaded, a warning message should be logged to indicate that the plugin
is outdated and has to be updated with appropriate naming schema.
note::
This backwards-compatibility mode may be considered a way to deprecate the
old plugin naming scheme. In several releases this fallback logic may be
removed completely.
Alternatives
------------
We may postpone the change of the naming scheme till the next version of the
plugin system is introduced, which will some different format of plugins and
their content etc.
However postponing is bad, since the plugin development for murano is already
active, and having inconsistently named classes makes a bad example in the
community. Waiting for another cycle or two may greatly increase the number of
such bad examples.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Security impact
---------------
Because of the proposed change it will be now possible to define classes having
the names colliding with the Core Library and thus overriding the behavior of
the standard murano classes.
This should not be allowed unless the deployer explicitly allows that in the
configuration.
Deployer impact
---------------
Deployers will have to monitor the logs of Murano Engine for the Warning
messages produced by the loading of outdated plugins in the backwards
compatibility mode.
Developer impact
----------------
Plugin developers will have to update their plugin according the correct
naming scheme.
Application developers using the plugins will have to rename the class usages
in their MuranoPL code to use the new names.
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ativelkov
Other contributors:
ksnihyr
Work Items
----------
* Modify the plugin loader
* Add backwards compatibility mode to the class loader
* Update existing plugins to utilize the proper naming schema
* Update the documentation
Dependencies
============
None
Testing
=======
No new testing needed. Existing plugin-loading tests should be updated so no
warnings are displayed.
Documentation Impact
====================
Plugin development manual has to be updated.
References
==========
#. https://docs.oracle.com/javase/tutorial/java/package/namingpkgs.html
#. https://docs.openstack.org/stevedore/latest/
#. https://opendev.org/openstack/murano/src/tag/mitaka-eol/contrib/plugins/murano_exampleplugin/setup.cfg

394
specs/newton/implemented/plugin_networking_sfc.rst
View File

@ -1,394 +0,0 @@
===============================================
Support for Service Function Chaining in Murano
===============================================
One of the foundations for NFV enabled clouds is to have Networking Service
Function Chaining which provides an ability to define an ordered list of
network services which to form a “chain” of services. This could be used by
f.e. telcos to simplify management of their infrastructure.
Problem description
===================
Service chaining allow to pass traffic from endpoint A to endpoint B thru any
number of additional network service VMs. Those VMs are managed by tenant
admin.
With standard OpenStack networking traffic goes directly from A to B::
+--------+ +--------+
| VM A |-->| VM B |
+--------+ +--------+
With SFC traffic goes thru additional hops, which can perform any action
(firewall, LB, ...)::
+------+ +-------+ +-------+ +-------+ +------+
| | |Service| |Service| |Service| | |
| VM A |-->| VM A |-->| VM B |-->| VM C |-->| VM B |
+------+ +-------+ +-------+ +-------+ +------+
Those service VMs are transparent for end-user.
To achieve this, it would require to extend some of the OpenStack components.
Murano is suitable for defining such chains, but currently lacks of necessary features.
SFC use few entities to deliver chaining:
* Port pair
Represents vNIC (ingress/input, egress/output) attached to service VM.
Those vNICs will be used to receive (ingress) and send (egress) SFC traffic.
* Port group
Represents list of port pairs of different VMs with the same service function.
* Flow classifier
Used to classify network traffic, based on different metrics (e.g. source IP,
destination IP, source port, destination port, ...).
Only traffic which matches flow classifier will be handled by SFC.
* Port chain
Represents a releationship between flow classifiers and port groups.
To be able to use NFV functionality it is needed to have Service Chaining
functionality available in OpenStack.
Neutron does not support Networking SFC out of the box.
Currently Heat does not support configuration for Networking SFC API.
Currently Murano does not support Networking SFC management at all.
If we want to use Murano as a Resource Orchestrator it needs to have support
for configuring Service Chaining.
.. note::
Networking-SFC caveats:
* Current SFC implementation is not mature
* Supports only Open vSwitch
* API under heavy development
* Multiple API limitations
* No ready service VM software (capture SFC traffic, do *thing*, pass SFC
traffic to next hop)
Proposed change
===============
To solve problems above it is needed to introduce changes in Murano.
Having Murano as a Resource Orchestrator we need to have an ability to
configure Networking SFC from within Murano application templates.
This involves to implement changes in several areas.
Murano will be extended with:
* `Murano plugin`_ that will provide Networking SFC class which will be
a wrapper for lower level Neutron API.
* `Murano library package`_ that will provide Murano PL classes for
Networking SFC entities (e.g. PortPair, PortPairGroup, etc.)
Murano plugin
-------------
A python plugin that provides low-level wrappers for Neutron API functions
which are provided by Networking SFC extension. It implements direct method
wrappers for neutronclient library:
.. note:: Interface may change in the future
* ``createPortPair``
* ``deletePortPair``
* ``listPortPairs``
* ``showPortPair``
* ``updatePortPair``
* ``createPortPairGroup``
* ``deletePortPairGroup``
* ``listPortPairGroups``
* ``showPortPairGroup``
* ``updatePortPairGroup``
* ``createFlowClassifier``
* ``deleteFlowClassifier``
* ``listFlowClassifiers``
* ``showFlowClassifier``
* ``updateFlowClassifier``
* ``createPortChain``
* ``deletePortChain``
* ``listPortChains``
* ``showPortChain``
* ``updatePortChain``
Murano library package
----------------------
Application developer should have a possibility to configure Networking SFC
entities from Murano PL classes. We will introduce Murano plugin which will
extend to support configuration of following SFC classes:
* Port Pair
* Port Pair Group
* Flow Classifier
* Port Chain
All above parameters should be provided to successfully configure application
chaining. This should be done in Murano template for meta application
responsible for provisioning whole set of applications. Only mandatory SFC
parameters should be available for editing in Murano application UI.
PortPair class
~~~~~~~~~~~~~~
A Port Pair represents a service function instance. The ingress port and the
egress port of the service function may be specified. If a service function
has one bidirectional port, the ingress port has the same value as the egress
port.
**Properties:**
* ``id`` - Port pair ID
* ``name`` - Port pair name (Default: '')
* ``description`` - Readable description (Default: '')
* ``ingressPort`` - Ingress port (Reference to NeutronPort object)
* ``egressPort`` - Egress port (Reference to NeutronPort object)
.. note:: ingressPort and egressPort need to be changed to references once
Murano Core Library has support of network objects.
**Methods:**
* ``PortPair.deploy()`` - Create port pair object
* ``PortPair..destroy()`` - Cleanup resources (called implicitly)
**Example:**
.. code-block:: yaml
Namespaces:
=: com.example
netsfc: io.murano.extensions.networkingSfc
Name: DemoApp
Properties:
...
portPair: $.class(netsfc:PortPair).notNull()
Methods:
deploy:
Body:
- ...
- $.portPair.deploy()
PortPairGroup class
~~~~~~~~~~~~~~~~~~~
Inside each "port-pair-group", there could be one or more port-pairs.
Multiple port-pairs may be included in a "port-pair-group" to allow the
specification of a set of functionally equivalent SFs that can be be used for
load distribution, i.e., the "port-pair" option may be repeated for multiple
port-pairs of functionally equivalent SFs.
**Properties:**
* ``id`` - Port pair group ID
* ``name`` - Readable name (Default: '')
* ``description`` - Readable description (Default: '')
* ``portPairs`` - List of PortPair class objects.
**Methods:**
See `PortPair class`_
Example:
.. code-block:: yaml
Name: DemoApp
Properties:
...
portPairs: [$.class(netsfc:PortPair).notNull()]
Methods:
deploy:
Body:
- ...
- $.portPairGroup: new(netsfc:PortPairGroup, portPairs => $.portPairs)
- $.portPairGroup.deploy()
FlowClassifier class
~~~~~~~~~~~~~~~~~~~~
Flow classifiers are used to select the traffic that can access the chain.
Traffic that matches any flow classifier will be directed to the first port
in the chain. The flow classifier will be a generic independent module and
may be used by other projects like FW, QOS, etc.
A flow classifier cannot be part of two different port-chains otherwise
ambiguity will arise as to which chain path that flow's packets should go.
A check will be made to ensure no ambiguity. But multiple flow classifiers
can be associated with a port chain since multiple different types of flows
can request the same service treatment path.
**Properties:**
* ``name`` - Classifier name
* ``ethertype`` - IPv4 (default) / IPv6
* ``protocol`` - IP protocol
* ``sourcePortRange`` - Source protocol port; port number or tuple of min and
max ports
* ``destinationPortRange`` - Destination protocol port; port number or tuple of
min and max ports
* ``sourceIpPrefix``
* ``destinationIpPrefix``
* ``logicalSourcePort`` - neutron source port (required)
* ``logicalDestinationPort`` - neutron destination port (required)
* ``l7Parameters`` - dictionary of L7 parameters (not used)
**Methods:**
See `PortPair class`_
PortChain class
~~~~~~~~~~~~~~~
A Port Chain is a directional service chain. The first port of the first
port-pair is the head of the service chain. The second port of the last
port-pair is the tail of the service chain. A bidirectional service chain would
be composed of two unidirectional Port Chains.
**Properties:**
* ``id`` - Port chain ID
* ``name`` - Readable name
* ``description`` - Readable description
* ``portPairGroups`` - List of PortPair class objects
* ``flowClassifiers`` - List of FlowClassifier class objects
**Methods:**
See `PortPair class`_
Delivering Murano extensions
----------------------------
Murano uses stevedore [0] library to support plugins. Basically stevedore
plugin is a python package with special metadata (plugin entry points).
It can be installed as python package (via ``pip``) or distribution specific
package (e.g. DEB, RPM, etc.). Murano plugins provides new classes
that are referenced in the plugin metadata.
Murano library package is a simple **zip** archive which contains
``metadata.yaml`` file and Murano PL classes.
It can be installed from file using ``murano`` command-line tool
or from the repository.
Alternatives
------------
Extend Heat to support Networking SFC.
Instead of accessing Neutron API directly from Murano plugins, Murano can
manage Networking SFC through Heat resources. It will require creating Heat
extension which will delegate SFC configuration from Heat templates.
Heat extension should provide set of resources which will implement.
Networking SFC entities:
* Port Chain
* Port Pair Group
* Port Pair
* Flow Classifier
Heat can be extended by adding new resources using plugins. It provides mapping
between heat resources and python classes. It will implement hooks on heat
events like resource create, update etc.
**PROS:**
* Resource management delegated to Heat. Heat can be used for resource cleanup
* Heat can provide some helpful transactional capabilities
* Cloud operator will be able to use SFC w/ and wo/ Murano
**CONS:**
* Security-related implications for customers, who wants to keep vanilla
OpenStack installation.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
End user will be able to use networking SFC from Murano templates.
Networking SFC will introduce additional delay in RTT (round-trip-time).
This mean some applications prone to network delays can stop working.
Deployer impact
---------------
To use SFC with Murano cloud operator needs to install and enable SFC Murano
plugin.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Work Items
----------
* Create Murano Networking SFC plugin
* Create Murano library package
* Create Murano test application
* Implement support of multiple Neutron ports to Murano Core Library
* Tests
Dependencies
============
Murano plugin depends on networking-sfc package, which should be installed.
Testing
=======
Proper functional tests should be implemented.
Documentation Impact
====================
User will be informed how to enable and start using networking SFC inside
Murano templates.
References
==========
[0] - http://docs.openstack.org/developer/stevedore/

385
specs/newton/implemented/static_actions.rst
View File

@ -1,385 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==============
Static actions
==============
https://blueprints.launchpad.net/murano/+spec/static-actions
Currently Murano provides a way to invoke MuranoPL methods through the
actions API. However it doesn't cover all the use cases where one may want
to expose some MuranoPL code. This specification further improves the idea
of actions.
Problem description
===================
Existing actions approach has a number of limitations:
* Only instance methods can be exposed as actions. In other words you need
to have a class instance in order to call a method on it. And instance means
it has to be a part of object model. This makes impossible to expose MuranoPL
code that doesn't operate on object model or even produces it like factory
methods;
* Actions bring large overhead and complexity for simple methods. For each
action call a database record is created and never deleted then. All action
results are persisted as well;
* No two actions in the same environment can be called simultaneously.
All of above is not a problem for regular application actions that usually
involve deployment steps and can take long time to complete. However it makes
actions a bad choice in case when one just wants to expose some application
code to outside to keep all of his code in MuranoPL in single place.
Proposed change
===============
In order to overcome above issues it is proposed to add ability to expose and
call static MuranoPL methods through new API and engine RPC calls. Because
the methods are static they are not bound to any object model (environment)
and thus can be called simultaneously.
Also because now there are two types of action methods (static and instance)
the fact that method needs to be exposed cannot be expressed with the Usage
attribute of the method (static methods already have `Usage: Static`).
We could introduce yet another usage (e.g. `StaticAction`). But because action
means that regular MuranoPL method need to be exposed to outside and it has
much more to do with a method visibility rather then its usage it is proposed
to introduce another method keyword to declare its scope.
Thus current syntax for the action declaration
::
methodName:
Usage: Action
will become
::
methodName:
Scope: Public
and static actions will be declared as
::
methodName:
Usage: Static
Scope: Public
`Scope` attribute must be one of the two possible values:
* `Session` - regular method that is accessible from anywhere in the current
execution session. This is the default if the attribute is omitted;
* `Public` - accessible anywhere, both within the session and from outside
through the API call (i.e. it is an action in current terminology).
In the future new scopes might be added to further restrict the access
to the method (e.g. `Package`, `Class`)
For backward compatibility `Usage: Action` should remain valid and acceptable
but be deprecated for future format versions. An exception must be raised if
both `Usage: Action` and `Scope: Session` are specified.
Synchronous and asynchronous actions
------------------------------------
Current instance actions are always asynchronous and this is achieved by means
of API with action result being persisted into database. For static methods
that doesn't usually do any deployment and much more suitable for quick
request-response interactions that do not modify object model this would be
an overkill that greatly reduces their performance. However if we make
static actions synchronous this will cause confusion because we get two
completely different usage pattern for method that differ only by the fact
that one is static and another is not. This is especially confusing due to the
fact that one can easily imagine synchronous instance actions and asynchronous
static ones. Thus it is proposed to:
#. On MuranoPL side all actions are synchronous i.e. they are plain methods
that return the result;
#. RPC calls for action execution are always asynchronous. I.e. the call to
invoke the method and call to notify API on the method result are two
different calls rather than single request-response. However the request
may indicate that the response must be sent to specific API instance to
make synchronization on the API without database persistence possible.
#. On python-muranoclient side all actions are asynchronous in terms that you
get a Future object from which one can get result later as well as check for
its readiness. For synchronous call the result becomes available immediately.
However from the user's perspective all methods are asynchronous.
Current python-muranoclient method for action execution returns action ID
that can be later provided to another method rather that a Future object.
For consistency it is proposed to change that method to return Future
instead. For backward compatibility the method that accepted action ID
must accept task Future too.
#. It is the API who decides if the method is going to be executed
synchronously or not. In the former case API does RPC request to the engine
without creating database records and instead waits for response that
must be sent to that API instance. As soon as result is available it is
returned to the caller (python-muranoclient). For now the API is going
to do synchronous execution for static actions and asynchronous for
instance actions. However in the future we may consider making it
configurable or introduce some level of automation when API can wait for
some time and then switch to asynchronous mode (create database record
and so on) if there is still no result.
Alternatives
============
* Use actions instead for cases where static methods will be more appropriate;
* Declare static actions with yet another Usage value or make Usage be a list
([Static, Action])
* Use Meta to declare actions
Data model impact
=================
None
REST API impact
===============
**POST /actions**
Execute static MuranoPL method. Method must have a Public scope.
*Request*
+----------+----------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==================================+==================================+
| POST | /actions | Execute static action |
+----------+----------------------------------+----------------------------------+
Body:
::
{
"className": "my.class.fqn",
"methodName": "myMethod",
"packageName": "optional.package.fqn",
"classVersion": "1.2.3",
"parameters": {
"arg1": "value1",
"arg2": "value2"
}
}
packageName is optional. If provided the class must be in that package.
classVersion is optional. If provided it may contain either full SemVer version
or version spec (e.g. ">1.2"). If omitted "=0" is assumed.
*Response*
JSON-serialized method response or exception is returned.
HTTP codes:
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Action was executed successfully |
+----------------+-----------------------------------------------------------+
| 400 | Bad request. Either the format of the body is invalid or |
| | parameters doesn't match the contracts |
+----------------+-----------------------------------------------------------+
| 403 | User is not allowed to execute the action |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified class or method doesn't exist |
| | or not exposed |
+----------------+-----------------------------------------------------------+
| 503 | Unhandled exception in the action |
+----------------+-----------------------------------------------------------+
Versioning impact
=================
None
Other end user impact
=====================
python-muranoclient must be updated to provide method for the new API call.
Deployer impact
===============
None
Developer impact
================
Existing MuranoPL applications shouldn't be affected by the change.
For the next package format version old syntax of action declaration
should still work but produce a deprecation warning.
Murano-dashboard / Horizon impact
=================================
None
Implementation
==============
Assignee(s)
-----------
Stan Lagun <istalker2>
Work Items
----------
* Update DSL to make use Scope method attribute
* Update code that inspects if method is an action for the new syntax
(including object model serializer)
* Implement new RPC call in murano-engine
* Implement API call
* Implement method in python-muranoclient
Dependencies
============
None
Testing
=======
RPC call implementation can be tested by unit tests alone.
However to test the new API several tempest tests need to be added:
* Test that executes sample MuranoPL static action (hello world)
* Test that executes method that throws an exception
* Test that executes method with invalid parameters
Documentation Impact
====================
* Documentation for the actions need to be updated
* Developer's guide need to be extended with static actions info
* REST API specification need to be updated with new call info
References
==========
None
Versioning impact
-----------------
None
Other end user impact
---------------------
python-muranoclient must be updated to provide method for the new API call.
Deployer impact
---------------
None
Developer impact
----------------
Existing MuranoPL applications shouldn't be affected by the change.
For the next package format version old syntax of action declaration
should still work but produce a deprecation warning.
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Stan Lagun <istalker2>
Work Items
----------
* Update DSL to make use Scope method attribute
* Update code that inspects if method is an action for the new syntax
(including object model serializer)
* Implement new RPC call in murano-engine
* Implement API call
* Implement method in python-muranoclient
Dependencies
============
None
Testing
=======
RPC call implementation can be tested by unit tests alone.
However to test the new API several tempest tests need to be added:
* Test that executes sample MuranoPL static action (hello world)
* Test that executes method that throws an exception
* Test that executes method with invalid parameters
Documentation Impact
====================
* Documentation for the actions need to be updated
* Developer's guide need to be extended with static actions info
* REST API specification need to be updated with new call info
References
==========
None

668
specs/newton/implemented/validation-tool.rst
View File

@ -1,668 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===============================================
Validation tool for Murano Application Packages
===============================================
https://blueprints.launchpad.net/murano/+spec/murano-app-validation
Murano doesn't have validation mechanisms for MuranoPL language and
application packages.
This spec proposes a validation toolkit that improves workflow
by making error detection possible at the early stages of application
development lifecycle.
Problem description
===================
Early error detection simplifies application development,
troubleshooting and support.
Currently Murano does not provide mechanisms to perform validation of the
Murano application package before running deployment process.
Thus Murano cannot prevent uploading of invalid package and developer
cannot verify that application package contains valid structure
and code before deployment process.
Many problems with application packages (such as invalid package structure,
errors in code, typos, etc.) can be detected and solved by using
static validation tool.
Proposed change
===============
Introduce a new toolkit for validation MuranoPL language and Murano application
packages.
* Validation toolkit SHOULD be implemented as a standalone Python package
(library) with CLI interface.
* Validation library will be used to validate Murano packages during loading
process.
* CLI tool can be used by application developer to perform early checks before
package is assembled.
* Toolkit should be extensible with custom validators. It will provide API for
extension purposes.
* Toolkit should be configurable to run list of specified inspections
or to ignore specified inspections.
Toolkit should implement the following validation suites:
* Manifest validation
* Classes validation
* UI definition validation
Other validators will be implemented later as part of this tool or as external
packages
Manifest validation
-------------------
* ``manifest.yaml`` file should exist and should be a valid YAML document.
* ``manifest.yaml`` structure should respond to manifest structure described in
:ref:`manifest-structure-section`.
* All files referenced in ``Classes`` section should exist in ``Classes``
subdirectory and should be valid YAML documents.
* Warning should be reported in case when additional files exist in ``Classes``
subdirectory but not referenced in the manifest file.
* UI definition file should exist for Application package (MuranoPL =< 1.4)
Class validation
----------------
* Valid classes and methods should check if properties are defined
correctly and there are no extra keys that are not supported.
* Check if class is in correct namespaces according to manifest and
class name matches.
* Contract fields should be valid YAQL expressions.
* Methods blocks should be checked for MuranoPL code structure correctness.
Method blocks may consist only of Expressions, Assignments and Block
constructs. See `full terms description <http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_pl.html#method-body>`_
for details.
* (Optional) Detect code that cannot be parsed as YAQL expression but is
considered to be a valid expression.
UI validation
-------------
* UI definition should contain ``Application`` and ``Forms`` sections
* YAQL expressions in UI definition should be valid
* ``Form`` section should contain only valid forms
`<http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#forms>`_
Errors
------
Error code consists of prefix and error number. Prefix is an upper case
alphabetical sequence. It can be empty for general errors. Errors specific for
some validator should use validator short name as prefix. Error number is an
upper case alphanumerical sequence (one single letter ``E`` or ``W``
following by three digits). Examples: ``E007``, ``MPL:E042`` for errors and
``W777``, ``UI:W123`` for warnings. Validation tool should have ability to
produce documentation of supported errors and warnings.
Package structure reference
---------------------------
http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#package-structure
Package consists of:
* Manifest - File in package root directory named ``manifest.yaml``.
* MuranoPL classes - YAML files located in ``Classes`` directory or its
subdirectories (optional).
* UI definition - YAML file in ``UI`` directory (optional).
* Resources - files in ``Resources`` directory (optional).
* Logo (optional)
* images.lst file (optional)
.. _manifest-structure-section:
Manifest structure
^^^^^^^^^^^^^^^^^^
**Format**
Application package format. Can be specified as ``<FORMAT>/<VERSION>``,
where format is a format name string (can be ``MuranoPL`` or ``Heat.HOT``).
If format is omitted, ``MuranoPL`` is used by default.
* Required: no
* Default: ``MuranoPL/1.0``
* Format: ``<FORMAT>/<VERSION>``
**Type**
* Required: yes
* Type: Enum(Application, Library)
**Name**
Package human readable name
* Required: no
* Type: String
**FullName**
Package fully qualified name. Should be in lowerCamelCase notation
separated by dots.
* Required: yes
* Type: String
**Version**
Version of package
* Required: no
* Default: ``0.0.0``
* Type: String, should have valid `SemVer format <http://semver.org/>`_
**Description**
* Required: no
* Type: String
**Author**
* Required: no
* Type: String
**Tags**
* Required: no
* Type: List(String)
**Classes**
* Required: no
* Type: Dict
- Keys: String, Fully qualified class name
- Values: String, Filename relative to ``Classes`` directory
**Require**
Package external requirements.
* Required: no
* Type: Dict
- Keys: Dependent package ``FullName``
- Values: ``null`` or string with exact version ('1', '2.3', '4.5.6') or
valid spec_strings according to `python-semanticversion docs <https://python-semanticversion.readthedocs.io/en/latest/reference.html#semantic_version.SpecItem>`_
**UI**
File with package UI definition
* Required: yes (for package type ``Application``)
* Type: String, Filename relative to root directory
* Default: ``logo.png``
**Logo**
Package logo in png format.
* Required: no
* Type: String, Filename relative to ``UI`` directory
* Default: ``ui.yaml``
**Meta**
* Required: no
* Type: Dict or List(Dict) with metadata information
Class structure
^^^^^^^^^^^^^^^
**Name**
* Required: yes, if more than one class defined in the file
* Type: String
* Format:
- Matches Python class name format, except leading
double underscore (``__ClassName``).
- Matches CamelCase naming style.
**Properties**
* Required: no
* Type: Dict
For more details see :ref:`class-properties-section`.
**Methods**
* Required: no
* Type: Dict
For more details see :ref:`class-methods-section`.
**Extends**
Single or multiple class names.
* Required: no
* Type: String | YAML List
**Namespaces**
* Required: no
* Type: Dict
* Keys: String; Namespace alias
* Values: String; Namespace full qualified name
* Format:
- Namespace alias is a valid YAQL identifier or ``=``.
- Namespace FQN is a string divided by dots (``.``).
- Recommendation is to use lowerCamelCase naming style.
**Import**
* Required: no
* Type: String, Class name, List(Class names)
**Usage**
* Required: no
* Type: Enum(Class, Meta)
* Default: ``Class``
**Meta**
* Required: no
* Type: Dict or List(Dict) with metadata information
MetaClass structure
^^^^^^^^^^^^^^^^^^^
In addition to regular Class structure, MetaClass (Class with Usage: Meta) can
have additional properties:
**Cardinality**
* Required: no
* Type: Enum(One, Many)
**Applies**
* Required: no
* Type: Enum(Package, Type, Method, Property, Argument, All) or List(Enum)
**Inherited**
* Required: no
* Type: boolean
.. _class-properties-section:
Class properties
^^^^^^^^^^^^^^^^
**Contract**
* Required: yes
* Type: YAQL Expression
**Usage**
* Required: no
* Type: Enum
========= ====================
Usage
========= ====================
In Default
Out
InOut
Const
Runtime
Config Since version 1.1
Static Since version 1.2
========= ====================
**Default**
* Required: no
* Type: Any value
**Meta**
* Required: no
* Type: Dict or List(Dict) with metadata information
.. _class-methods-section:
Class methods
^^^^^^^^^^^^^
**Scope**
.. versionadded:: 1.4
* Required: no
* Type: Enum
* Values: Public, Session
**Arguments**
* Required: no
* Type: List(Dict)
For more details see :ref:`methods-argument-section`.
**Body**
* Required: no
* Type: YAQL Expression | List(YAQL Expression)
**Usage**
* Required: no
* Type: Enum
========= ====================
Usage
========= ====================
Runtime Default
Static Since 1.3
Extension Since 1.3
Action Deprecated since 1.4
========= ====================
**Meta**
* Required: no
* Type: Dict or List(Dict) with metadata information
.. _methods-argument-section:
Method's Argument
^^^^^^^^^^^^^^^^^
* Type: Dict(String, Dict)
* Key: Argument name
**Values:**
* Contract: YAQL Expression
* Default: YAQL Expression
* Usage: Enum added since 1.4
========= ====================
Usage
========= ====================
Standard Default
VarArgs
KwArgs
========= ====================
**Meta**
* Required: no
* Type: List with metadata information
Extensions
----------
Validation tool should be extensible and provide unified mechanism for builtin
checks and extensions.
Extensions should be implemented as a python package and registered using
entry points.
Extensions are loaded using
`stevedore <http://docs.openstack.org/developer/stevedore/>`_ library.
CLI Usage
---------
::
Usage: mpl-check [options] <PACKAGE | DIRECTORY>
Arguments:
DIRECTORY Application working directory (e.g. contains
manifest.yaml, Classes/, etc.)
PACKAGE Compressed application (ZIP-archive)
Options:
--ignore=errors Skip errors and warnings.
--select=errors Check only for selected errors and warnings.
--scope=validators List of validators to execute
--strict Consider warnings as an errors.
Alternatives
------------
Implement validation as part of Murano and validate the package
when it is uploaded to Murano.
Pros:
- Easier to implement.
Cons:
- Suitable for Murano itself, but not for developers.
- Integration with CI which requires execution of functional
tests and is slower than static validation.
- Cannot be integrated with text editors or IDEs.
Data model impact
-----------------
None
REST API impact
---------------
Murano API for upload packages should use this tool to do package validation
before store it. If package is not valid, HTTP error with code 400 should be
returned to user with list of check errors in description.
Versioning impact
-----------------
None
Other end user impact
---------------------
* Murano application package developer will be able to validate package
during development process.
Deployer impact
---------------
None
Developer impact
----------------
Application developer will be able to validate application code
during development process without importing package to Murano
and running deployment process or test suite.
Murano-dashboard / Horizon impact
---------------------------------
Error message is displayed in Horizon when trying to upload
malformed package.
Implementation
==============
Application class
-----------------
Application class provides an entry point to the checker.
It is responsible for configuration, extensions loading and validation
execution.
Application options are registered using oslo.config
**Methods:**
* __init__(self, options)
* ``load_plugins(self)`` - Finds validators from installed plugins
* ``prepare_check_suit(self, loaded_package, validator_list)`` - Collect
prepared checks from all validators in validator_list against specified
package. Returns list with prepared checks.
* ``list_errors(self, filters)``- Lists errors/warnings exposed by all available
validators
Validator classes
-----------------
Validator class represents set for specific checks and logic to register them.
**Classes**
::
BaseValidator
+-- PackageValidator
+-- ManifestValidator
+-- UiValidator
+-- ClassValidator
**Attributes**
* ``short_name`` short name for validator. Used as prefix for validator specific
errors
**Methods**
* ``__init__(loaded_package)`` - Init validator for specific format
* ``run(self)`` - Runs validator against specified suit of files or
whole package if file_suit=None (for Package validator)
Checkers
--------
Checker is a function that performs checking logic. It takes data from args
and yields error if it is not valid. Checkers should not contain logic for
data preparation. Checker can be reused multiple times with different options
in a single suit of checks. Checker can be wrapped with custom filters
(i.e. manifest version), so it is called only for matching conditions.
Exceptions
----------
Class ``CheckerError`` contains information including error code, message,
file name and position in that file. Additionally it can include
code snippet, if it's available. For YAML data it's available using
``yaml.error.Mark`` class. Otherwise code snippet can be generated during
error formatting.
**Attributes**
* code - Error code (Usually single letter and three digits)
* message - Human readable error reason
* filename - File name relative to package root
* line - Line number (starts from 0, optional)
* column - Column number (starts from 0, optional)
* source - One line code snippet with highlighted column (optional)
Package loaders
---------------
Package loaders provide interface to Murano packages in multiple formats.
Currently packages can be loaded from directory, single file or zip archive
package.
::
PackageLoader
+-- DirectoryLoader
+-- FileLoader
+-- ZipLoader
**Methods**
* ``try_load(cls, path, *options)`` - [classmethod] Tries to load package, if it
can be loaded returns instance of loaded package, otherwise None.
Loaded package
^^^^^^^^^^^^^^
Loaded package class introduces a general interface for access to packages of
different format.
**Methods**
* ``get_file(self, name)`` - File wrapper if file exists, None otherwise
* ``format(self)`` - Ru
* ``exists(self, name)`` - Returns true if file exists in the package.
* ``list(self, subdir=None)`` - Returns list of all files in the package
(or in it's subdirectory).
* ``search_for(self, regex)`` - Return iterator of files fit to regex
File wrapper allows to access to raw file context with method ``raw()`` and
to parsed dict with method ``parsed()``
Formatters
----------
Formatter classes help to represent error report in various ways.
Errors can be printed to ``stdout``/``stderr`` using text representation,
which suits developer or written to file in ``JSON``/``YAML`` format.
Assignee(s)
-----------
Primary assignee:
Alexander Saprykin <cutwater>
Other contributors:
Krzysztof Szukielojc <kszukielojc>
Sergey Slipushenko <sslypushenko>
Work Items
----------
#. Develop core framework with support of extensions.
#. Implement manifest and package structure checks.
#. Implement Murano PL classes checks.
#. Implement UI definition file validation.
#. Implement YAQL checks.
#. Integrate package validation with murano API.
#. Develop CLI tool based on core framework
#. Create documentation.
Dependencies
============
* `PyYAML <http://pyyaml.org/>`_
* `stevedore <http://docs.openstack.org/developer/stevedore/>`_
Testing
=======
Package should include unit and functional tests.
Documentation Impact
====================
Documentation for the library should include:
* Usage information that describes how to use the application.
* API reference.
* Plugin developer documentation.
References
==========
None

669
specs/ocata/approved/application-policies.rst
View File

@ -1,669 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
====================
Application policies
====================
https://blueprints.launchpad.net/murano/+spec/application-policies
The spec proposes mechanism that cloud operators can use to impose
constraints on applications and alter application behavior without
making modifications to the source code of the apps.
Problem description
===================
Cloud operators often want to have control over what Murano applications can
or cannot do, impose limitations on environment content or alter applications
behavior somehow in a way that can be applied to a wide range of applications.
Possible policies that operator might want:
* Have a black list of applications (that either has explicit application names
or all apps in particular namespace) including their inheritors for
particular tenant
* Make each deployed environment have particular application
* Want particular script to be executed on each VM spawned regardless of the
image being used by the application
* Want to have quota on the number of instances that can be created by a single
application
* Do not allow to use large instance flavors for all applications except for
explicitly approved
* Change application defaults so that particular promoted image will become
the default value in UI for all applications that work on this OS
* Send notification when number of spawned instances in particular environment
exceeds quota
* Change default network topology for particular tenant
* Implement custom billing based on the application activities without
requiring all applications to emit notifications for them.
* Honor policies from the third party policy engine
As you can see all of the policies above are very different in their nature,
require different inputs and different ways to apply them. Also there can
be much more use cases and their variants in order have upfront support for
all of them.
Thus Murano needs a generic way to write policy enforcement rules and actions.
Proposed change
===============
Each policy case consists of two parts: rules specification and code that
applies them.
The code is going to be written as MuranoPL classes that implement required
interface. These classes are called aspects. Each aspect captures and applies
one policy pattern.
Policy rules are going to be the input for those classes (their property
values, similar to our regular object model).
Examples of such input might be:
* Endpoint of the third party policy engine
* Name of the image
* List of YAQL predicates (policy rules)
* Parameters of the largest flavor possible
So each instance of aspect class provided with the values for the rules
parameters can be called a policy as it contains both parts of the policy
definition.
Aspects are provided using the same Murano package format and reside in the
same catalog as applications are. Aspects are going to be located in the
catalog in the packages of the new type ``Aspect``. All aspect classes will be
inherited from the base ``Aspect`` class which should implement some basic
interface for aspects as well as some boilerplate code.
Policy UI workflow
------------------
To use policies, admin should execute the following workflow in UI:
* Import packages with aspect classes (usual package upload).
* Create policies by adding aspects to the list of the ones he wants to apply
and specifying the policy rules as an input parameters. The same UI workflow
(including dynamic UI parts) that is used for adding regular Murano
applications to environments is used here. The difference is that no
environment is needed.
* Apply aspects for particular tenants or globally. New type of GUI form must
be created for that but it should be quite simple.
Instances of aspects, a.k.a policies (i.e. their serialized representation) are
going to be created independently from environments and stored in a dedicated
table in the database. Terms "aspect instance" and "policy" can be used
interchangeable further in this spec to make accent on the one or another shade
of the meaning.
In order for aspects to do their job they are going to be provided with
additional capabilities that are not available to the rest of MuranoPL apps.
Aspects event hooking
---------------------
Aspects should have ability to hook themselves to various system events
produced by the engine. These could be:
* Object model load events
* Package load events
* Object creation events that happen before/after new() function is used
anywhere in the code
* Garbage collection events
In the initial implementation it is proposed to introduce hooking just to
object creation and package load events. It covers the most important use
cases of policies (changing MuranoPL class code and changing object property
values).
Two new MuranoPL classes need to be added to the Murano core library:
* ``io.murano.policy.Aspect`` - the basic interface and boilerplate code
provider for all custom aspects. It has the following methods:
+ ``subscribe(notifier)`` - method which invokes notifier's ``subscribe``
method passing the calling aspect, event name and aspect's method names
as arguments. It subscribes aspect to notifications from notifier.
Subscribes to all events in the base ``Aspect`` class and can be
overridden by inheritor not to subscribe to some events or to subscribe
to some event several times with different conditions and handlers.
This method is called by the engine right after aspects and notifier
objects initialization
+ *condition methods* - methods which define the condition needed to be
checked against some item (e.g. initializing object in case of object
init event, class in case of package load event) to determine whether
event handler method of the aspect should be called with that item. This
group of methods include ``objectInitCondition(object)`` and
``packageLoadCondition(class)``. ``modelLoadCondition(model)`` can be
added later. These methods return ``false`` in the base class which means
no item is handled. It should be overridden by inheritor to specify
actual condition. For example, ``objectInitCondition`` can check that
object is the instance of particular class or that it has some property
+ *handling methods* - methods which apply policy rules to some item (e.g.
initializing object in case of object init event, class in case of
package load event) when corresponding event occurs. So it is the core
part of the policy. This group of methods include
``handleObjectInit(object)`` and ``handlePackageLoad(class)``.
``handleModelLoad(model)`` can be added later. The body of these methods
is empty in the base class. Other handling methods can be also added and
used in the inheritor class
* ``io.murano.policy.Notifier`` - the class to manage notifications of
the proper aspects about system events. It has one property ``_handlers``
which holds the dict with event names as keys and lists of event handlers
as values. Each handler is a dict with aspect object, its condition method
name and handling method name. ``Notifier`` has the following methods:
+ ``subscribe(subscriber, eventName, methodName, conditionMethodName)`` -
method to populate notifier's ``_handlers`` dict. It is called by each
aspect that wants to subscribe to some event
+ ``callHandler(handler, item)`` - method that firstly invokes
conditionMethod stated in the handler dict, in case it returns ``true``
for the item, invokes handling method with the item
+ ``onEvent(event, item)`` - method that invokes ``callHandler`` for all
handlers under the ``event`` key of ``_handlers`` dict. This method is
invoked by engine when the event occurs
Extended MuranoPL reflection
----------------------------
In addition to standard MuranoPL reflection aspects must be provided with
methods that can modify MuranoPL code model:
* change property declarations including default values and contracts
* change methods: wrap it in another method, change argument contracts and
default values
Also aspect may want to modify the metadata which defines UI layout of the
application. For example, add ``io.murano.metadata.forms.Hidden`` class to
some property along with the inserting some predefined default value for this
property. It will result in hiding the corresponding field in GUI and using
the defined value. This part can be used once the new dynamic UI generation
from schema is introduced.
These capabilities can be written as new MuranoPL methods similar to ordinary
reflection and provided to aspects by registering them to the package context
of ``Aspect`` packages. Thus, other MuranoPL code will be unable to make use
of these methods.
The new methods should include ``setDefault``, ``setContract``, ``addMeta``,
``wrapMethod``.
Also the ability to set property values for other object should be provided to
aspects by setting ``CTX_ALLOW_PROPERTY_WRITES`` flag to ``true`` in the
``Aspect`` packages context.
Changes to engine workflow
~~~~~~~~~~~~~~~~~~~~~~~~~~
With all that, the engine workflow with policies is the following:
#. User adds package to the environment
#. The class schema generation in engine triggers package load event
#. Engine obtains a list of policies that need to be applied for particular
tenant (aspects and their input parameters)
#. Engine instantiates and initializes all those aspects and a notifier
#. Engine subscribes aspects to notifications from notifier
#. Engine loads the package and invokes ``Notifier``'s method
``onEvent(packageLoad)``.
#. ``Notifier`` finds out what aspects want to modify the package and invokes
their ``handlePackageLoad()`` method.
#. Handler gets a chance to examine the code of the MuranoPL class and make
necessary modifications using extended reflection capabilities.
# During the model load, objects initializing, garbage collection engine
instantiates aspects and notifier again and runs the process of
subscription => notification => modification with these events in a similar
fashion.
Alternatives
------------
None
Data model impact
-----------------
Model to store policies should look roughly like this:
Table name: policy
+------------------+--------------+------+-----+---------+
| Field | Type | Null | Key | Default |
+==================+==============+======+=====+=========+
| created | datetime | NO | | NULL |
+------------------+--------------+------+-----+---------+
| updated | datetime | NO | | NULL |
+------------------+--------------+------+-----+---------+
| id | varchar(255) | NO | PRI | NULL |
+------------------+--------------+------+-----+---------+
| object_model | longtext | YES | | NULL |
+------------------+--------------+------+-----+---------+
| description | text | YES | | NULL |
+------------------+--------------+------+-----+---------+
object_model field stores json-serialized representation of the policy
including its rules.
description field contains optional textual information.
Model to store policies assignment to tenants:
Table name: policy_assignment
+------------------+--------------+------+-----+---------+
| Field | Type | Null | Key | Default |
+==================+==============+======+=====+=========+
| created | datetime | NO | | NULL |
+------------------+--------------+------+-----+---------+
| updated | datetime | NO | | NULL |
+------------------+--------------+------+-----+---------+
| policy_id | varchar(255) | NO | PRI | NULL |
+------------------+--------------+------+-----+---------+
| tenant_id | varchar(36) | YES | PRI | NULL |
+------------------+--------------+------+-----+---------+
NULL value of the tenant_id field means that policy should be applied to all
tenants in the cloud.
REST API impact
---------------
+---------------------+------------+-----------------------------------------+
| Attribute | Type | Description |
+=====================+============+=========================================+
| objectModel | object | JSON representation of policy |
+---------------------+------------+-----------------------------------------+
| tenantIds | array | Array of the tenants to apply policy to |
+---------------------+------------+-----------------------------------------+
**List policies**
*Request*
+----------+------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==============================+==================================+
| GET | /policies | List available policies |
+----------+------------------------------+----------------------------------+
*Response*
List of policies with their basic properties.
::
{
"policies": [
{
"created": "2014-05-14T13:02:46",
"updated": "2014-05-14T13:02:54",
"id": "2fa5ab704749444bbeafe7991b412c33",
"description": "Policy to limit possible flavor",
"tenant_ids": ["726ed856965f43cc8e565bc991fa76c3"]
},
{
"created": "2014-05-14T13:02:51",
"updated": "2014-05-14T13:02:55",
"id": "744e44812da84e858946f5d817de4f72",
"description": "Policy to restrict access to apps",
"tenant_ids": ["726ed856965f43cc8e565bc991fa76c3"]
}
]
}
Response codes:
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. List of policies received successfully |
+----------------+-----------------------------------------------------------+
| 403 | User is not allowed to browse policies |
+----------------+-----------------------------------------------------------+
**Create policy**
*Request*
+----------+------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==============================+==================================+
| POST | /policies | Create policy |
+----------+------------------------------+----------------------------------+
Body:
::
{
"description": "Policy to limit possible flavor",
"objectModel": {
"maxFlavor": "m1.medium",
"?": {
"type": "com.example.MyAspect",
"id": "446373ef-03b5-4925-b095-6c56568fa518"
}
}
}
*Response*
::
{
"id": "ce373a477f211e187a55404a662f968",
"description": "Policy to limit possible flavor",
"created": "2013-11-30T03:23:42Z",
"updated": "2013-11-30T03:23:44Z",
"objectModel": {
"maxFlavor": "m1.medium",
"?": {
"type": "com.example.MyAspect",
"id": "446373ef-03b5-4925-b095-6c56568fa518"
}
}
}
Response codes:
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Policy has been created successfully |
+----------------+-----------------------------------------------------------+
| 400 | Bad request. Either the format of the body is invalid or |
| | parameters doesn't match the contracts |
+----------------+-----------------------------------------------------------+
| 403 | User is not allowed to create policies |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified class doesn't exist or it is not |
| | an Aspect |
+----------------+-----------------------------------------------------------+
**Update policy**
*Request*
+----------+------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==============================+==================================+
| PUT | /policies/<policy_id> | Update policy |
+----------+------------------------------+----------------------------------+
Body:
::
{
"description": "Changed description",
"tenantIds": ["726ed856965f43cc8e565bc991fa76d8"],
"objectModel": {
"maxFlavor": "m1.large",
"?": {
"type": "com.example.MyAspect",
"id": "446373ef-03b5-4925-b095-6c56568fa518"
}
}
}
*Response*
::
{
"id": "ce373a477f211e187a55404a662f968",
"description": "Changed description",
"created": "2013-11-30T03:23:42Z",
"updated": "2013-11-30T03:39:08Z",
"tenant_ids": ["726ed856965f43cc8e565bc991fa76d8"],
"objectModel": {
"maxFlavor": "m1.large",
"?": {
"type": "com.example.MyAspect",
"id": "446373ef-03b5-4925-b095-6c56568fa518"
}
}
}
Response codes:
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Policy has been updated successfully |
+----------------+-----------------------------------------------------------+
| 400 | Bad request. Either the format of the body is invalid or |
| | parameters doesn't match the contracts |
+----------------+-----------------------------------------------------------+
| 403 | User is not allowed to update this policy |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified policy doesn't exist |
+----------------+-----------------------------------------------------------+
| 409 | Policy with specified name already exists |
+----------------+-----------------------------------------------------------+
**Get policy details**
*Request*
+----------+------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==============================+==================================+
| GET | /policies/<policy_id> | Get policy details |
+----------+------------------------------+----------------------------------+
*Response*
::
{
"id": "ce373a477f211e187a55404a662f968",
"description": "Policy description",
"created": "2013-11-30T03:23:42Z",
"updated": "2013-11-30T03:39:08Z",
"tenant_ids": ["726ed856965f43cc8e565bc991fa76d8"],
"objectModel": {
"maxFlavor": "m1.medium",
"?": {
"type": "com.example.MyAspect",
"id": "446373ef-03b5-4925-b095-6c56568fa518"
}
}
}
Response codes:
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Policy details have been received successfully |
+----------------+-----------------------------------------------------------+
| 403 | User is not allowed to look up policies |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified policy doesn't exist |
+----------------+-----------------------------------------------------------+
**Delete policy**
*Request*
+----------+------------------------------+----------------------------------+
| Method | URI | Description |
+==========+==============================+==================================+
| DELETE | /policies/<policy_id> | Remove policy |
+----------+------------------------------+----------------------------------+
*Response*
::
{
"id": "ce373a477f211e187a55404a662f968",
"description": "Policy description",
"created": "2013-11-30T03:23:42Z",
"updated": "2013-11-30T03:39:08Z",
"tenant_ids": ["726ed856965f43cc8e565bc991fa76d8"],
"objectModel": {
"maxFlavor": "m1.medium",
"?": {
"type": "com.example.MyAspect",
"id": "446373ef-03b5-4925-b095-6c56568fa518"
}
}
}
Response codes:
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | OK. Policy has been removed successfully |
+----------------+-----------------------------------------------------------+
| 403 | User is not allowed to remove this policy |
+----------------+-----------------------------------------------------------+
| 404 | Not found. Specified policy doesn't exist |
+----------------+-----------------------------------------------------------+
Versioning impact
-----------------
None
Other end user impact
---------------------
python-muranoclient needs to be extended with support of new REST API calls.
Deployer impact
---------------
Deployers will need to check the logs and UI messages informing that some
parameters of deployment were changed by the policies, or for the reasons why
certain applications can not be deployed etc.
Developer impact
----------------
Business logic of the murano applications should not be affected by the
change. Policies will have the ability to modify its code on the fly during the
execution.
Murano-dashboard / Horizon impact
---------------------------------
The following changes need to be made in the GUI:
* Form to instantiate and initialize aspect with the rules (create policy)
* Form to map policies to tenants
* Page to display the list of policies
* Page to display each policy details
Implementation
==============
Assignee(s)
-----------
Primary assignee:
vakovalchuk
Work Items
----------
* Implement join points in engine where aspects can subscribe for events and
apply changes (object init and packages load during the first phase
of implementation, model load, garbage collection later).
* Create MuranoPL class ``Aspect`` that defines the basic interface for custom
aspects and place it to the core library.
* Create MuranoPL class ``Notifier`` that manages notifying process and place
it to the core library.
* Implement extended MuranoPL reflection capabilities and make it available
only to aspects.
* Create database model to store policies and model to store their assignment
to tenants.
* Implement API for CRUD operations for the aspect instances and their
assignment to tenants.
* Create python-muranoclient methods to support new API.
* Implement UI workflow to create policies and apply it to tenants.
* Create demo packages with example policies.
Dependencies
============
* https://blueprints.launchpad.net/murano/+spec/dependency-driven-resource-deallocation
* https://blueprints.launchpad.net/murano/+spec/schema-driven-ui
Testing
=======
* DSL unit tests for extended MuranoPL reflection
* Testrunner-based tests for the example policies functionality
* Tempest tests for the new REST API calls
* Unit and functional tests for the new python-muranoclient methods
* Selenium tests for the new GUI workflow
Documentation Impact
====================
* New functionality must be properly documented
* REST API specification need to be updated with new calls info
References
==========
* https://blueprints.launchpad.net/murano/+spec/dependency-driven-resource-deallocation
* https://blueprints.launchpad.net/murano/+spec/schema-driven-ui

363
specs/ocata/approved/environment-edit.rst
View File

@ -1,363 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==================================================
Capability to edit existing environment properties
==================================================
https://blueprints.launchpad.net/murano/+spec/environment-edit
The spec proposes mechanism for users to edit any details of an existing
environment, for example edit its regions, set home region, change the
networks or any other possible input field.
Problem description
===================
Current Murano API and python-muranoclient only allow to specify its home
region and default network during the environment creation. These parameters
cannot be seen and cannot be updated by the user after the environment
creation. Users may want to examine and change these or other environment
properties after it is created. Adding a mechanism to apply custom JSON-patch
to environment object model would provide this capability and add a great
flexibility in managing environments.
Proposed change
===============
Changes in API
--------------
It is proposed to create a new API endpoint
`/environments/<env_id>/model` and two calls to it using GET and PATCH
requests.
**GET** request provides environment object model where all environment
properties can be seen. See the `REST API impact` section for the example of
such output. If session is specified in the request and it is not in deploying
state, response gives modified environment model from the session, otherwise
actual environment model.
**PATCH** request allows to perform custom change to the environment object
model using JSON-patch object. It requires session to be specified by request
and saves changes to environment model in the specified session.
JSON-patch is a valid JSON that contains a list of changes to be applied to
the current object. Each change contains a dictionary with three keys: ``op``,
``path`` and ``value``. ``op`` (operation) can be one of the three values:
`add`, `replace` or remove`. *See RFC 6902 [1] for details*.
To indicate that body of the request is JSON-patch and validate correct
format of the body, a new media type should be added -
`application/env-model-json-patch`. During the validation in the middleware,
the following conditions must be checked against the passed object:
* It is a valid JSON-patch object (already implemented for
`application/murano-packages-json-patch` media type and should be reused).
Otherwise `400 Bad Request` response is given.
* Not all kinds of operations can be applied to all sections of the
environment model. The list of allowed operations is the following:
+ defaultNetworks: replace
+ name: replace
+ region: replace
+ regions: add, replace, remove
+ services: add, replace, remove
+ ?: add, replace, remove
Attempt to perform the restricted change will result in the `403 Forbidden`
response.
* Types of the environment properties and possible values are validated by the
appropriate JSON-schema. Otherwise `400 Bad Request` response is given.
Changes in python-muranoclient
------------------------------
To support the new API calls, corresponding python-muranoclient methods for
environments are needed: ``get_model`` and ``update_model`` respectively.
New CLI command that displays the result of ``get_model`` method:
.. code-block:: shell
murano environment-model-show <ENV_ID> [--path <PATH>] [--session-id <SESSION_ID>]
<PATH> allows to get a specific section of the model, for example
`defaultNetworks`, `region` or `?` or any of the subsections. It defaults to
'/' which means getting the whole model. Specifying <SESSION_ID> allows to get
the state of the environment with pending changes, while omitting it gives
the actual state.
New CLI command that allows to update environment model with a custom patch:
.. code-block:: shell
murano environment-model-edit <ENV_ID> <FILE> --session-id <SESSION_ID>
The command calls ``update_model`` method with data (JSON-patch) provided in
the <FILE> file. Changes are to be saved in session <SESSION_ID>.
Alternatives
------------
Separate CLI command can be created for updating each environment property.
It will be somewhat easier for users because file with JSON-patch will not be
needed. On the other hand, it will require creating a bunch of commands
instead of one generic and necessity to add new commands in future for the
possible new properties.
Data model impact
-----------------
None
REST API impact
---------------
**Get environment model**
+----------+-------------------------------------+------------------------+--------------------------+
| Method | URI | Header | Description |
+==========+=====================================+========================+==========================+
| GET | /environments/<env_id>/model/<path> | X-Configuration-Session| Get an Environment model |
| | | (optional) | |
+----------+-------------------------------------+------------------------+--------------------------+
Specifying <path> allows to get a specific section of the model, for example
`defaultNetworks`, `region` or `?` or any of the subsections.
*Response*
**Content-Type**
application/json
::
{
"defaultNetworks": {
"environment": {
"internalNetworkName": "net_two",
"?": {
"type": "io.murano.resources.ExistingNeutronNetwork",
"id": "594e94fcfe4c48ef8f9b55edb3b9f177"
}
},
"flat": null
},
"region": "RegionTwo",
"name": "new_env",
"regions": {
"": {
"defaultNetworks": {
"environment": {
"autoUplink": true,
"name": "new_env-network",
"externalRouterId": null,
"dnsNameservers": [],
"autogenerateSubnet": true,
"subnetCidr": null,
"openstackId": null,
"?": {
"dependencies": {
"onDestruction": [{
"subscriber": "c80e33dd67a44f489b2f04818b72f404",
"handler": null
}]
},
"type": "io.murano.resources.NeutronNetwork/0.0.0@io.murano",
"id": "e145b50623c04a68956e3e656a0568d3",
"name": null
},
"regionName": "RegionOne"
},
"flat": null
},
"name": "RegionOne",
"?": {
"type": "io.murano.CloudRegion/0.0.0@io.murano",
"id": "c80e33dd67a44f489b2f04818b72f404",
"name": null
}
},
"RegionOne": "c80e33dd67a44f489b2f04818b72f404",
"RegionTwo": {
"defaultNetworks": {
"environment": {
"autoUplink": true,
"name": "new_env-network",
"externalRouterId": "e449bdd5-228c-4747-a925-18cda80fbd6b",
"dnsNameservers": ["8.8.8.8"],
"autogenerateSubnet": true,
"subnetCidr": "10.0.198.0/24",
"openstackId": "00a695c1-60ff-42ec-acb9-b916165413da",
"?": {
"dependencies": {
"onDestruction": [{
"subscriber": "f8cb28d147914850978edb35eca156e1",
"handler": null
}]
},
"type": "io.murano.resources.NeutronNetwork/0.0.0@io.murano",
"id": "72d2c13c600247c98e09e2e3c1cd9d70",
"name": null
},
"regionName": "RegionTwo"
},
"flat": null
},
"name": "RegionTwo",
"?": {
"type": "io.murano.CloudRegion/0.0.0@io.murano",
"id": "f8cb28d147914850978edb35eca156e1",
"name": null
}
}
},
services: []
"?": {
"type": "io.murano.Environment/0.0.0@io.murano",
"_actions": {
"f7f22c174070455c9cafc59391402bdc_deploy": {
"enabled": true,
"name": "deploy",
"title": "deploy"
}
},
"id": "f7f22c174070455c9cafc59391402bdc",
"name": null
}
}
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | Environment model received successfully |
+----------------+-----------------------------------------------------------+
| 403 | User is not authorized to access environment |
+----------------+-----------------------------------------------------------+
| 404 | Environment is not found or specified section of the |
| | model does not exist |
+----------------+-----------------------------------------------------------+
**Update environment model**
*Request*
+----------+--------------------------------+------------------------+-----------------------------+
| Method | URI | Header | Description |
+==========+================================+========================+=============================+
| PATCH | /environments/<env_id>/model/ | X-Configuration-Session| Update an Environment model |
+----------+--------------------------------+------------------------+-----------------------------+
* **Content-Type**
application/env-model-json-patch
* **Example**
::
[{
"op": "replace",
"path": "/defaultNetworks/flat",
"value": true
}]
*Response*
**Content-Type**
application/json
See GET request response.
+----------------+-----------------------------------------------------------+
| Code | Description |
+================+===========================================================+
| 200 | Environment is edited successfully |
+----------------+-----------------------------------------------------------+
| 400 | Body format is invalid |
+----------------+-----------------------------------------------------------+
| 403 | User is not authorized to access environment or specified |
| | operation is forbidden for the given property |
+----------------+-----------------------------------------------------------+
| 404 | Environment is not found or specified section of the |
| | model does not exist |
+----------------+-----------------------------------------------------------+
Versioning impact
-----------------
None
Other end user impact
---------------------
Users will get new CLI commands to examine and edit environment object model.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
New python-muranoclient method can be used for providing environment edit
capability from GUI, but this perspective is out of scope of this spec.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
vakovalchuk
Work Items
----------
* Implement new API call to get object model of environment
* Implement new API calls to update the object model of environment
* Implement new python-muranoclient methods to support new API calls
* Add shell command that allows to display environment object model
* Add shell command that allows to apply JSON-patch to the environment model
* Add the same commands to the OpenStack client shell
Dependencies
============
None
Testing
=======
* API calls should be tested by unit tests and tempest tests
* CLI commands should be tested by unit tests
Documentation Impact
====================
* API calls should be described in the Murano API specification
* CLI commands should be described in the end user guide
References
==========
[1] https://tools.ietf.org/html/rfc6902

293
specs/ocata/approved/metadata-assignment-and-propagation.rst
View File

@ -1,293 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===================================
Metadata Assignment And Propagation
===================================
https://blueprints.launchpad.net/murano/+spec/metadata-assignment-and-propagation
Cloud users and operators may need to assign some arbitrary key-value pairs to
environments and applications deployed by murano to associate some metadata
information with them. If applicable this metadata attributes should be
propagated to OpenStack resources provisioned by Murano.
Problem description
===================
The meta information may describe various aspects of the deployed applications,
for example:
* Deployed software: names, versions, descriptions and license terms of the
software components deployed on the VMs;
* VM-related configuration features, like listening ports or interfaces;
* Hardware specs of provisioned virtual machines;
* Guest OS info, like the type of operating system, its family, version
etc;
* Information about the deployments purpose: type of the deployed
environment (development, staging, production), usage purpose, security
consideration etc;
* Information about the user who deployed the app or environment: name,
department, contact information, etc.
This information may be consumed directly by inspecting murano applications
with dashboard or CLI, by indexing these metadata attributes into specialized
search software, such as Searchlight [1] or by building custom reports with
grouping or aggregation by these metadata.
For the latter case it is especially important to propagate the metadata from
the initial objects it is assigned to (application or environment) to the
resource objects spawned by them (VMs, Volumes, Networks etc).
Proposed change
===============
UI change
---------
It is proposed to utilize the functionality provided by Glance Meta Definition
Catalog [2] to define the schemas of the applicable metadata attributes. It is
proposed to add two types of resources for murano applications and murano
environments: ``OS::Murano::Application`` and ``OS::Murano::Environment``
accordingly (ability to add metadata attributes for murano packages is out of
scope for this spec, however in future this may also become possible, so
``OS::Murano::Package`` is likely to be appropriate resource type).
The standard Angular-based Metadata assignment dialog (the one being currently
used to assign metadata to Instances, Images and Volumes) should be extended to
support Murano Dashboard and these new resource types. This extended dialog
will load the metadata definition schemas for attributes which are assignable
to appropriate type of resources and will submit them to Murano API for
persistence once the input is saved.
Murano Object Model change
--------------------------
For the applications it is proposed to persist the metadata attributes in the
object model, in the new block called 'metadata', which should be placed under
the '?' section of each object. For example, the '?' block of Apache
application tagged with a couple of attributes may look like this:
::
"?": {
"_26411a1861294160833743e45d0eaad9": {
"name": "Apache HTTP Server"
},
"type": "com.example.apache.ApacheHttpServer",
"id": "19dfb844-6eee-48bd-826a-c6206d065e85",
"metadata": {
"sw_webserver_apache_version": "2.1",
"sw_webserver_apache_http_port": 80
}
}
Since the ?-headers of objects placed in the environment's object model may be
modified with the existing Murano API no changes in the API is needed.
For the Environments a similar approach may be followed — to store the
metadata attributes under the `/?/metadata` key of the Environment object in
the object model. However an alternative may be considered: since each
Environments have a dedicated entry in murano's database, the metadata
attributes for environments may be stored in database in normalized form. See
the "Alternatives" section below for more details.
Regardless of the chosen approach it is required to introduce a new API call
to assign metadata attributes to the Environment, since the existing API does
not have capabilities to edit the ?-headers of Environment object. The
appropriate change into API is out of scope here and is covered with a separate
spec [3]
MuranoPL Change
---------------
MuranoPL language should be extended with an ability to read the metadata
attributes assigned to any given object. It is proposed to create a global yaql
function named ``metadata()`` for this purpose (similar to ``id()`` and
``name()`` which read appropriate information from the same ?-header).
Murano's serializer/deserializer should be properly updated to load the
metadata attributes and persist them back.
Core Library Change
-------------------
Core library should be extended with an ability to communicate with Glance
Metadata Definition API to query the namespaces applicable to a given resource
type and extract all the names of attributes from those namespaces. This will
allow the resource class to check if the given metadata attribute attached to
the application the class belongs to is applicable for the resource itself. The
binding to the Glance API should be done via a python-backed class
``MetadefBrowser``, while the rest of the logic may be implemented in MuranoPL.
A new MuranoPL class called ``MetadataAware`` should be implemented. This class
will have two methods: ``collectMetadata`` and ``getResourceType``.
``collectMetadata`` method will return all the metadata attributes assigned to
the object itself and all the attributes assigned to the owners of the object,
traversing the ownership tree to the root (i.e. to the Environment). All the
attributes from the traversed objects may be checked for applicability to the
particular resource type. To do this check, the code should call the
``getResourceType`` method which is supposed to return the resource type as
defined in Glance (``OS::Nova::Server`` for VMs, ``OS::Cinder::Volume`` for
volumes etc) and then use the ``MetadefBrowser`` described above to check if
the attribute is applicable to the objects of this type.
The resource-specific classes of the Core Library (``Instance``,
``CinderVolume`` etc) have to extend the ``MetadataAware`` and override the
``getResourceType`` method.
Alternatives
------------
There are a number of alternatives to consider.
First, instead of using Glance Metadef catalog to store the possible attribute
names and constraints, we may assign attributes without any predefined schema.
This will reduce our dependency on other OpenStack components (Glance in this
case), but will introduce extra development burden (since we won't be able to
reuse the existing metadata-assignment dialog) and will prevent us from using
convenient schema-based validators of tags which are made for metadefs. So, it
is proposed to stay with metadef-compliant implementation.
Second option to consider is the storage of metadata attributes on environment
level. Most other OpenStack projects store metadata attributes in normalized
form: they have the separate table mapped to the primary object table as
one-to-many. The approach suggested above uses denormalized storage: the
attributes are stored inside the json-document describing the complete
environment, which is stored as a single value in the database. This is much
easier to implement, also having metadata in object model of environment object
is still needed, so duplicating data storage seems like an extra work.
Third option to consider: not to check for attribute applicability on resource
types. This simplifies the development (no need to create ``MetadefBrowser``
class and much simpler logic in ``MetadataAware``), however this will degrade
user experience, as resources may get inappropriate attributes assigned, which
will be confusing.
Data model impact
-----------------
This proposal adds new fields to the ?-headers of objects in object model. If
the normalized alternative is chosen, it will also introduce extra table in
database and a relationship with the ``Environments`` table. This will require
to create a database migration as well.
REST API impact
---------------
A modification of Murano API is required to allow assigning of metadata
attributes to the Environment objects, however it is out of scope here. See [3]
for details.
Versioning impact
-----------------
Since the proposed change requires to add a new yaql function to the DSL and
modifies the serializer/deserializer logic this requires to bump the MuranoPL
format version.
Other end user impact
---------------------
The users will see new action buttons "Update Metadata" in Murano Dashboard.
This buttons will be available as row actions for Environment Components and
Environments.
Deployer impact
---------------
The proposed change introduces a dependency on Glance API. To configure the
connectivity to this API (endpoint details, encryption etc) a new optional
configuration block should be added to Murano config.
Developer impact
----------------
Developers may implement the ``MetadataAware`` class to add metadata handling
logic for their classes.
Murano-dashboard / Horizon impact
---------------------------------
This change will add a custom version of metadata.service javascript module to
Murano Dashboard. It will have the similar logic as the standard Horizon's
module, but will add the references to murano api's, so the Metadata dialog can
update the attributes assigned to Murano entities.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
ativelkov
Other contributors:
TBD
Work Items
----------
* Add a rest api handlers for murano-dashboard to handle metadata updates and
fetching;
* Add a javascript API to this handlers;
* Create a custom version of metadata.service, so the metadata modal dialog may
utilize murano's javascript APIs to metadata;
* Modify Murano Dashboard to call metadata modal dialog as row handlers for
Environments and Services;
* Modify Murano API to allow edits of ?-section of Environment objects;
* Add a ``MetadefBrowser`` class to Core Library;
* Implement a ``MetadataAware`` class and extend ``Instance`` and
``CinderVolume`` classes with it.
Dependencies
============
* Glance Metadefinition Catalog [2]
* Environment modification API changes [3]
Testing
=======
The integration tests should verify that the attributes are propagated properly
Documentation Impact
====================
The new capabilities to add attributes should be documented in user manual.
The changes in core library and the usage of ``MetadataAware`` class should be
reflected in developers' manual.
References
==========
[1] https://wiki.openstack.org/wiki/Searchlight
[2] http://docs.openstack.org/developer/glance/glancemetadefcatalogapi.html
[3] https://review.opendev.org/#/c/378602

141
specs/ocata/approved/rework_exceptions.rst
View File

@ -1,141 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=========================
Murano exception handling
=========================
https://blueprints.launchpad.net/murano/+spec/murano-exception-handling
Current exception handling in murano project differs from the way
all OpenStack projects handle their exception. Murano have very small list
of its own exceptions and use webob or general exceptions everywhere instead of
store all needed exceptions in one place.
Problem description
===================
* Murano is OpenStack project and should follow community practices.
* Using murano specific exceptions can make debugging easier.
* Now we have 3 different places where exceptions are stored: murano.packages.exceptions,
murano.dsl.exceptions, murano.common.exception depends on part of murano which
throws the exception.
Proposed change
===============
It's planned to create 2 new basic exceptions in murano.common.exceptions which
will be parents for others. First one will be general exception and the second
one will be exception for HTTP requests. Exceptions in murano.packages.exceptions
should have general MuranoException as a parent. Exceptions in murano.dsl.exceptions
should be kept as is since dsl is mostly separate part of murano.
These exceptions will be replaced with exceptions inherited from MuranoHTTPException:
* webob.exc.HTTPBadRequest
* webob.exc.HTTPInternalServerError
* webob.exc.HTTPForbidden
* webob.exc.HTTPUnsupportedMediaType
* webob.exc.HTTPUnauthorized
* webob.exc.HTTPForbidden
* webob.exc.HTTPConflict
* webob.exc.HTTPClientError
These exceptions will be replaced with exceptions inherited from MuranoException:
* ValueError
* Exception
* RuntimeError
* NotImplementedError
* SyntaxError
* TypeError
* NameError
All custom exceptions which defined in murano should be moved to one of the general
exceptions files.
Alternatives
------------
Keep everything as it is.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
End users will see new murano specific exceptions.
Deployer impact
---------------
None
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
New exceptions should be introduced and handled in horizon.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
starodubcena
Work Items
----------
* Create new exception in murano-api, change exception handling where it will be
needed
* Create new exception in packages, change exception handling where it will be
needed
* Handle new exceptions in python-muranoclient
* Handle new exception in murano-dashboard
Dependencies
============
None
Testing
=======
This change need a huge refactoring for unit and functional tests.
Documentation Impact
====================
None

175
specs/pike/approved/allow-encrypting-of-muranopl-properties.rst
View File

@ -1,175 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
============================
Encrypt Murano PL Properties
============================
https://blueprints.launchpad.net/murano/+spec/allow-encrypting-of-muranopl-properties
Currently the object model for an application in Murano is stored in the Murano
database as plain text. This can pose a security risk for applications whose
object model contains passwords or other sensitive data.
Problem description
===================
Many Murano applications request input data from users, which in most cases is
entered via the murano dashboard. Currently all input from users is stored in
database in plain text. In the case of a MySQL [#]_ app for example, the
password that the user enters will be stored in plain text in both the
``session`` and ``environment`` database tables. This info is available both to
the cloud admin, other users in the same project, and also to an attacker
should the database become compromised.
Proposed change
===============
This spec proposes to add a two new yaql functions; `encrypt_data` and
`decrypt_data`. The former will be processed at the dashboard stage, as it will
be applied to input fields presented to the user. `decrypt_data` will be
processed in the engine during app execution.
These functions will make use of Castellan [#]_ which is a generic
key manager interface for OpenStack. Castellan is designed to be used with a
variety of secret storage backends, the most common being Barbican [#]_. The
Castellan key manager is already being used by the Nova and Cinder project to
name two [#]_. Because Barbican is the default storage backend, Barbican and
Castellan will be referred to interchangeably for the rest of this document.
Alternatives
------------
Some questions have been raised as to the usefulness of Barbican as a whole
[#]_. Some community members are concerned with adding it as "yet another
dependency" to their project, while others have concerns about its integration
with Keystone with regards to secret storage.
With regards to dependency concerns, these functions will be optional to Murano.
If Castellan is not configured, an error message will be shown in Horizon
informing the user to contact their administrator if they wish to use apps
requiring encryption. The use of Castellan also helps here, as it gives
operators flexibility in which secret backend they wish to use. It should be
noted however that Castellan does not appear to currently provide any "dummy"
backend drivers [#]_.
With regards to security, the argument that Barbican is the common solution for
secret storage in OpenStack seems reasonable; security is something best left
to specialists in the field. If a flaw is discovered in Barbican, it can be
patched once by an operator and all projects using it will automatically
benefit.
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-----------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
Deployers wishing to use the new encryption functionality will be required to
deploy a key manager such as Barbican (it is worth noting Barbican may already
be available in some cloud environments in which case there would be minimal
impact).
The functionality will be made optional via a configuration item, hence
there will be no impact for deployers who don't wish to take advantage of this
feature.
Developer impact
----------------
Developers should be made aware of these functions via documentation, and also
how to use them.
Murano-dashboard / Horizon impact
---------------------------------
Will need to be updated to understand calls to `encrypt_data` in forms.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
paul.bourke@oracle.com
Other contributors:
None
Work Items
----------
* Integrate Castellan into murano-dashboard.
* Add a `encrypt_data` function to the dashboard to make use of the above.
* Integrate Castellan into the murano engine.
* Add a `decrypt_data` function to the dashboard to make use of the above.
* Testing.
Dependencies
============
* Castellan
* Barbican
Testing
=======
* Appropriate unit and functional tests will need to be added.
* Potentially the tempest tests could be updated for full end-to-end testing
with Barbican.
Documentation Impact
====================
Deployment docs will need info on Barbican configuration for Murano and
murano-dashboard. MuranoPL Docs will also be needed on the new functions.
References
==========
.. [#] https://opendev.org/openstack/murano-apps/src/branch/master/MySQL/package/UI/ui.yaml
.. [#] https://opendev.org/openstack/castellan
.. [#] https://opendev.org/openstack/barbican
.. [#] https://review.opendev.org/#/c/247561/
.. [#] http://lists.openstack.org/pipermail/openstack-dev/2017-January/110192.html
.. [#] https://opendev.org/openstack/castellan/src/branch/stable/ocata/castellan/key_manager
* Cinder discussion around their alternative insecure key manager for Castellan:
http://lists.openstack.org/pipermail/openstack-dev/2016-January/083241.html
* Murano discussion around implementation details for this blueprint:
http://lists.openstack.org/pipermail/openstack-dev/2017-June/118290.html

161
specs/pike/approved/policy-in-code.rst
View File

@ -1,161 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==============
Policy in code
==============
`bp policy-in-code <https://blueprints.launchpad.net/murano/+spec/policy-in-code>`_
Deployers currently have to maintain policy files regardless if they change
the default policy provided from murano. They also have to trace through
code in order to determine the purpose behind each policy operation in the
policy file. Maintaining the policy files and tracing through code to provide
context to the operations in the policy files is cumbersome and error-prone.
By moving policy into code, we can leverage tooling to make maintenace easier
for deployers, provide a centralized location for default policy values,
and more effectively implement self-documenting policies.
Problem Description
===================
Today policy exists in a file that deployers are expected to maintain in their
deployment. If a deployer needs to change the default policy rules for an
operation, they have to make those changes and continuously check to make sure
conflicts are resolved with each new release of the policy file. This is
cumbersome to maintain, even if a deployment is only using the default policy.
Deployers must also trace through code to identify where certain policies
are enforced, which is also cumbersome.
Proposed Change
===============
The proposed solution is to check policy into the code base and register it
using the oslo.policy library. This is very similar to how projects register
and use configuration options using oslo.config. If policies are provided in a
policy file on disk, those policies will be registered instead of the in-code
default. This provides a way for deployers to override the defaults provided
in the in-code policies.
The registration will need two pieces of data:
1. The operation, e.g. "create_environment" or "list_environments_all_tenants"
2. The rule, e.g. "role:admin" or "rule:default"
Descriptions can also be provided in the registered policy object that help
describe the operation and the rule or role that is required to execute it,
in addition to which API endpoints enforce the policy operation. This
description can be used when generating sample policy files from registered
rules, as well as help operators to better understand policy enforcement
in murano.
This is the exact same approach
`nova <http://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/policy-in-code.html>`_
and
`keystone <http://specs.openstack.org/openstack/keystone-specs/specs/keystone/pike/policy-in-code.html>`_
used to codify policy.
The following are benefits from the approach:
* There is no longer a need to maintain a policy file in tree.
* A tool can be written to auto-generate a policy.json file from the default
policy operations in code.
* It will be easier for operators to understand the intent of the policy
operations and where they are enforced in the system.
* It will be easier to provide a description of each policy much like we do
configuration options. This will ensure that the policies are well-documented
and maintained.
Alternatives
------------
An alternative approach was to pull policy into murano as an official
resource. This would still require some sort of policy override ablility for
deployments that do not wish to deploy the default.
Security Impact
---------------
None.
Notifications Impact
--------------------
None.
Other End User Impact
---------------------
None. Policy will continue to be evaluated and enforced like it does today.
Performance Impact
------------------
The performance impact of moving policy in code should be minimal. If the
deployment doesn't have a policy file on disk, the service will not have to
fetch it. Instead the default will be registered and used from within code. In
the event the deployment is using policy overrides, the combination of the two
approaches might cause some performance impact compared to defaults in code,
but the overall impact should be negligible.
Other Deployer Impact
---------------------
If a deployer already makes modifications to the default policy file, they
will have to continue maintaining those changes. For deployers who modify a
subset or none of the policy entries, they can essentially remove their policy
file, or the policies that are the default. The end result should be a policy
file that purely consists of overrides the deployer wishes to enforce.
Another deployer impact is that deployers no longer need to double-check they
are protecting all new operations by manually inspecting policy files across
releases. Instead, they can be notified about new policies available in a
release via release notes and then choose to either use the well-documented
defaults or to override them in the policy file.
Developer Impact
----------------
Any policies added to the code should be registered before they are used. There
should also be checks and tests added that make sure new policy entries are
accompanied with a release note.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Felipe Monteiro (felipemonteiro)
Work Items
----------
* Investigate the process for adding oslo.policy into keystone's policy.
* Gradually move policy checks from ``policy.json`` into oslo.policy objects.
This can be done incrementally and should remove the check from
``policy.json``.
* Change genconfig tox environment to generate sample policy.json file.
* Update documentation.
* Remove the murano policy file from devstack and murano.
Dependencies
============
None.
Documentation Impact
====================
Documentation for deployers about the policy file will be updated to mention
that only policies which differ from the default will need to be included.
References
==========
* `nova specification <http://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/policy-in-code.html>`_
* `keystone specification <http://specs.openstack.org/openstack/keystone-specs/specs/keystone/pike/policy-in-code.html>`_

220
specs/queens/approved/message-signing.rst
View File

@ -1,220 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=====================
Agent message signing
=====================
https://blueprints.launchpad.net/murano/+spec/message-signing
Problem description
===================
In order to deploy software, Murano sends scripts, playbooks and other files
to agents that run on each VM created. Murano uses RabbitMQ message broker
and its AMQP protocol to deliver messages to the agents. During first boot
each VM gets its RabbitMQ credentials, the queue name where the messages are
going to appear as well as instructions on where
to put execution results. Typical Murano installation has a dedicated
RabbitMQ instance that is used for such communications.
However, since user management is not included in AMQP protocol, Murano
requires RabbitMQ credentials to be provided in its config file. Hence all
the created VMs share the same credentials and it is only the queue name that
distinguishes one machine from another. Since murano agents steady pull new
commands from the RabbitMQ, if the queue name gets compromised, anyone could
send commands to the VM and take complete control over it. Luckily current AMQP
version does not have a function to list existing queues. However, since
queue name is generally not considered to be a secret information such
function might be added in the future. Also this information might be
exposed by the RabbitMQ plugins installed in the system. For example, there
is a so called Management plugin that can be used for that purpose. Besides,
the queue name can be found in the Heat stack definition for particular
Murano environment and anyone who can access it could take it from there.
Default Heat policy restricts it to users of that tenant plus the admin user.
However, particular OpenStack installations might have it configured
differently. Also even with default policy it is possible for users of a
single project (tenant) to control each other's VMs regardless of their role
in the tenant.
Proposed change
===============
The proposed solution is to implement message signing so that agents can
verify that the messages come from the murano-engine. There is a keypair
that is shared by all the murano-engine instances in the cloud. When murano
creates a VM, it provides it with a public part of the keypair in addition
to currently provided RabbitMQ credentials and queue name. Each message sent
by the murano-engine is going to be signed with the private key. Agents
will verify the signature with the public key they gave and ignore all the
messages that are not proven to be coming from the engine.
Here is how the communication workflow is going to become:
0. Murano engine is provided with the keypair (key file that has both the
private and the public parts). The default is `~/.ssh/id_rsa` which is a
standard RSA SSH keypair of the user that is running the murano-engine.
However, different key might be provided with the newly introduced config
file setting.
1. When a VM is created, its config that is passed to it via the user-data is
going to include the public part of the key.
2. Every time murano-engine is about to send message `M` to the agent with
queue name `Q` it calculates ``sign(Q + M, private_key)`` i.e. the
signature of the message bytes with prepended queue name in latin1 encoding.
The queue name is prepended so that the messages sent to one VM could not be
resent to other VMs. The signature is then put into a custom AMQP message
header so that the message body does not change.
3. Once the message is received by an gent, it performs
``verify(signature, Q + M, public_key)`` and drops the message if the
verification fails. Since the queue name is different for each agent, it
will also reject all messages that were signed for other VMs.
For the first version it all the cryptographic parameters are going to be
fixed so that no additional configuration is needed:
* The keypair must be PEM-encoded RSA without password protection
* The padding is PKCS#1 v1.5
* Digest function used for signing is SHA256
Engine-agent compatibility
--------------------------
Since the agent can be baked into the images Murano and updated
independently from the server, Murano must still work in situations where
the updated server talks to the old agents or vice versa (old engine, new
agents).
For the new engine / old agents case the compatibility is retained because
old agents will just ignore the new (and unknown to them) setting in the
config file (the public key) as well as custom AMQP header and just won't
verify the signature.
In the old engine / new agents case the server won't provide the public key
to the agents. Agents won't try to verify the messages in this case.
Thus the message signing is going to help only if both the server and the
agents support it. Otherwise everything is going to work the way it
currently does.
Replay attack prevention
------------------------
Having queue names prepended to the message bodies prevents messages sent to
one VM from being resent to another. However it is still possible to resend
the same message to the same VM later.
In order to prevent it, the message body JSON is going to get additional key
called ``Stamp``. It is a long integer value that is monotonic i.e. the
server guarantees that the number is greater from that of the previous
message. The field is optional (since older engines won't send it), but if
it is present, the agent must ensure that it exceeds the value of the
previously seen messages and ignores the message if it not the case. Murano
engine is going to derive the value from the current timestamp.
Messages with additional field remain compatible with the agents because
they ignore all unknown fields. As a bonus, this feature is going to protect
the agent against messages that for some reason got delivered twice which
can happen in rare cases due to the message was requeued because of
unexpected connectivity loss with RabbitMQ upon its acknowledgement or due
to some RabbitMQ cluster inconsistencies.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Versioning impact
-------------------------
None
Other end user impact
---------------------
None
Deployer impact
---------------
There should be generated and provisioned for all the engine instances
keypair file. However it is going to be optional in order not to break
existing murano deployment playbooks.
Developer impact
----------------
None
Murano-dashboard / Horizon impact
---------------------------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Stan Lagun (slagun)
Work Items
----------
1. Add keypair to the murano-engine config file schema
2. Extract the public key from the keypair and inject it into the generated
agent config data
3. Implement message signing on the server
4. Implement signature verification in the python agent
5. Implement signature verification in the legacy Windows (PowerShell) C#
agent
6. Add stamp value to the messages
7. Validate stamp value on the agent (including persistent tracking of the
previous stamp value) for both agents
Dependencies
============
There are going to be new dependencies on the cryptography python package
that provides all the required crypto functionality. Several such libraries
are already present in the global requirements list.
Testing
=======
New functionality can be fully tested with unit tests.
Documentation Impact
====================
New config setting for the Murano engine should be documented.
References
==========
None

279
specs/template.rst
View File

@ -1,279 +0,0 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Example Spec - The title of your blueprint
==========================================
Include the URL of your launchpad blueprint:
https://blueprints.launchpad.net/murano/+spec/example
Introduction paragraph -- why are we doing anything? A single paragraph of
prose that operators can understand. The title and this first paragraph
should be used as the subject line and body of the commit message
respectively.
Some notes about using this template:
* Your spec should be in ReSTructured text, like this template.
* Please wrap text at 79 columns.
* The filename in the git repository should match the launchpad URL, for
example a URL of: https://blueprints.launchpad.net/murano/+spec/awesome-thing
should be named awesome-thing.rst
* Please do not delete any of the sections in this template. If you have
nothing to say for a whole section, just write: None
* For help with syntax, see http://sphinx-doc.org/rest.html
* To test out your formatting, build the docs using tox, or see:
http://rst.ninjs.org
* If you would like to provide a diagram with your spec, ascii diagrams are
required. http://asciiflow.com/ is a very nice tool to assist with making
ascii diagrams. The reason for this is that the tool used to review specs is
based purely on plain text. Plain text will allow review to proceed without
having to look at additional files which can not be viewed in gerrit. It
will also allow inline feedback on the diagram itself.
* If your specification proposes any changes to the Murano REST API such
as changing parameters which can be returned or accepted, or even
the semantics of what happens when a client calls into the API, then
you should add the APIImpact flag to the commit message. Specifications with
the APIImpact flag can be found with the following query:
https://review.opendev.org/#/q/status:open+project:openstack/murano-specs+message:apiimpact,n,z
Problem description
===================
A detailed description of the problem:
* For a new feature this might be use cases. Ensure you are clear about the
actors in each use case: End User vs Deployer
* For a major reworking of something existing it would describe the
problems in that feature that are being addressed.
Proposed change
===============
Here is where you cover the change you propose to make in detail. How do you
propose to solve this problem?
If this is one part of a larger effort make it clear where this piece ends. In
other words, what's the scope of this effort?
Alternatives
------------
What other ways could we do this thing? Why aren't we using those? This doesn't
have to be a full literature review, but it should demonstrate that thought has
been put into why the proposed solution is an appropriate one.
Data model impact
-----------------
Changes which require modifications to the data model often have a wider impact
on the system. The community often has strong opinions on how the data model
should be evolved, from both a functional and performance perspective. It is
therefore important to capture and gain agreement as early as possible on any
proposed changes to the data model.
Questions which need to be addressed by this section include:
* What new data objects and/or database schema changes is this going to
require?
* What database migrations will accompany this change.
* How will the initial set of new data objects be generated, for example if you
need to take into account existing instances, or modify other existing data
describe how that will work.
REST API impact
---------------
Each API method which is either added or changed should have the following
* Specification for the method
* A description of what the method does suitable for use in
user documentation
* Method type (POST/PUT/GET/DELETE)
* Normal http response code(s)
* Expected error http response code(s)
* A description for each possible error code should be included
describing semantic errors which can cause it such as
inconsistent parameters supplied to the method, or when an
instance is not in an appropriate state for the request to
succeed. Errors caused by syntactic problems covered by the JSON
schema definition do not need to be included.
* URL for the resource
* Parameters which can be passed via the url
* Example use case including typical API samples for both data supplied
by the caller and the response
* Discuss any policy changes, and discuss what things a deployer needs to
think about when defining their policy.
Versioning impact
-----------------
Discuss how your change affects versioning and backward compatibility:
* Can it break any existing DSL code even in theory?
* Can it break API consumers that are using older python-muranoclient versions
or non-python clients?
* If you make changes to Murano package please state how the version number
should be incremented?
* Does your change require newer version of external or internal component?
* How to keep backward compatibility with code and consumers that were
available prior to your change?
Other end user impact
---------------------
Aside from the API, are there other ways a user will interact with this
feature?
* Does this change have an impact on python-muranoclient? What does the user
interface there look like?
Deployer impact
---------------
Discuss things that will affect how you deploy and configure OpenStack
that have not already been mentioned, such as:
* What config options are being added? Should they be more generic than
proposed (for example a flag that other hypervisor drivers might want to
implement as well)? Are the default values ones which will work well in
real deployments?
* Is this a change that takes immediate effect after its merged, or is it
something that has to be explicitly enabled?
* If this change is a new binary, how would it be deployed?
* Please state anything that those doing continuous deployment, or those
upgrading from the previous release, need to be aware of. Also describe
any plans to deprecate configuration values or features. For example, if we
change the directory name that instances are stored in, how do we handle
instance directories created before the change landed? Do we move them? Do
we have a special case in the code? Do we assume that the operator will
recreate all the instances in their cloud?
Developer impact
----------------
Discuss things that will affect other developers working on OpenStack,
such as:
* If the blueprint proposes a change to the driver API, discussion of how
other hypervisors would implement the feature is required.
Murano-dashboard / Horizon impact
---------------------------------
Does it require changes to the murano-dashboard / horizon? If so, changes
should be described well. If it's about complex changes than probably a
separate blueprint / spec should be created for it.
Implementation
==============
Assignee(s)
-----------
Who is leading the writing of the code? Or is this a blueprint where you're
throwing it out there to see who picks it up?
If more than one person is working on the implementation, please designate the
primary author and contact.
Primary assignee:
<launchpad-id or None>
Other contributors:
<launchpad-id or None>
Work Items
----------
Work items or tasks -- break the feature up into the things that need to be
done to implement it. Those parts might end up being done by different people,
but we're mostly trying to understand the timeline for implementation.
Dependencies
============
* Include specific references to specs and/or blueprints in murano, or in other
projects, that this one either depends on or is related to.
* If this requires functionality of another project that is not currently used
by Murano, document that fact.
* Does this feature require any new library dependencies or code otherwise not
included in OpenStack? Or does it depend on a specific version of library?
Testing
=======
Please discuss how the change will be tested. We especially want to know what
tempest tests will be added. It is assumed that unit test coverage will be
added so that doesn't need to be mentioned explicitly, but discussion of why
you think unit tests are sufficient and we don't need to add more tempest
tests would need to be included.
Is this untestable in gate given current limitations (specific hardware /
software configurations available)? Is this untestable in murano-ci? If so,
are there mitigation plans (3rd party testing, gate enhancements, etc).
Documentation Impact
====================
What is the impact on the docs team of this change? Some changes might require
donating resources to the docs team to have the documentation updated. Don't
repeat details discussed above, but please reference them here.
References
==========
Please add any useful references here. You are not required to have any
reference. Moreover, this specification should still make sense when your
references are unavailable. Examples of what you could include are:
* Links to mailing list or IRC discussions
* Links to notes from a summit session
* Links to relevant research, if appropriate
* Related specifications as appropriate (e.g. if it's an EC2 thing, link the
EC2 docs)
* Anything else you feel it is worthwhile to refer to

0
tests/__init__.py
View File

102
tests/test_titles.py
View File

@ -1,102 +0,0 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import glob
import re
import docutils.core
import testtools
class TestTitles(testtools.TestCase):
def _get_title(self, section_tree):
section = {
'subtitles': [],
}
for node in section_tree:
if node.tagname == 'title':
section['name'] = node.rawsource
elif node.tagname == 'section':
subsection = self._get_title(node)
section['subtitles'].append(subsection['name'])
return section
def _get_titles(self, spec):
titles = {}
for node in spec:
if node.tagname == 'section':
section = self._get_title(node)
titles[section['name']] = section['subtitles']
return titles
def _check_titles(self, fname, titles):
expected_titles = ('Problem description', 'Proposed change',
'Implementation', 'Dependencies',
'Testing', 'Documentation Impact',
'References')
self.assertEqual(
sorted(expected_titles),
sorted(titles.keys()),
"Expected titles not found in document %s" % fname)
proposed = 'Proposed change'
self.assertIn('Alternatives', titles[proposed])
self.assertIn('Data model impact', titles[proposed])
self.assertIn('REST API impact', titles[proposed])
self.assertIn('Versioning impact', titles[proposed])
self.assertIn('Other end user impact', titles[proposed])
self.assertIn('Deployer impact', titles[proposed])
self.assertIn('Developer impact', titles[proposed])
self.assertIn('Murano-dashboard / Horizon impact', titles[proposed])
impl = 'Implementation'
self.assertIn('Assignee(s)', titles[impl])
self.assertIn('Work Items', titles[impl])
def _check_lines_wrapping(self, tpl, raw):
for i, line in enumerate(raw.split("\n")):
if "http://" in line or "https://" in line:
continue
self.assertTrue(
len(line) < 80,
msg="%s:%d: Line limited to a maximum of 79 characters." %
(tpl, i+1))
def _check_no_cr(self, tpl, raw):
matches = re.findall('\r', raw)
self.assertEqual(
len(matches), 0,
"Found %s literal carriage returns in file %s" %
(len(matches), tpl))
def _check_trailing_spaces(self, tpl, raw):
for i, line in enumerate(raw.split("\n")):
trailing_spaces = re.findall(" +$", line)
self.assertEqual(len(trailing_spaces),0,
"Found trailing spaces on line %s of %s" % (i+1, tpl))
def test_template(self):
files = ['specs/template.rst'] + glob.glob('specs/*/*/*')
for filename in files:
self.assertTrue(filename.endswith(".rst"),
"spec's file must uses 'rst' extension.")
with open(filename) as f:
data = f.read()
spec = docutils.core.publish_doctree(data)
titles = self._get_titles(spec)
self._check_titles(filename, titles)
self._check_lines_wrapping(filename, data)
self._check_no_cr(filename, data)
self._check_trailing_spaces(filename, data)

30
tox.ini
View File

@ -1,30 +0,0 @@
[tox]
minversion = 3.1.1
envlist = docs,pep8
skipsdist = True
ignore_basepython_conflict = True
[testenv]
basepython = python3
usedevelop = True
setenv = VIRTUAL_ENV={envdir}
deps =
-c{env:UPPER_CONSTRAINTS_FILE:https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt}
-r{toxinidir}/requirements.txt
commands = python setup.py testr --slowest --testr-args='{posargs}'
whitelist_externals = find
[testenv:venv]
commands = {posargs}
[testenv:docs]
commands =
find . -type f -name "*.pyc" -delete
sphinx-build -W -b html doc/source doc/build/html
[testenv:doc8]
deps =
-c{env:UPPER_CONSTRAINTS_FILE:https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt}
-r{toxinidir}/requirements.txt
doc8
commands = doc8 doc/source