openstack/charm-guide
Code Issues Proposed changes

Split Charm Author Docs into Anatomy and Template

Browse Source 
The aim is to have a guide foe each template which is then supported
by a central Charm Anatomy. This change includes guides for API and
SDN. Guides for writing backends for Dashboard, Barbican, Cinder and
Heat should follow soonish.

Change-Id: I6d9578aa83c7c2a93aeb38c9ed9f38023fa8f873
This commit is contained in:
Liam Young 2016-10-17 08:11:37 +00:00
parent 0d3ed9e2ed
commit 5b8da47d88
6 changed files with 879 additions and 376 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

460
doc/source/charm-anatomy.rst Normal file
View File

@ -0,0 +1,460 @@
.. _charm_anatomy:
=============
Charm Anatomy
=============
Overview
--------
The new OpenStack charms (charms written in 2016 onwards) are written using the
`reactive framework <https://pythonhosted.org/charms.reactive>`__ in Python.
An introduction on the reactive framework and building charms from layers can
be found in the `Authors Charm Building Guide <https://jujucharms.com/docs/devel/authors-charm-building>`__ .
This guide covers only the new reactive charms.
Configuration Files
-------------------
.. _`layers.yaml`:
layers.yaml
~~~~~~~~~~~
The **src/layers.yaml** file defines what layers and interfaces will be imported
and included in the charm when the charm is built. See the `OpenStack Layers`_
section and `OpenStack Interfaces`_ section below. If additional interfaces or
layers add them to the **includes** list within **src/layers.yaml**.
Below is an example of the layers.yaml for an OpenStack API charm which has a
relation with MongoDB:
.. code:: yaml
includes: ['layer:openstack-api', 'interface:mongodb']
options:
basic:
use_venv: True
include_system_packages: True
When the charm is built the openstack-api layer and mongodb interface will
be included in the built charm. The charm will run in a virtual env with
system packages exposed in that virtual env. See the *Layer Configuration*
section in `Basic Layer README <https://github.com/juju-solutions/layer-basic>`__
for more details of the configurable options in a **layers.yaml**
config.yaml
~~~~~~~~~~~
The charm authors guide contains a section on the `config.yaml <https://jujucharms.com/docs/2.0/authors-charm-config>`__
and is a good place to start. The config.yaml of the built charm is
constructed from each layer that contains a config.yaml.
metadata.yaml
~~~~~~~~~~~~~
The charm
`metadata.yaml <https://jujucharms.com/docs/2.0/authors-charm-metadata>`__
describes the charm and how it relates to other charms. This is also
constructed from each layer that defines a metadata.yaml
.. _`OpenStack Layers`:
OpenStack Layers
----------------
Basic Layer
~~~~~~~~~~~
The `Basic Layer <https://github.com/juju-solutions/layer-basic>`__ is the
base layer for all charms built using layers. It provides all of the standard
Juju hooks and runs the charms.reactive.main loop for them. It also bootstraps
the charm-helpers and charms.reactive libraries and all of their dependencies
for use by the charm.
.. _`OpenStack Layer`:
OpenStack Layer
~~~~~~~~~~~~~~~
The `Openstack Layer <https://github.com/openstack/charm-layer-openstack>`__
provides the base OpenStack configuration options, templates, template
fragments and dependencies for authoring OpenStack Charms. Typically this layer
is used for subordinate charms. The openstack-api or openstack-principle layers
are probably more appropriate for principle charms and both of those layers
inherit this one.
This layer includes a wheelhouse to pull in `charms.openstack <https://github.com/openstack/charms.openstack>`__
. See `charms.openstack`_ for more details.
Openstack Principle Layer
~~~~~~~~~~~~~~~~~~~~~~~~~
The `Openstack Principle Layer <https://github.com/openstack/charm-layer-openstack-principle>`__
provides the base layer for OpenStack charms that are intended for
use as principle (rather than subordinate)
Openstack API Layer
~~~~~~~~~~~~~~~~~~~
The `Openstack API Layer <https://github.com/openstack/charm-layer-openstack-api>`__
provides the base layer for OpenStack charms that are will deploy API services,
and provides all of the core functionality for:
- HA (using the hacluster charm)
- SSL (using configuration options or keystone for certificates)
- Juju 2.0 network space support for API endpoints
- Configuration based network binding of API endpoints
It also pulls in interfaces mysql-shared, rabbitmq and keystone which are
common to API charms.
.. _`OpenStack Interfaces`:
OpenStack Interfaces
--------------------
Interfaces define the data exchange between each of the charms. A list of all
available interfaces is available `here <http://interfaces.juju.solutions>`__.
A list of OpenStack specific interfaces can be found `here <https://github.com/openstack?query=charm-interface>`__
The interfaces a charm needs are defines in the `layers.yaml`_. Below is a list
of the typical interfaces needed by different OpenStack charm types:
**API Charm**
- `mysql-shared <https://github.com/openstack/charm-interface-mysql-shared>`__
- `rabbitmq <https://github.com/openstack/charm-interface-rabbitmq>`__
- `keystone <https://github.com/openstack/charm-interface-keystone>`__
**Neutron SDN Plugin**
- `neutron-plugin <https://github.com/openstack/charm-interface-neutron-plugin>`__
- `service-control <https://github.com/openstack/charm-interface-service-control>`__
**Neutron ODL Based SDN Plugin**
- `neutron-plugin <https://github.com/openstack/charm-interface-neutron-plugin>`__
- `service-control <https://github.com/openstack/charm-interface-service-control>`__
- `ovsdb-manager <https://github.com/openstack/charm-interface-ovsdb-manager>`__
- `odl-controller-api <https://github.com/openstack/charm-interface-odl-controller-api>`__
**Neutron API Plugin**
- `neutron-plugin-api-subordinate <https://github.com/openstack/charm-interface-neutron-plugin-api-subordinate>`__
- `service-control <https://github.com/openstack/charm-interface-service-control>`__
.. _`charms.openstack`:
charms.openstack
----------------
The `charms.openstack <https://github.com/openstack/charms.openstack>`__ python
module provides helpers for building layered, reactive OpenStack charms. It is
installed by the `OpenStack Layer`_ .
Defining the Charm
------------------
The charm is defined be extending the OpenStackCharm or OpenStackCharmAPI base
classes in **src/lib/charm/openstack/new_charm_name.py** and overriding the
class attributes as needed.
For example to define a charm for a service called 'new-service':
.. code:: python
import charms_openstack.charm
class NewServiceCharm(charms_openstack.charm.OpenStackCharm):
# The name of the charm (for printing, etc.)
name = 'new-service'
# List of packages to install
packages = ['glance-common']
# The list of required services that are checked for assess_status
# e.g. required_relations = ['identity-service', 'shared-db']
required_relations = ['keystone']
# A dictionary of:
# {
# 'config.file': ['list', 'of', 'services', 'to', 'restart'],
# 'config2.file': ['more', 'services'],
# }
# The files that for the keys of the dict are monitored and if the file
# changes the corresponding services are restarted
restart_map = {
'/etc/new-svc/new-svc.conf': ['new-charm-svc']}
# first_release = this is the first release in which this charm works
release = 'icehouse'
def configure_foo(self):
...
The charm definition above can also define methods, like configure_foo, that
the charm handlers can call to run charm specific code.
Reacting to Events
------------------
Reactive charms react to events. These events could be raised by interfaces or
by other handlers. A number of event handlers are added by default by the
`charms.openstack`_ module. For example, an install handler runs by default and
will install the packages which were listed in NewServiceCharm.packages. Once
complete the 'charm.installed' state is raised. The charms handlers specific
to the new charm are defined in
**src/reactive/new_charm_name_handlers.py**
For example, once the packages are installed it is likely that additional
configuration is needed e.g. rendering config, configuring bridges or updating
remote services via their interfaces. To perform an action once the initial
package installation has been done a handler needs to be added to listen for
the **charm.installed** event. To do this edit
**src/reactive/new_charm_name_handlers.py** and add the reactive handler:
.. code:: python
@reactive.when('charm.installed')
def configure_foo():
with charm.provide_charm_instance() as new_charm:
new_charm.configure_foo()
If configure_foo() should only be run once then the handler can emit a new
state and the running of configure_foo gated on the state not being present
e.g.
.. code:: python
@reactive.when_not('foo.configured')
@reactive.when('charm.installed')
def configure_foo():
with charm.provide_charm_instance() as new_charm:
new_charm.configure_foo()
reactive.set_state('foo.configured')
File Templates
--------------
Most charms need to write a configuration file from a template. The templates
are stored in **src/templates** see `Templates Directory`_ for more details. The
context used to populate the template has a number of namespaces which are
populated from different sources. Below outlines those namespaces.
.. NOTE::
Hypens are always automatically converted to underscores in the template
context.
Template properties from Interfaces
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
By default some interfaces are automatically allocated a namespace within the
template context. Those namespaces are also automatically populated with some
options directly from the interface. For example if a charm is related to
Keystone's `keystone interface <https://github.com/openstack/charm-interface-keystone>`__
then a number of **service\_** variables are set in the
identity\_service namespace. So, charm template could contain the following to
access those variables:
.. code:: python
[keystone_authtoken]
auth_uri = {{ identity_service.service_protocol }}://{{ identity_service.service_host }}:{{ identity_service.service_port }}
auth_url = {{ identity_service.auth_protocol }}://{{ identity_service.auth_host }}:{{ identity_service.auth_port }}
See the **auto\_accessors** list in `charm-interface-keystone <https://github.com/openstack/charm-interface-keystone/blob/master/requires.py>`__
for a complete list
However, most interface data is accessed via Adapters...
Template properties from Adapters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Adapters are used to take the data from an interface and create new variables
in the template context. For example the **RabbitMQRelationAdapter** (which can
be found in the `adapters.py <https://github.com/openstack/charms.openstack/blob/master/charms_openstack/adapters.py>`__
from charms.openstack.) adds an **ssl\_ca\_file** variable to the amqp
namespace. This setting is really independent of the interface with rabbit but
should be consistent across the OpenStack deployment. This variable can then
be accessed in the same way as the rest of the amqp setting ``{{amqp.ssl_ca_file }}``
Template properties from user config
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The settings exposed to the user via the config.yaml are added to the
**options** namespace. The value the user has set for option **foo** can be
retrieved inside a template by including ``{{ options.foo }}``
Template properties added to user config
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is useful to be able to set a property based on examining multiple config
options or examining other aspects of the runtime system. The
**charms_openstack.adapters.config_property** decorator can be used to achieve
this. In the example below if the user has set the boolean config option
**angry** to **True** and set the **radiation** string config option to
**gamma** then the **hulk_mode** property is set to True.
.. code:: python
@charms_openstack.adapters.config_property
def hulk_mode(config):
if config.angry and config.radiation =='gamma':
return True
else:
return False
This can be accessed in the templates with ``{{ options.hulk_mode }}``
Template properties added to an Adapter
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To be able to set a property based on the settings retrieved from an interface.
In the example below the charm sets a pipeline based on the Keystone API
version advertised by the keystone interface,
.. code:: python
@charms_openstack.adapters.adapter_property('identity_service')
def charm_pipeline(keystone):
return {
"2": "cors keystone_authtoken context apiapp",
"3": "cors keystone_v3_authtoken context apiapp",
"none": "cors unauthenticated-context apiapp"
}[keystone.api_version]
This can be accessed in the templates with ``{{ identity_service.charm_pipeline }}``
.. _`Templates Directory`:
Templates Directory
~~~~~~~~~~~~~~~~~~~
Template are loaded from several places in the following order:
- From the most recent OS release-specific template dir (if one exists)
- Working back through the template directories for each earlier OpenStack Release
- The base templates_dir
For the example above, 'templates' contains the following structure:
::
templates/nova.conf
templates/api-paste.ini
templates/kilo/api-paste.ini
templates/newton/api-paste.ini
If the charm is deploying the Newton release, it first searches
the newton directory for nova.conf, then the templates dir. So
**templates/nova.conf** will be used.
When writing api-paste.ini, it will find the template in the newton
directory.
However if Liberty was being installed then the charm would fall back to the
kilo template for api-paste.ini since there is no Liberty specific version.
Rendering a Template
~~~~~~~~~~~~~~~~~~~~
Rendering the templates does not usually make sense until all the interfaces
that are going to supply the template context with data are ready and
available. The ``@reactive.when`` decorator not only ensures that the wrapped
method is not run until the interface is ready, it also passes an instance of
the interface to the method it is wrapping. These interfaces can then be passed
to the render_with_interfaces class which looks after finding the templates
and rendering them. render_with_interfaces decides which files need rendering
by examining the keys of the restart_map dict which was specified as part of
the charm class. Taking all this together results in a handler like this:
.. code:: python
@reactive.when('shared-db.available')
@reactive.when('identity-service.available')
@reactive.when('amqp.available')
def render_config(*args):
with charm.provide_charm_instance() as new_charm:
new_charm.render_with_interfaces(args)
new_charm.assess_status()
Sending data via an Interface
-----------------------------
Some interfaces are used to send as well as receive data. The interface will
expose a method for sending data to a remote application if it is supported.
For example the `neutron-plugin interface <https://github.com/openstack/charm-interface-neutron-plugin>`__
can be used to send configuration to the principle charm.
The handler below waits for the neutron-plugin relation with the principle to
be complete at which point the **neutron-plugin.connected** state will be set
which will fire this trigger. An instance of the interface is passed by the
decorator to the **configure_neutron_plugin** method. This is in turn passed to
the **configure_neutron_plugin** method in the charm class.
.. code:: python
@reactive.when('neutron-plugin.connected')
def configure_neutron_plugin(neutron_plugin):
with charm.provide_charm_instance() as new_charm:
new_charm.configure_neutron_plugin(neutron_plugin)
In the charm class the instance of the interface is used to update the
principle
.. code:: python
def configure_neutron_plugin(self, neutron_plugin):
neutron_plugin.configure_plugin(
plugin='mysdn',
config={
"nova-compute": {
"/etc/nova/nova.conf": {
"sections": {
'DEFAULT': [
('firewall_driver',
'nova.virt.firewall.'
'NoopFirewallDriver'),
('libvirt_vif_driver',
'nova.virt.libvirt.vif.'
'LibvirtGenericVIFDriver'),
('security_group_api', 'neutron'),
],
}
}
}
})
On receiving this data from the neutron_plugin relation the principle will add
the requested config into **/etc/nova/nova.conf**
.. NOTE::
The amqp, shared-db and identity-service interfaces are automatically
updated so there is no need to add code for them unless a bespoke
configuration is needed.
Displaying Charm Status
-----------------------
The charm can declare what state it is in and this status is displayed to the
user via *juju status*. By default the charm code will look for the
``required_relations`` attribute of the charm class. ``required_relations`` is
a list of interfaces. e.g. for an API charm ...
.. code:: python
required_relations = ['shared-db', 'amqp', 'identity-service']
The in built ``assess_status()`` method will check that each interface has
raised the `{relation}.available` state. If the relation is missing altogether
or if the relation has yet to raise the `{relation}.available` state then a
message is returned via ``juju status``

12
doc/source/creating-charms.rst Normal file
View File

@ -0,0 +1,12 @@
==============
Create A Charm
==============
.. toctree::
:maxdepth: 1
:hidden:
charm-anatomy
new-sdn-charm
new-api-charm

1
doc/source/index.rst
View File

@ -12,6 +12,7 @@ with lose coupling between OpenStack Services.
getting-started
deployment
openstack-charms
creating-charms
how-to-contribute
find-us
releases

277
doc/source/new-api-charm.rst Normal file
View File

@ -0,0 +1,277 @@
.. _new_api_charm:
=============
New API Charm
=============
The example below will walk through the creation of a basic API charm for the
Openstack `Congress <https://wiki.openstack.org/wiki/Congress>`__ service.
The charm will use prewritten Openstack `layers <https://github.com/openstack?query=charm-layer>`__
and `interfaces <https://github.com/openstack?query=charm-interface>`__. Once the charm
is written it will be composed using `charm tools <https://github.com/juju/charm-tools/>`__.
For more details of the internal of a charm see Charm Anatomy.
Before writing a new charm the charm author needs to have a clear idea of what
applications the charm is going to need to relate to, what files and services
the charm is going to manage and possibly what files or services do other
charms manage that need updating.
The Congress service needs to register endpoints with Keystone. It needs a
service username and password and it also needs a MySQL backend to store its
schema.
Create the skeleton charm
=========================
Prerequists
~~~~~~~~~~~
The charm-tools package and charm-templates-openstack python module are both
needed to construct the charm from a template and to build the resulting charm.
.. code:: bash
sudo apt-get install charm-tools python-jinja2
mkdir ~/congress-charm
cd ~/congress-charm
git clone git@github.com:openstack-charmers/charm-templates-openstack.git
cd charm-templates-openstack
sudo ./setup.py install
Create Charm
~~~~~~~~~~~~
Charm tools provides a utility for building an initial charm from a template.
The charm can be thought of as the top layer, the OpenStack layers sit beneath
it and the reactive base layer is at the bottom.
During the charm generation charm tools asks a few questions about the charm.
.. code::
cd ~/congress-charm
charm-create -t openstack-api congress
All the questions are optional, below are the responses for Congress.
.. code::
What port does the primary service listen on ? 1789
What is the name of the api service? congress-server
What type of service is this (used for keystone registration)? congress
What is the earliest OpenStack release this charm is compatable with? mitaka
Where command is used to sync the database? congress-db-manage --config-file /etc/congress/congress.conf upgrade head
What packages should this charm install (space seperated list)? congress-server congress-common python-antlr3 python-pymysql
List of config files managed by this charm (space seperated) /etc/congress/congress.conf
What is the name of the init script which controls the primary service congress-server
Configuration Files
~~~~~~~~~~~~~~~~~~~
The charm code searches through the templates directories looking for a
directory corresponding to the OpenStack release being installed or earlier.
Since Mitaka is the earliest release the charm is supporting a directory called
mitaka will house the templates and files.
A template for congress.conf is needed which will have have connection
information for MySQL and Keystone as well as user controllable config options.
Create **~/congress-charm/congress/src/templates/mitaka/congress.conf** with
the following contents:
.. code:: bash
[DEFAULT]
bind_host = {{ options.service_listen_info.congress_server.ip }}
bind_port = {{ options.service_listen_info.congress_server.port }}
auth_strategy = keystone
drivers = congress.datasources.neutronv2_driver.NeutronV2Driver,congress.datasources.glancev2_driver.GlanceV2Driver,congress.datasources.nova_driver.NovaDriver,congress.datasources.keystone_driver.KeystoneDriver,congress.datasources.ceilometer_driver.CeilometerDriver,congress.datasources.cinder_driver.CinderDriver,congress.datasources.swift_driver.SwiftDriver,congress.datasources.plexxi_driver.PlexxiDriver,congress.datasources.vCenter_driver.VCenterDriver,congress.datasources.murano_driver.MuranoDriver,congress.datasources.ironic_driver.IronicDriver
[database]
connection = {{ shared_db.uri }}
{% include "parts/section-keystone-authtoken" %}
.. _`Build Charm`:
Build Charm
~~~~~~~~~~~
The charm now needs to be built to pull down all the interfaces and layers the
charm depends on and rolled into the built charm which can be deployed.
.. code:: bash
cd ~/congress-charm/congress
charm build -o build src
Deploy Charm
~~~~~~~~~~~~
Asumming that an OpenStack cloud is already deployed, add the new Congress
charm.
.. code:: bash
juju deploy ~/congress-charm/congress/build/builds/congress
juju add-relation congress keystone
juju add-relation congress rabbitmq-server
juju add-relation congress mysql
``juju status`` will show the deployment as it proceeds.
Test Charm
~~~~~~~~~~
.. code:: bash
$ openstack catalog show congress
+-----------+---------------------------------------+
| Field | Value |
+-----------+---------------------------------------+
| endpoints | RegionOne |
| | publicURL: http://10.5.3.128:1789 |
| | internalURL: http://10.5.3.128:1789 |
| | adminURL: http://10.5.3.128:1789 |
| | |
| name | congress |
| type | policy |
+-----------+---------------------------------------+
$ openstack congress policy list
+--------------------------------------+----------------+----------+--------------+-----------------------+
| id | name | owner_id | kind | description |
+--------------------------------------+----------------+----------+--------------+-----------------------+
| 0801bffe-acd0-4644-adab-12321efa0aaf | classification | user | nonrecursive | default policy |
| 38e375ec-b769-45e6-89ad-9eb62da85c57 | action | user | action | default action policy |
+--------------------------------------+----------------+----------+--------------+-----------------------+
Scaling Out
~~~~~~~~~~~
Another unit can be added to the application to share the workload.
.. code:: bash
juju add-unit congress
Juju now shows two units of the Congress application.
.. code:: bash
$ juju status congress --format=oneline
- congress/1: 10.5.3.128 (agent:idle, workload:active)
- congress/2: 10.5.3.129 (agent:idle, workload:active)
The charm configures an instance of haproxy on each unit of the application.
Haproxy has all the backends registered within it and load balances traffic
across them.
.. code:: bash
$ juju ssh congress/1 "tail -11 /etc/haproxy/haproxy.cfg"
frontend tcp-in_congress-server_admin
bind \*:1789
acl net_10.5.3.128 dst 10.5.3.128/255.255.0.0
use_backend congress-server_admin_10.5.3.128 if net_10.5.3.128
default_backend congress-server_admin_10.5.3.128
backend congress-server_admin_10.5.3.128
balance leastconn
server congress-2 10.5.3.129:1779 check
server congress-1 10.5.3.128:1779 check
However, the congress endpoint registered in Keystone is still 10.5.3.128, so
if congress/1 dies clients will fail to connect unless they explicitly set
congress url. To fix this a Congress VIP can be registered in Keystone and
the VIP floated accross the Congress units using the hacluster charm.
Adding HA
~~~~~~~~~
The hacluster charm can manage a VIP which is registered with keystone. In
the event of a unit failure the VIP fails over to another application unit and
clients can continue without having to amend their clients.
The congress charm exposes a vip and vip_cidr config options which it passes
to the hacluster charm when the two are joined.
.. code:: bash
juju deploy hacluster
juju set-config congress vip=10.5.100.1 vip_cidr=24
juju add-relation hacluster congress
Juju status now reflects the new charms
.. code:: bash
$ juju status congress --format=oneline
- congress/1: 10.5.3.128 (agent:idle, workload:active)
- hacluster/0: 10.5.3.128 (agent:idle, workload:active)
- congress/2: 10.5.3.129 (agent:idle, workload:active)
- hacluster/1: 10.5.3.129 (agent:idle, workload:active)
Querying keystone now shows the VIP being used for the congress endpoint, and
the congress client still works unaltered.
.. code:: bash
$ openstack catalog show congress
+-----------+---------------------------------------+
| Field | Value |
+-----------+---------------------------------------+
| endpoints | RegionOne |
| | publicURL: http://10.5.100.1:1789 |
| | internalURL: http://10.5.100.1:1789 |
| | adminURL: http://10.5.100.1:1789 |
| | |
| name | congress |
| type | policy |
+-----------+---------------------------------------+
$ openstack congress policy list
+--------------------------------------+----------------+----------+--------------+-----------------------+
| id | name | owner_id | kind | description |
+--------------------------------------+----------------+----------+--------------+-----------------------+
| 0801bffe-acd0-4644-adab-12321efa0aaf | classification | user | nonrecursive | default policy |
| 38e375ec-b769-45e6-89ad-9eb62da85c57 | action | user | action | default action policy |
+--------------------------------------+----------------+----------+--------------+-----------------------+
Tidy Up
=======
License File
~~~~~~~~~~~~
The template assumes that the charm will be covered by the `Apache 2.0 License
<https://www.apache.org/licenses/LICENSE-2.0>`__. If another license is to be
used please review the copyright files.
Metadata Description
~~~~~~~~~~~~~~~~~~~~
The `src/metadata.yaml <https://jujucharms.com/docs/2.0/authors-charm-metadata>`__
describes the charm. Update the description and tags in here.
Publish Charm
~~~~~~~~~~~~~
Push charm up to your namespace in the charmstore:
.. code:: bash
cd ~/congress-charm/congress/build
charm push . cs:~<lp-usrname>/xenial/congress
To make the charm available to others:
.. code:: bash
charm grant cs:~<lp-usrname>/xenial/congress everyone

376
doc/source/new-charm.rst
View File

@ -1,376 +0,0 @@
.. _new_api_charm:
New API Charm
=============
Overview
--------
This guide will walk through the creation of a basic API charm for the Openstack
`Congress <https://wiki.openstack.org/wiki/Congress>`__ service.
The charm will use prewritten Openstack `layers and interfaces <https://github.com/openstack-charmers>`__.
Once the charm is written it will be composed using `charm tools <https://github.com/juju/charm-tools/>`__.
The Congress service needs to register endpoints with Keystone. It needs
a service username and password and it also needs a MySQL backend to
store its schema.
Create the skeleton charm
-------------------------
Firstly create a directory for the new charm and manage the charm with git.
.. code:: bash
mkdir -p congress/src
cd congress
git init
The top layer of this charm is the Congress specific code this code will live in the charm subdirectory.
.. code:: bash
mkdir -p src/{reactive,lib/charm/openstack}
Describe the Service and required layer(s)
------------------------------------------
The new charm needs a basic src/metadata.yaml to describe what service the charm provides. Edit src/metadata.yaml
.. code:: yaml
name: congress
summary: Policy as a service
description: |
Congress is an open policy framework for the cloud. With Congress, a cloud
operator can declare, monitor, enforce, and audit "policy" in a heterogeneous
cloud environment.
The `openstack-api layer <https://github.com/openstack-charmers/charm-layer-openstack-api>`__
defines a series of config options and interfaces which are mostly common across Openstack
API services e.g. including the openstack-api-layer will pull in the Keystone and MySQL
interfaces (among others) as well as the charm layers the new Congress charm can
leverage.
To instruct "charm build" to pull in the openstack-api layer edit src/layer.yaml:
.. code:: yaml
includes: ['layer:openstack-api']
Add Congress configuration
--------------------------
Define Congress attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~
There is a base OpenStackCharm class which provides the skeleton for the charm.
Creating a child class from OpenStackCharm allows Congress specific attributes
to be set, like which packages to install, which config files need rendering
etc. This is all done in the src/lib/charm/openstack/congress.py file.
.. code:: python
import charmhelpers.contrib.openstack.utils as ch_utils
import charms_openstack.charm
import charms_openstack.adapters
import charms_openstack.ip as os_ip
class CongressCharm(charms_openstack.charm.OpenStackCharm):
# name of service to register into keystone
service_name = 'congress'
# Internal name of charm - used for HA support + others
name = 'congress'
# First release of openstack this charm supports
release = 'mitaka'
# Packages the service needs installed
packages = ['congress-server', 'congress-common', 'python-antlr3',
'python-pymysql']
# Init services the charm manages
services = ['congress-server']
# Standard interface adapters class to use.
adapters_class = charms_openstack.adapters.OpenStackRelationAdapters
# Ports that need exposing.
default_service = 'congress-api'
api_ports = {
'congress-api': {
os_ip.PUBLIC: 1789,
os_ip.ADMIN: 1789,
os_ip.INTERNAL: 1789,
}
}
# Database sync command used to initalise the schema.
sync_cmd = ['congress-db-manage', '--config-file',
'/etc/congress/congress.conf', 'upgrade', 'head']
# The restart map defines which services should be restarted when a given
# file changes
restart_map = {
'/etc/congress/congress.conf': ['congress-server'],
'/etc/congress/api-paste.ini': ['congress-server'],
'/etc/congress/policy.json': ['congress-server'],
}
def __init__(self, release=None, **kwargs):
"""Custom initialiser for class
If no release is passed, then the charm determines the release from the
ch_utils.os_release() function.
"""
if release is None:
release = ch_utils.os_release('python-keystonemiddleware')
super(CongressCharm, self).__init__(release=release, **kwargs)
def install(self):
"""Customise the installation, configure the source and then call the
parent install() method to install the packages
"""
self.configure_source()
# and do the actual install
super(CongressCharm, self).install()
For reasons methods are needed to wrap the calls to the Congress charms class
methods. These can be appended to the bottom of the
src/lib/charm/openstack/congress.py file.
.. code:: python
def install():
"""Use the singleton from the CongressCharm to install the packages on the
unit
"""
CongressCharm.singleton.install()
def restart_all():
"""Use the singleton from the CongressCharm to restart services on the
unit
"""
CongressCharm.singleton.restart_all()
def db_sync():
"""Use the singleton from the CongressCharm to run db migration
"""
CongressCharm.singleton.db_sync()
def setup_endpoint(keystone):
"""When the keystone interface connects, register this unit in the keystone
catalogue.
"""
charm = CongressCharm.singleton
keystone.register_endpoints(charm.service_name,
charm.region,
charm.public_url,
charm.internal_url,
charm.admin_url)
def render_configs(interfaces_list):
"""Using a list of interfaces, render the configs and, if they have
changes, restart the services on the unit.
"""
CongressCharm.singleton.render_with_interfaces(interfaces_list)
def assess_status():
"""Just call the CongressCharm.singleton.assess_status() command to update
status on the unit.
"""
CongressCharm.singleton.assess_status()
Add Congress code to react to events
------------------------------------
Install Congress Packages
~~~~~~~~~~~~~~~~~~~~~~~~~
The reactive framework is going to emit events that the Congress charm can react
to. The charm needs to define how its going to react to these events and also
raise new events as needed.
The first action a charm needs to do is to install the Congress code. This is
by done running the install method from CongressCharm created earlier.
Edit src/reactive/handlers.py.
.. code:: python
import charms.reactive as reactive
import charmhelpers.core.hookenv as hookenv
# This charm's library contains all of the handler code associated with
# congress
import charm.openstack.congress as congress
# use a synthetic state to ensure that it get it to be installed independent of
# the install hook.
@reactive.when_not('charm.installed')
def install_packages():
congress.install()
reactive.set_state('charm.installed')
Configure Congress Relation
~~~~~~~~~~~~~~~~~~~~~~~~~~~
At this point the charm could be built and deployed and it would deploy a unit,
and install congress. However there is no code to specify how this charm should
interact with the services it depend on. For example when joining the database
the charm needs to specify the user and database it requires. The following code
configures the relations with the dependant services.
.. note:: ``assess_status()``: when a relation changes the workload
status may be changed. e.g. if a interface is complete in the sense
that it is connected and all information is available, then that
interface will set the `{relation}.available` (by convention).
Thus the workload status could change to 'waiting' from 'blocked'.
Append to src/reactive/handlers.py:
.. code:: python
@reactive.when('amqp.connected')
def setup_amqp_req(amqp):
"""Use the amqp interface to request access to the amqp broker using our
local configuration.
"""
amqp.request_access(username='congress',
vhost='openstack')
congress.assess_status()
@reactive.when('shared-db.connected')
def setup_database(database):
"""On receiving database credentials, configure the database on the
interface.
"""
database.configure('congress', 'congress', hookenv.unit_private_ip())
congress.assess_status()
@reactive.when('identity-service.connected')
def setup_endpoint(keystone):
congress.setup_endpoint(keystone)
congress.assess_status()
Configure Congress
------------------
Now that the charm has the relations defined that it needs the Congress charm
is in a postion to generate its configuration files.
Create templates
~~~~~~~~~~~~~~~~
The charm code searches through the templates directories looking for a directory
corresponding to the Openstack release being installed or earlier. Since Mitaka
is the earliest release the charm is supporting a directory called mitaka will
house the templates and files.
.. code:: bash
( cd /tmp; apt-get source congress-server; )
mkdir -p templates/mitaka
cp /tmp/congress*/etc/{api-paste.ini,policy.json} templates/mitaka
A template for congress.conf is needed which will have have connection
information for MySQL, RabbitMQ and Keystone as well as user controllable
config options
.. code:: bash
[DEFAULT]
auth_strategy = keystone
drivers = congress.datasources.neutronv2_driver.NeutronV2Driver,congress.datasources.glancev2_driver.GlanceV2Driver,congress.datasources.nova_driver.NovaDriver,congress.datasources.keystone_driver.KeystoneDriver,congress.datasources.ceilometer_driver.CeilometerDriver,congress.datasources.cinder_driver.CinderDriver,congress.datasources.swift_driver.SwiftDriver,congress.datasources.plexxi_driver.PlexxiDriver,congress.datasources.vCenter_driver.VCenterDriver,congress.datasources.murano_driver.MuranoDriver,congress.datasources.ironic_driver.IronicDriver
[database]
connection = {{ shared_db.uri }}
[keystone_authtoken]
{% if identity_service.auth_host -%}
auth_uri = {{ identity_service.service_protocol }}://{{
identity_service.service_host }}:{{ identity_service.service_port }}
auth_url = {{ identity_service.auth_protocol }}://{{ identity_service.auth_host
}}:{{ identity_service.auth_port }}
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = {{ identity_service.service_tenant }}
username = {{ identity_service.service_username }}
password = {{ identity_service.service_password }}
{% endif -%}
Render the config
~~~~~~~~~~~~~~~~~
Now the templates and interfaces are in place the configs can be
rendered. A side-effect of rendering the configs is that any associated
services are restarted. Finally, set the config.complete state this
will be used later to trigger other events.
Append to charm/reactive/handlers.py
.. code:: python
@reactive.when('shared-db.available')
@reactive.when('identity-service.available')
@reactive.when('amqp.available')
def render_stuff(*args):
congress.render_configs(args)
reactive.set_state('config.complete')
Run DB Migration
~~~~~~~~~~~~~~~~
The DB migration can only be run once the config files are in place
since as congress.conf will contain the DB connection information.
To achieve this the DB migration is gated on the config.complete
being set. Finally set the db.synched event so that this is only
run once.
Append to src/reactive/handlers.py
.. code:: python
@reactive.when('config.complete')
@reactive.when_not('db.synced')
def run_db_migration():
congress.db_sync()
congress.restart_all()
reactive.set_state('db.synced')
congress.assess_status()
Build and Deploy charm
----------------------
Build the charm to pull down the interfaces and layers.
.. code:: bash
mkdir build
charm build -obuild src
The built charm can now be deployed with Juju.
.. code:: bash
juju deploy <full path>/build/congress
juju add-relation congress mysql
juju add-relation congress keystone
juju add-relation congress rabbitmq-server
Deploying an existing Openstack environment is not covered here.

129
doc/source/new-sdn-charm.rst Normal file
View File

@ -0,0 +1,129 @@
.. _new_sdn_charm:
=============
New SDN Charm
=============
Before writing the charm the charm author needs to have a clear idea of what
applications the charm is going to need to relate to, what files and services
the charm is going to manage and possibly what files or services do other
charms manage that need updating.
In the example below we will assume that a new charm, VirtualTokenRing, is
needed to install a component on compute nodes and to inject some
configuration into nova.conf.
Prerequisites
===========
This will change once the OpenStack templates are on pypi
.. code:: bash
mkdir sdn-charm
cd ~/sdn-charm
git clone git@github.com:gnuoy/charm_templates_openstack.git
cd charm_templates_openstack
sudo ./setup.py install
Create Charm
============
Charm tools provides a utility for building an initial charm from a template.
During the charm generation charm tools asks a few questions about the charm.
.. code:: bash
cd ~/sdn-charm
charm-create -t openstack-neutron-plugin virtual-token-ring
INFO: Generating charm for virtual-token-ring in ./virtual-token-ring
INFO: No virtual-token-ring in apt cache; creating an empty charm instead.
What is the earliest OpenStack release this charm is compatable with? liberty
What packages should this charm install (space seperated list)?
.. _`Build Charm`:
Build Charm
===========
The charm now needs to be built to pull down all the interfaces and layers the
charm depends on and rolled into the built charm which can be deployed.
.. code:: bash
cd ~/sdn-charm/virtual-token-ring
charm build -o build src
Deploy Charm
============
.. code:: bash
cd build
juju deploy cs:xenial/nova-compute
juju deploy ~/sdn-charm/virtual-token-ring/build/builds/virtual-token-ring
juju add-relation nova-compute virtual-token-ring
``juju status`` will now show both charms deployed. The ``nova-compute`` status
will show some missing relations but that's not an issue for this demonstration.
Updating nova.conf
==================
During the initial install of this SDN charm, the standard charms.openstack
default installer will install the packages specified in the class
CharmName.packages, but it will not do any other configuration.
In order to update nova.conf in the nova-compute principal charm, this
virtual-token-ring subordinate charm will need to access the `neutron plugin <https://github.com/openstack/charm-interface-neutron-plugin>`__
interface, which will allow it to send configuration information to the
nova-computer principal charm for inclusion in nova.conf on the co-located
machine.
Return to the **virtual-token-ring** directory and edit
**src/reactive/virtual_token_ring_handlers.py**. Add any config that needs
setting in nova.conf.
.. code:: python
@reactive.when('neutron-plugin.connected')
def configure_neutron_plugin(neutron_plugin):
neutron_plugin.configure_plugin(
plugin='ovs',
config={
"nova-compute": {
"/etc/nova/nova.conf": {
"sections": {
'DEFAULT': [
('random_option', 'true'),
],
}
}
}
})
This tells the charm to send that configuration to the principle where the
**neutron-plugin.connected** event has been raised. Then repeat the `Build Charm`_
steps.
Deploy Update
=============
The freshly built charm which contains the update now needs to be deployed to
the environment.
.. code:: bash
juju upgrade-charm --path ~/sdn-charm/virtual-token-ring/build/builds/virtual-token-ring virtual-token-ring
Check Update
============
.. code:: bash
juju run --unit nova-compute/0 "grep random_option /etc/nova/nova.conf"
random_option = true