openstack/magnum
Code Issues Proposed changes

Update Dev Quick-Start links to officail docs

Browse Source 
Moved all Dev Quick-Start realted documents to docs/source folder
(html files will be genrated for them at
http://docs.openstack.org/developer/magnum),
Updated all Dev Quick-Start interanl links to
http://docs.openstack.org/developer/magnum, converted Mesos cluster
with Heat to rst (requred for docs)

Change-Id: Ibba5f44fc5ce9cab30adc0957b6103c811329a06
This commit is contained in:
Egor Guz 2015-10-04 21:54:37 -07:00
parent 6e5256b809
commit 020fae5d84
6 changed files with 308 additions and 271 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

107
contrib/templates/example/README.rst
View File

@ -2,109 +2,4 @@
Example Bay Template Example Bay Template
==================== ====================
This project is an example to demonstrate the necessary pieces of a Bay See `<http://docs.openstack.org/developer/magnum/dev/dev-bay-template-example.html>`_ for instructions.
template. There are three key pieces to a bay template:
1. Heat template - The Heat template that Magnum will use to generate a Bay.
2. Template definition - Magnum's interface for interacting with the Heat
template.
3. Definition Entry Point - Used to advertise the available template
definitions.
The Heat Template
-----------------
The heat template is where most of the real work happens. The result of the
Heat template should be a full Container Orchestration Environment.
The Template Definition
-----------------------
Template definitions are a mapping of Magnum object attributes and Heat
template parameters, along with Magnum consumable template outputs. Each
definition also denotes which Bay Types it can provide. Bay Types are how
Magnum determines which of the enabled Template Definitions it will use for a
given Bay.
The Definition Entry Point
--------------------------
Entry points are a standard discovery and import mechanism for Python objects.
Each Template Definition should have an Entry Point in the
`magnum.template_definitions` group. This example exposes it's Template
Definition as `example_template = example_template:ExampleTemplate` in the
`magnum.template_definitions` group.
Installing Bay Templates
------------------------
Because Bay Templates are basically Python projects, they can be worked with
like any other Python project. They can be cloned from version control and
installed or uploaded to a package index and installed via utilities such as
pip.
Enabling a template is as simple as adding it's Entry Point to the
`enabled_definitions` config option in magnum.conf.::
# Setup python environment and install Magnum
$ virtualenv .venv
$ source .venv/bin/active
(.venv)$ git clone https://github.com/openstack/magnum.git
(.venv)$ cd magnum
(.venv)$ python setup.py install
# List installed templates, notice default templates are enabled
(.venv)$ magnum-template-manage list-templates
Enabled Templates
magnum_vm_atomic_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster.yaml
magnum_vm_coreos_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster-coreos.yaml
Disabled Templates
# Install example template
(.venv)$ cd contrib/templates/example
(.venv)$ python setup.py install
# List installed templates, notice example template is disabled
(.venv)$ magnum-template-manage list-templates
Enabled Templates
magnum_vm_atomic_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster.yaml
magnum_vm_coreos_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster-coreos.yaml
Disabled Templates
example_template: /home/example/.venv/local/lib/python2.7/site-packages/ExampleTemplate-0.1-py2.7.egg/example_template/example.yaml
# Enable example template by setting enabled_definitions in magnum.conf
(.venv)$ sudo mkdir /etc/magnum
(.venv)$ sudo bash -c "cat > /etc/magnum/magnum.conf << END_CONF
[bay]
enabled_definitions=magnum_vm_atomic_k8s,magnum_vm_coreos_k8s,example_template
END_CONF"
# List installed templates, notice example template is now enabled
(.venv)$ magnum-template-manage list-templates
Enabled Templates
example_template: /home/example/.venv/local/lib/python2.7/site-packages/ExampleTemplate-0.1-py2.7.egg/example_template/example.yaml
magnum_vm_atomic_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster.yaml
magnum_vm_coreos_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster-coreos.yaml
Disabled Templates
# Use --details argument to get more details about each template
(.venv)$ magnum-template-manage list-templates --details
Enabled Templates
example_template: /home/example/.venv/local/lib/python2.7/site-packages/ExampleTemplate-0.1-py2.7.egg/example_template/example.yaml
Server_Type OS CoE
vm example example_coe
magnum_vm_atomic_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster.yaml
Server_Type OS CoE
vm fedora-atomic kubernetes
magnum_vm_coreos_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster-coreos.yaml
Server_Type OS CoE
vm coreos kubernetes
Disabled Templates

110
doc/source/dev/dev-bay-template-example.rst Normal file
View File

@ -0,0 +1,110 @@
====================
Example Bay Template
====================
This project is an example to demonstrate the necessary pieces of a Bay
template. There are three key pieces to a bay template:
1. Heat template - The Heat template that Magnum will use to generate a Bay.
2. Template definition - Magnum's interface for interacting with the Heat
template.
3. Definition Entry Point - Used to advertise the available template
definitions.
The Heat Template
-----------------
The heat template is where most of the real work happens. The result of the
Heat template should be a full Container Orchestration Environment.
The Template Definition
-----------------------
Template definitions are a mapping of Magnum object attributes and Heat
template parameters, along with Magnum consumable template outputs. Each
definition also denotes which Bay Types it can provide. Bay Types are how
Magnum determines which of the enabled Template Definitions it will use for a
given Bay.
The Definition Entry Point
--------------------------
Entry points are a standard discovery and import mechanism for Python objects.
Each Template Definition should have an Entry Point in the
`magnum.template_definitions` group. This example exposes it's Template
Definition as `example_template = example_template:ExampleTemplate` in the
`magnum.template_definitions` group.
Installing Bay Templates
------------------------
Because Bay Templates are basically Python projects, they can be worked with
like any other Python project. They can be cloned from version control and
installed or uploaded to a package index and installed via utilities such as
pip.
Enabling a template is as simple as adding it's Entry Point to the
`enabled_definitions` config option in magnum.conf.::
# Setup python environment and install Magnum
$ virtualenv .venv
$ source .venv/bin/active
(.venv)$ git clone https://github.com/openstack/magnum.git
(.venv)$ cd magnum
(.venv)$ python setup.py install
# List installed templates, notice default templates are enabled
(.venv)$ magnum-template-manage list-templates
Enabled Templates
magnum_vm_atomic_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster.yaml
magnum_vm_coreos_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster-coreos.yaml
Disabled Templates
# Install example template
(.venv)$ cd contrib/templates/example
(.venv)$ python setup.py install
# List installed templates, notice example template is disabled
(.venv)$ magnum-template-manage list-templates
Enabled Templates
magnum_vm_atomic_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster.yaml
magnum_vm_coreos_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster-coreos.yaml
Disabled Templates
example_template: /home/example/.venv/local/lib/python2.7/site-packages/ExampleTemplate-0.1-py2.7.egg/example_template/example.yaml
# Enable example template by setting enabled_definitions in magnum.conf
(.venv)$ sudo mkdir /etc/magnum
(.venv)$ sudo bash -c "cat > /etc/magnum/magnum.conf << END_CONF
[bay]
enabled_definitions=magnum_vm_atomic_k8s,magnum_vm_coreos_k8s,example_template
END_CONF"
# List installed templates, notice example template is now enabled
(.venv)$ magnum-template-manage list-templates
Enabled Templates
example_template: /home/example/.venv/local/lib/python2.7/site-packages/ExampleTemplate-0.1-py2.7.egg/example_template/example.yaml
magnum_vm_atomic_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster.yaml
magnum_vm_coreos_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster-coreos.yaml
Disabled Templates
# Use --details argument to get more details about each template
(.venv)$ magnum-template-manage list-templates --details
Enabled Templates
example_template: /home/example/.venv/local/lib/python2.7/site-packages/ExampleTemplate-0.1-py2.7.egg/example_template/example.yaml
Server_Type OS CoE
vm example example_coe
magnum_vm_atomic_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster.yaml
Server_Type OS CoE
vm fedora-atomic kubernetes
magnum_vm_coreos_k8s: /home/example/.venv/local/lib/python2.7/site-packages/magnum/templates/heat-kubernetes/kubecluster-coreos.yaml
Server_Type OS CoE
vm coreos kubernetes
Disabled Templates

191
doc/source/dev/dev-heat-mesos.rst Normal file
View File

@ -0,0 +1,191 @@
A Mesos cluster with Heat
=========================
These `Heat <https://wiki.openstack.org/wiki/Heat>`__ templates will
deploy a `Mesos <http://mesos.apache.org/>`__ cluster.
Requirements
------------
OpenStack
~~~~~~~~~
These templates will work with the Kilo version of Heat.
Guest image
~~~~~~~~~~~
These templates will work with Ubuntu 14.04 base image with the
following middleware pre-installed:
- ``docker``
- ``zookeeper``
- ``mesos``
- ``marathon``
Building an image
~~~~~~~~~~~~~~~~~
If you do not have a suitable image you can build one easily using one
of two methods:
Disk Image Builder
^^^^^^^^^^^^^^^^^^
`elements <http://git.openstack.org/cgit/openstack/magnum/tree/magnum/templates/heat-mesos/elements/>`__
directory contains `diskimage-builder <http://docs.openstack.org/developer/diskimage-builder>`__
elements to build an image which contains mesos and its frameworks
required to use the heat template mesoscluster.yaml.
Currently, only Ubuntu 14.04 is supported. An example Ubuntu based image
can be built and uploaded to glance as follows:
::
$ sudo apt-get update
$ sudo apt-get install git qemu-utils python-pip
$ sudo pip install pyyaml
$ git clone https://git.openstack.org/openstack/magnum
$ git clone https://git.openstack.org/openstack/diskimage-builder.git
$ git clone https://git.openstack.org/openstack/dib-utils.git
$ export PATH="${PWD}/dib-utils/bin:$PATH"
$ export ELEMENTS_PATH=magnum/magnum/templates/heat-mesos/elements
$ export DIB_RELEASE=trusty
$ diskimage-builder/bin/disk-image-create ubuntu vm docker mesos \
-o ubuntu-mesos.qcow2
$ glance image-create --name ubuntu-mesos --visibility public \
--disk-format=qcow2 --container-format=bare \
--property os_distro=ubuntu --file=ubuntu-mesos.qcow2
Docker
^^^^^^
Install docker using ``curl -sSL http://get.docker.com | sudo bash`` or
use the appropriate system packaging.
Use the provided `Dockerfile <http://git.openstack.org/cgit/openstack/magnum/tree/magnum/templates/heat-mesos/Dockerfile>`__ to build the image (it
uses the same DIB scripts as above). The resultant image will be saved
as ``/tmp/ubuntu-mesos.qcow2``
::
$ sudo docker build -t magnum/mesos-builder .
$ sudo docker run -v /tmp:/output --rm -ti --privileged magnum/mesos-builder
...
Image file /output/ubuntu-mesos.qcow2 created...
$ glance image-create --name ubuntu-mesos --is-public True \
--disk-format=qcow2 --container-format=bare \
--property os_distro=ubuntu --file=/tmp/ubuntu-mesos.qcow2
Creating the stack
------------------
Creating an environment file ``local.yaml`` with parameters specific to
your environment:
::
parameters:
ssh_key_name: testkey
external_network: public
dns_nameserver: 8.8.8.8
server_image: ubuntu-mesos
And then create the stack, referencing that environment file:
::
heat stack-create -f mesoscluster.yaml -e local.yaml my-mesos-cluster
You must provide value for:
- ``ssh_key_name``
You can optionally provide values for:
- ``server_image`` (ubuntu-mesos if not provided)
- ``external_network`` (public if not provided)
- ``dns_nameserver`` (8.8.8.8 if not provided)
Interacting with Mesos
----------------------
You can get the ip address of the Mesos master using the
``heat output-show`` command:
::
$ heat output-show my-mesos-cluster mesos_master
"192.168.200.86"
You can ssh into that server as the ``ubuntu`` user:
::
$ ssh ubuntu@192.168.200.86
You can log into your slaves using the ``ubuntu`` user as well. You can
get a list of slaves addresses by running:
::
$ heat output-show my-mesos-cluster mesos_slaves
[
"192.168.200.182"
]
Testing
-------
Docker containers can be deployed via Marathon's REST API. Marathon is a
mesos framework for long running applications.
We can 'post' a JSON app description to http://:8080/apps to deploy a
Docker container.
::
$ cat > app.json << END
{
"container": {
"type": "DOCKER",
"docker": {
"image": "libmesos/ubuntu"
}
},
"id": "ubuntu",
"instances": 1,
"cpus": 0.5,
"mem": 512,
"uris": [],
"cmd": "while sleep 10; do date -u +%T; done"
}
END
$ MASTER_IP=$(heat output-show my-mesos-cluster mesos_master | tr -d '"')
$ curl -X POST -H "Content-Type: application/json" \
http://${MASTER_IP}:8080/v2/apps -d@app.json
Using the Marathon web console (at http://:8080/), you will see the
application you created.
License
-------
Copyright 2015 Huawei Technologies Co.,LTD.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use these files except in compliance with the License. You may
obtain a copy of the License at
::
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

8
doc/source/dev/dev-quickstart.rst
View File

@ -75,7 +75,7 @@ To run only the pep8/flake8 syntax and style checks::
tox -epep8 tox -epep8
To discover and interact with templates, please refer to To discover and interact with templates, please refer to
`<http://git.openstack.org/cgit/openstack/magnum/tree/contrib/templates/example/README.rst>`_ `<http://docs.openstack.org/developer/magnum/dev/dev-bay-template-example.html>`_
Exercising the Services Using Devstack Exercising the Services Using Devstack
====================================== ======================================
@ -92,7 +92,7 @@ This session has only been tested on Ubuntu 14.04 (Trusty) and Fedora 20/21.
We recommend users to select one of them if it is possible. We recommend users to select one of them if it is possible.
For in-depth guidance on adding magnum manually to a devstack instance, please For in-depth guidance on adding magnum manually to a devstack instance, please
refer to the `http://git.openstack.org/cgit/openstack/magnum/tree/doc/source/dev/dev-manual-devstack.rst`_ refer to the `<http://docs.openstack.org/developer/magnum/dev/dev-manual-devstack.html>`_
Clone devstack:: Clone devstack::
@ -368,7 +368,7 @@ Note: If you are using Magnum behind a firewall then see:
.. _Using_Magnum_Behind_Firewall: .. _Using_Magnum_Behind_Firewall:
http://docs.openstack.org/developer/magnum/magnum-proxy.rst http://docs.openstack.org/developer/magnum/magnum-proxy.html
Finally, create the bay. Use the baymodel 'swarmbaymodel' as a template for Finally, create the bay. Use the baymodel 'swarmbaymodel' as a template for
bay creation. This bay will result in one swarm manager node and two extra bay creation. This bay will result in one swarm manager node and two extra
@ -439,7 +439,7 @@ Building and Using a Mesos Bay
Provisioning a mesos bay requires a Ubuntu-based image with some packages Provisioning a mesos bay requires a Ubuntu-based image with some packages
pre-installed. To build and upload such image, please refer to pre-installed. To build and upload such image, please refer to
`http://git.openstack.org/cgit/openstack/magnum/tree/magnum/templates/heat-mesos/README.md`_ `<http://docs.openstack.org/developer/magnum/dev/dev-heat-mesos.html>`_
Then, create a baymodel by using 'mesos' as the coe, with the rest of arguments Then, create a baymodel by using 'mesos' as the coe, with the rest of arguments
similar to the Kubernetes baymodel:: similar to the Kubernetes baymodel::

161
magnum/templates/heat-mesos/README.md
View File

@ -1,163 +1,4 @@
A Mesos cluster with Heat A Mesos cluster with Heat
========================= =========================
These [Heat][] templates will deploy a [Mesos][] cluster. See [Mesos cluster with Heat](http://docs.openstack.org/developer/magnum/dev/dev-heat-mesos.html) for instructions.
[heat]: https://wiki.openstack.org/wiki/Heat
[mesos]: http://mesos.apache.org/
## Requirements
### OpenStack
These templates will work with the Kilo version of Heat.
### Guest image
These templates will work with Ubuntu 14.04 base image with the following
middleware pre-installed:
- `docker`
- `zookeeper`
- `mesos`
- `marathon`
### Building an image
If you do not have a suitable image you can build one easily using one of two methods:
#### Disk Image Builder
[elements/](elements/) directory contains [diskimage-builder](https://github.com/openstack/diskimage-builder)
elements to build an image which contains mesos and its frameworks required to
use the heat template mesoscluster.yaml.
Currently, only Ubuntu 14.04 is supported. An example Ubuntu based image can be
built and uploaded to glance as follows:
$ sudo apt-get update
$ sudo apt-get install git qemu-utils python-pip
$ sudo pip install pyyaml
$ git clone https://git.openstack.org/openstack/magnum
$ git clone https://git.openstack.org/openstack/diskimage-builder.git
$ git clone https://git.openstack.org/openstack/dib-utils.git
$ export PATH="${PWD}/dib-utils/bin:$PATH"
$ export ELEMENTS_PATH=magnum/magnum/templates/heat-mesos/elements
$ export DIB_RELEASE=trusty
$ diskimage-builder/bin/disk-image-create ubuntu vm docker mesos \
-o ubuntu-mesos.qcow2
$ glance image-create --name ubuntu-mesos --visibility public \
--disk-format=qcow2 --container-format=bare \
--property os_distro=ubuntu --file=ubuntu-mesos.qcow2
#### Docker
Install docker using `curl -sSL http://get.docker.com | sudo bash` or use
the appropriate system packaging.
Use the provided [Dockerfile](./Dockerfile) to build the image (it uses the
same DIB scripts as above). The resultant image will be saved as
`/tmp/ubuntu-mesos.qcow2`
$ sudo docker build -t magnum/mesos-builder .
$ sudo docker run -v /tmp:/output --rm -ti --privileged magnum/mesos-builder
...
Image file /output/ubuntu-mesos.qcow2 created...
$ glance image-create --name ubuntu-mesos --is-public True \
--disk-format=qcow2 --container-format=bare \
--property os_distro=ubuntu --file=/tmp/ubuntu-mesos.qcow2
## Creating the stack
Creating an environment file `local.yaml` with parameters specific to
your environment:
parameters:
ssh_key_name: testkey
external_network: public
dns_nameserver: 8.8.8.8
server_image: ubuntu-mesos
And then create the stack, referencing that environment file:
heat stack-create -f mesoscluster.yaml -e local.yaml my-mesos-cluster
You must provide value for:
- `ssh_key_name`
You can optionally provide values for:
- `server_image` (ubuntu-mesos if not provided)
- `external_network` (public if not provided)
- `dns_nameserver` (8.8.8.8 if not provided)
## Interacting with Mesos
You can get the ip address of the Mesos master using the `heat
output-show` command:
$ heat output-show my-mesos-cluster mesos_master
"192.168.200.86"
You can ssh into that server as the `ubuntu` user:
$ ssh ubuntu@192.168.200.86
You can log into your slaves using the `ubuntu` user as well. You
can get a list of slaves addresses by running:
$ heat output-show my-mesos-cluster mesos_slaves
[
"192.168.200.182"
]
## Testing
Docker containers can be deployed via Marathon's REST API.
Marathon is a mesos framework for long running applications.
We can 'post' a JSON app description to http://<master>:8080/apps to deploy
a Docker container.
$ cat > app.json << END
{
"container": {
"type": "DOCKER",
"docker": {
"image": "libmesos/ubuntu"
}
},
"id": "ubuntu",
"instances": 1,
"cpus": 0.5,
"mem": 512,
"uris": [],
"cmd": "while sleep 10; do date -u +%T; done"
}
END
$ MASTER_IP=$(heat output-show my-mesos-cluster mesos_master | tr -d '"')
$ curl -X POST -H "Content-Type: application/json" \
http://${MASTER_IP}:8080/v2/apps -d@app.json
Using the Marathon web console (at http://<master>:8080/), you will see the
application you created.
## License
Copyright 2015 Huawei Technologies Co.,LTD.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use these files except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

2
magnum/templates/heat-mesos/elements/README.md
View File

@ -1,4 +1,4 @@
Mesos elements Mesos elements
============== ==============
See [README.md](../README.md#disk-image-builder) for instructions. See [Building an image](http://docs.openstack.org/developer/magnum/dev/dev-heat-mesos.html) for instructions.