9eaf849012
Image as part of a driver for COE+distro. Describe image as part of the driver for a particular COE and particular distro: -Interaction with templates, scripts. -List software content and version. -Point to guide for building images. -How software are run: as service or container -Where to download supported images. Partially implements: blueprint user-guide Change-Id: I39b912f7474e3b4e93c775f766dbd519da9656ec
484 lines
15 KiB
ReStructuredText
484 lines
15 KiB
ReStructuredText
=================
|
|
Magnum User Guide
|
|
=================
|
|
|
|
This guide is intended for users who use Magnum to deploy and manage clusters
|
|
of hosts for a Container Orchestration Engine. It describes the infrastructure
|
|
that Magnum creates and how to work with them.
|
|
|
|
Section 1-3 describe Magnum itself, including an overview, the CLI and Horizon
|
|
interface. Section 4-8 describe the Container Orchestration Engine's supported
|
|
along with a guide on how to select one that best meets your needs. Section
|
|
9-14 describe the low level OpenStack infrastructure that is created and
|
|
managed by Magnum to support the Container Orchestration Engine's.
|
|
|
|
========
|
|
Contents
|
|
========
|
|
|
|
#. `Overview`_
|
|
#. `Python Client`_
|
|
#. `Horizon Interface`_
|
|
#. `Choosing COE`_
|
|
#. `Native clients`_
|
|
#. `Kubernetes`_
|
|
#. `Swarm`_
|
|
#. `Mesos`_
|
|
#. `Transport Layer Security`_
|
|
#. `Networking`_
|
|
#. `High Availability`_
|
|
#. `Scaling`_
|
|
#. `Storage`_
|
|
#. `Image Management`_
|
|
|
|
===========
|
|
Terminology
|
|
===========
|
|
|
|
Bay
|
|
A bay is the construct in which Magnum launches container orchestration
|
|
engines. After a bay has been created the user is able to add containers to
|
|
it either directly, or in the case of the Kubernetes container orchestration
|
|
engine within pods - a logical construct specific to that implementation. A
|
|
bay is created based on a baymodel.
|
|
|
|
Baymodel
|
|
A baymodel in Magnum is roughly equivalent to a flavor in Nova. It acts as a
|
|
template that defines options such as the container orchestration engine,
|
|
keypair and image for use when Magnum is creating bays using the given
|
|
baymodel.
|
|
|
|
Container Orchestration Engine (COE)
|
|
A container orchestration engine manages the lifecycle of one or more
|
|
containers, logically represented in Magnum as a bay. Magnum supports a
|
|
number of container orchestration engines, each with their own pros and cons,
|
|
including Docker Swarm, Kubernetes, and Mesos.
|
|
|
|
Pod
|
|
When using the Kubernetes container orchestration engine, a pod is the
|
|
smallest deployable unit that can be created and managed. A pod is a
|
|
co-located group of application containers that run with a shared context.
|
|
When using Magnum, pods are created and managed within bays. Refer to the
|
|
`pods section
|
|
<http://kubernetes.io/v1.0/docs/user-guide/pods.html>`_ in the `Kubernetes
|
|
User Guide`_ for more information.
|
|
|
|
Replication controller
|
|
A replication controller is used to ensure that at any given time a certain
|
|
number of replicas of a pod are running. Pods are automatically created and
|
|
deleted by the replication controller as necessary based on a template to
|
|
ensure that the defined number of replicas exist. Refer to the `replication
|
|
controller section
|
|
<http://kubernetes.io/v1.0/docs/user-guide/replication-controller.html>`_ in
|
|
the `Kubernetes User Guide`_ for more information.
|
|
|
|
Service
|
|
A service is an additional layer of abstraction provided by the Kubernetes
|
|
container orchestration engine which defines a logical set of pods and a
|
|
policy for accessing them. This is useful because pods are created and
|
|
deleted by a replication controller, for example, other pods needing to
|
|
discover them can do so via the service abstraction. Refer to the
|
|
`services section
|
|
<http://kubernetes.io/v1.0/docs/user-guide/services.html>`_ in the
|
|
`Kubernetes User Guide`_ for more information.
|
|
|
|
.. _Kubernetes User Guide: http://kubernetes.io/v1.0/docs/user-guide/
|
|
|
|
========
|
|
Overview
|
|
========
|
|
*To be filled in*
|
|
|
|
Magnum rationale, concept, compelling features
|
|
|
|
BayModel
|
|
---------
|
|
*To be filled in*
|
|
|
|
Bay
|
|
---
|
|
*To be filled in*
|
|
|
|
=============
|
|
Python Client
|
|
=============
|
|
|
|
Installation
|
|
------------
|
|
|
|
Follow the instructions in the OpenStack Installation Guide to enable the
|
|
repositories for your distribution:
|
|
|
|
* `RHEL/CentOS/Fedora
|
|
<http://docs.openstack.org/liberty/install-guide-rdo/>`_
|
|
* `Ubuntu/Debian
|
|
<http://docs.openstack.org/liberty/install-guide-ubuntu/>`_
|
|
* `openSUSE/SUSE Linux Enterprise
|
|
<http://docs.openstack.org/liberty/install-guide-obs/>`_
|
|
|
|
Install using distribution packages for RHEL/CentOS/Fedora::
|
|
|
|
$ sudo yum install python-magnumclient
|
|
|
|
Install using distribution packages for Ubuntu/Debian::
|
|
|
|
$ sudo apt-get install python-magnumclient
|
|
|
|
Install using distribution packages for OpenSuSE and SuSE Enterprise Linux::
|
|
|
|
$ sudo zypper install python-magnumclient
|
|
|
|
Verifying installation
|
|
----------------------
|
|
|
|
Execute the `magnum` command with the `--version` argument to confirm that the
|
|
client is installed and in the system path::
|
|
|
|
$ magnum --version
|
|
1.1.0
|
|
|
|
Note that the version returned may differ from the above, 1.1.0 was the latest
|
|
available version at the time of writing.
|
|
|
|
Using the command-line client
|
|
-----------------------------
|
|
|
|
Refer to the `OpenStack Command-Line Interface Reference
|
|
<http://docs.openstack.org/cli-reference/magnum.html>`_ for a full list of the
|
|
commands supported by the `magnum` command-line client.
|
|
|
|
=================
|
|
Horizon Interface
|
|
=================
|
|
*To be filled in with screenshots*
|
|
|
|
============
|
|
Choosing COE
|
|
============
|
|
*To be filled in*
|
|
|
|
Buyer's guide with example use case, can use this as model:
|
|
|
|
http://www.openstack.org/software/project-navigator
|
|
|
|
==============
|
|
Native clients
|
|
==============
|
|
*To be filled in*
|
|
|
|
==========
|
|
Kubernetes
|
|
==========
|
|
*To be filled in*
|
|
|
|
=====
|
|
Swarm
|
|
=====
|
|
*To be filled in*
|
|
|
|
=====
|
|
Mesos
|
|
=====
|
|
*To be filled in*
|
|
|
|
========================
|
|
Transport Layer Security
|
|
========================
|
|
*To be filled in*
|
|
|
|
Native Client Configuration guide for each COE
|
|
|
|
==========
|
|
Networking
|
|
==========
|
|
|
|
There are two components that make up the networking in a cluster.
|
|
|
|
1. The Neutron infrastructure for the cluster: this includes the
|
|
private network, subnet, ports, routers, load balancers, etc.
|
|
|
|
2. The networking model presented to the containers: this is what the
|
|
containers see in communicating with each other and to the external
|
|
world. Typically this consists of a driver deployed on each node.
|
|
|
|
The two components are deployed and managed separately. The Neutron
|
|
infrastructure is the integration with OpenStack; therefore, it
|
|
is stable and more or less similar across different COE
|
|
types. The networking model, on the other hand, is specific to the
|
|
COE type and is still under active development in the various
|
|
COE communities, for example,
|
|
`Docker libnetwork <https://github.com/docker/libnetwork>`_ and
|
|
`Kubernetes Container Networking
|
|
<https://github.com/kubernetes/kubernetes/blob/release-1.1/docs/design/networking.md>`_.
|
|
As a result, the implementation for the networking models is evolving and
|
|
new models are likely to be introduced in the future.
|
|
|
|
For the Neutron infrastructure, the following configuration can
|
|
be set in the baymodel:
|
|
|
|
external-network-id
|
|
The external Neutron network ID to connect to this bay. This
|
|
is used to connect the cluster to the external internet, allowing
|
|
the nodes in the bay to access external URL for discovery, image
|
|
download, etc. If not specified, the default value is "public" and this
|
|
is valid for a typical devstack.
|
|
|
|
fixed-network
|
|
The Neutron network to use as the private network for the bay nodes.
|
|
If not specified, a new Neutron private network will be created.
|
|
|
|
dns-nameserver
|
|
The DNS nameserver to use for this bay. This is an IP address for
|
|
the server and it is used to configure the Neutron subnet of the
|
|
cluster (dns_nameservers). If not specified, the default DNS is
|
|
8.8.8.8, the publicly available DNS.
|
|
|
|
http-proxy, https-proxy, no-proxy
|
|
The proxy for the nodes in the bay, to be used when the cluster is
|
|
behind a firewall and containers cannot access URL's on the external
|
|
internet directly. For the parameter http-proxy and https-proxy, the
|
|
value to provide is a URL and it will be set in the environment
|
|
variable HTTP_PROXY and HTTPS_PROXY respectively in the nodes. For
|
|
the parameter no-proxy, the value to provide is an IP or list of IP's
|
|
separated by comma. Likewise, the value will be set in the
|
|
environment variable NO_PROXY in the nodes.
|
|
|
|
For the networking model to the container, the following configuration
|
|
can be set in the baymodel:
|
|
|
|
network-driver
|
|
The network driver name for instantiating container networks.
|
|
Currently, the following network drivers are supported:
|
|
|
|
+--------+-------------+-----------+-------------+
|
|
| Driver | Kubernetes | Swarm | Mesos |
|
|
+========+=============+===========+=============+
|
|
| Flannel| supported | supported | unsupported |
|
|
+--------+-------------+-----------+-------------+
|
|
| Docker | unsupported | supported | supported |
|
|
+--------+-------------+-----------+-------------+
|
|
|
|
If not specified, the default driver is Flannel for Kubernetes, and
|
|
Docker for Swarm and Mesos.
|
|
|
|
Particular network driver may require its own set of parameters for
|
|
configuration, and these parameters are specified through the labels
|
|
in the baymodel. Labels are arbitrary key=value pairs.
|
|
|
|
When Flannel is specified as the network driver, the following
|
|
optional labels can be added:
|
|
|
|
flannel_network_cidr
|
|
IPv4 network in CIDR format to use for the entire Flannel network.
|
|
If not specified, the default is 10.100.0.0/16.
|
|
|
|
flannel_network_subnetlen
|
|
The size of the subnet allocated to each host. If not specified, the
|
|
default is 24.
|
|
|
|
flannel_backend
|
|
The type of backend for Flannel. Possible values are *udp, vxlan,
|
|
host-gw*. If not specified, the default is *udp*. Selecting the
|
|
best backend depends on your networking. Generally, *udp* is
|
|
the most generally supported backend since there is little
|
|
requirement on the network, but it typically offers the lowest
|
|
performance. The *vxlan* backend performs better, but requires
|
|
vxlan support in the kernel so the image used to provision the
|
|
nodes needs to include this support. The *host-gw* backend offers
|
|
the best performance since it does not actually encapsulate
|
|
messages, but it requires all the nodes to be on the same L2
|
|
network. The private Neutron network that Magnum creates does
|
|
meet this requirement; therefore if the parameter *fixed_network*
|
|
is not specified in the baymodel, *host-gw* is the best choice for
|
|
the Flannel backend.
|
|
|
|
|
|
=================
|
|
High Availability
|
|
=================
|
|
*To be filled in*
|
|
|
|
=======
|
|
Scaling
|
|
=======
|
|
*To be filled in*
|
|
|
|
Include Autoscaling
|
|
|
|
=======
|
|
Storage
|
|
=======
|
|
*To be filled in*
|
|
|
|
================
|
|
Image Management
|
|
================
|
|
|
|
When a COE is deployed, an image from Glance is used to boot the nodes
|
|
in the cluster and then the software will be configured and started on
|
|
the nodes to bring up the full cluster. An image is based on a
|
|
particular distro such as Fedora, Ubuntu, etc, and is prebuilt with
|
|
the software specific to the COE such as Kubernetes, Swarm, Mesos.
|
|
The image is tightly coupled with the following in Magnum:
|
|
|
|
1. Heat templates to orchestrate the configuration.
|
|
|
|
2. Template definition to map baymodel parameters to Heat
|
|
template parameters.
|
|
|
|
3. Set of scripts to configure software.
|
|
|
|
Collectively, they constitute the driver for a particular COE and a
|
|
particular distro; therefore, developing a new image needs to be done
|
|
in conjunction with developing these other components. Image can be
|
|
built by various methods such as diskimagebuilder, or in some case, a
|
|
distro image can be used directly. A number of drivers and the
|
|
associated images is supported in Magnum as reference implementation.
|
|
In this section, we focus mainly on the supported images.
|
|
|
|
All images must include support for cloud-init and the heat software
|
|
configuration utility:
|
|
|
|
- os-collect-config
|
|
- os-refresh-config
|
|
- os-apply-config
|
|
- heat-config
|
|
- heat-config-script
|
|
|
|
Additional software are described as follows.
|
|
|
|
Kubernetes on Fedora Atomic
|
|
---------------------------
|
|
|
|
This image is built manually by the instructions provided in this `Atomic guide
|
|
<https://github.com/openstack/magnum/blob/master/doc/source/dev/build-atomic-image.rst>`_
|
|
The Fedora site hosts the current image `fedora-21-atomic-5.qcow2
|
|
<https://fedorapeople.org/groups/magnum/fedora-21-atomic-5.qcow2>`_.
|
|
This image has the following OS/software:
|
|
|
|
+-------------+-----------+
|
|
| OS/software | version |
|
|
+=============+===========+
|
|
| Fedora | 21 |
|
|
+-------------+-----------+
|
|
| Docker | 1.8.1 |
|
|
+-------------+-----------+
|
|
| Kubernetes | 1.0.4 |
|
|
+-------------+-----------+
|
|
| etcd | 2.0.10 |
|
|
+-------------+-----------+
|
|
| Flannel | 0.5.0 |
|
|
+-------------+-----------+
|
|
|
|
The following software are managed as systemd services:
|
|
|
|
- kube-apiserver
|
|
- kubelet
|
|
- etcd
|
|
- flannel (if specified as network driver)
|
|
- docker
|
|
|
|
The following software are managed as Docker containers:
|
|
|
|
- kube-controller-manager
|
|
- kube-scheduler
|
|
- kube-proxy
|
|
|
|
The login for this image is *minion*.
|
|
|
|
Kubernetes on CoreOS
|
|
--------------------
|
|
|
|
CoreOS publishes a `stock image
|
|
<http://beta.release.core-os.net/amd64-usr/current/coreos_production_openstack_image.img.bz2>`_
|
|
that is being used to deploy Kubernetes.
|
|
This image has the following OS/software:
|
|
|
|
+-------------+-----------+
|
|
| OS/software | version |
|
|
+=============+===========+
|
|
| CoreOS | 4.3.6 |
|
|
+-------------+-----------+
|
|
| Docker | 1.9.1 |
|
|
+-------------+-----------+
|
|
| Kubernetes | 1.0.6 |
|
|
+-------------+-----------+
|
|
| etcd | 2.2.3 |
|
|
+-------------+-----------+
|
|
| Flannel | 0.5.5 |
|
|
+-------------+-----------+
|
|
|
|
The following software are managed as systemd services:
|
|
|
|
- kubelet
|
|
- flannel (if specified as network driver)
|
|
- docker
|
|
- etcd
|
|
|
|
The following software are managed as Docker containers:
|
|
|
|
- kube-apiserver
|
|
- kube-controller-manager
|
|
- kube-scheduler
|
|
- kube-proxy
|
|
|
|
The login for this image is *core*.
|
|
|
|
Kubernetes on Ironic
|
|
--------------------
|
|
|
|
This image is built manually using diskimagebuilder. The scripts and instructions
|
|
are included in `Magnum code repo
|
|
<https://github.com/openstack/magnum/tree/master/magnum/templates/kubernetes/elements>`_.
|
|
Currently Ironic is not fully supported yet, therefore more details will be
|
|
provided when this driver has been fully tested.
|
|
|
|
|
|
Swarm on Fedora Atomic
|
|
----------------------
|
|
|
|
This image is the same as the image for Kubernetes on Fedora Atomic and was
|
|
built manually by the instructions provided in this `Atomic guide
|
|
<https://github.com/openstack/magnum/blob/master/doc/source/dev/build-atomic-image.rst>`_
|
|
The Fedora site hosts the current image `fedora-21-atomic-5.qcow2
|
|
<https://fedorapeople.org/groups/magnum/fedora-21-atomic-5.qcow2>`_.
|
|
This image has the following OS/software:
|
|
|
|
+-------------+-----------+
|
|
| OS/software | version |
|
|
+=============+===========+
|
|
| Fedora | 21 |
|
|
+-------------+-----------+
|
|
| Docker | 1.8.1 |
|
|
+-------------+-----------+
|
|
| Kubernetes | 1.0.4 |
|
|
+-------------+-----------+
|
|
| etcd | 2.0.10 |
|
|
+-------------+-----------+
|
|
| Flannel | 0.5.0 |
|
|
+-------------+-----------+
|
|
|
|
The login for this image is *fedora*.
|
|
|
|
Mesos on Ubuntu
|
|
---------------
|
|
|
|
This image is built manually using diskimagebuilder. The instructions are
|
|
provided in this `Mesos guide
|
|
<https://github.com/openstack/magnum/blob/master/doc/source/dev/mesos.rst>`_.
|
|
The Fedora site hosts the current image `ubuntu-14.04.3-mesos-0.25.0.qcow2
|
|
<https://fedorapeople.org/groups/magnum/ubuntu-14.04.3-mesos-0.25.0.qcow2>`_.
|
|
|
|
+-------------+-----------+
|
|
| OS/software | version |
|
|
+=============+===========+
|
|
| Ubuntu | 14.04 |
|
|
+-------------+-----------+
|
|
| Docker | 1.8.1 |
|
|
+-------------+-----------+
|
|
| Mesos | 0.25.0 |
|
|
+-------------+-----------+
|
|
| Marathon | |
|
|
+-------------+-----------+
|
|
|