openstack/fuel-docs
Code Issues Proposed changes

break up reference-arch source into modular files

Browse Source 
Change-Id: I5b4458ec3e90f0974a7e070cdc9739727e7b8cc9
This commit is contained in:
Meg McRoberts
2014-03-27 14:18:18 -07:00
committed by Dmitry Borodaenko
parent db46e21a22
commit 1f6453a72c
34 changed files with 846 additions and 825 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
contents/contents-refarch.rst
View File

@@ -15,6 +15,6 @@
.. include:: /pages/reference-architecture/0040-network-setup.rst
.. include:: /pages/reference-architecture/0045-storage.rst
.. include:: /pages/reference-architecture/0050-technical-considerations-overview.rst
.. include:: /pages/reference-architecture/0060-quantum-vs-nova-network.rst
.. include:: /pages/reference-architecture/0060-neutron-vs-nova-network.rst
.. include:: /pages/reference-architecture/0090-savanna.rst
.. include:: /pages/reference-architecture/0100-bonding.rst

117
pages/reference-architecture/0020-logical-setup.rst
View File

@@ -1,114 +1,5 @@
.. raw:: pdf
PageBreak
.. index:: Reference Architectures: HA Logical Setup, HA Logical Setup
HA Logical Setup
================
.. contents :local:
An OpenStack Multi-node HA environment involves three types of nodes:
controller nodes, compute nodes, and storage nodes.
Controller Nodes
----------------
The first order of business in achieving high availability (HA) is
redundancy, so the first step is to provide multiple controller nodes.
As you may recall, the database uses Galera to achieve HA, and Galera is
a quorum-based system. That means that you should have at least 3
controller nodes.
.. image:: /_images/logical-diagram-controller.*
:width: 80%
:align: center
Every OpenStack controller runs HAProxy, which manages a single External
Virtual IP (VIP) for all controller nodes and provides HTTP and TCP load
balancing of requests going to OpenStack API services, RabbitMQ, and MySQL.
When an end user accesses the OpenStack cloud using Horizon or makes a
request to the REST API for services such as nova-api, glance-api,
keystone-api, neutron-api, nova-scheduler, MySQL or RabbitMQ, the
request goes to the live controller node currently holding the External VIP,
and the connection gets terminated by HAProxy. When the next request
comes in, HAProxy handles it, and may send it to the original
controller or another in the environment, depending on load conditions.
Each of the services housed on the controller nodes has its own
mechanism for achieving HA:
* nova-api, glance-api, keystone-api, neutron-api and nova-scheduler are
stateless services that do not require any special attention besides load
balancing.
* Horizon, as a typical web application, requires sticky sessions to be enabled
at the load balancer.
* RabbitMQ provides active/active high availability using mirrored queues.
* MySQL high availability is achieved through Galera active/active multi-master
deployment and Pacemaker.
* Neutron agents are managed by Pacemaker.
* Ceph monitors implement their own quorum based HA mechanism and
require time synchronization between all nodes. Clock drift higher
than 50ms may break the quorum or even crash the Ceph service.
Compute Nodes
-------------
OpenStack compute nodes are, in many ways, the foundation of your
environment; they are the servers on which your users will create their
Virtual Machines (VMs) and host their applications. Compute nodes need
to talk to controller nodes and reach out to essential services such
as RabbitMQ and MySQL. They use the same approach that provides
redundancy to the end-users of Horizon and REST APIs, reaching out to
controller nodes using the VIP and going through HAProxy.
.. image:: /_images/logical-diagram-compute.*
:width: 40%
:align: center
Storage Nodes
-------------
Depending on the :ref:`storage options <Storage_Architecture>` you
select for your environment, you may have Ceph, Cinder, and Swift
services running on your storage nodes.
Ceph_ implements its own HA, all you need is enough controller nodes
running Ceph Monitor service to `form a quorum
<http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-mon/>`_,
and enough Ceph OSD nodes to satisfy the `object replication factor
<http://ceph.com/docs/master/rados/operations/pools/>`_.
.. _Ceph: http://ceph.com/docs/master/architecture/
.. image:: /_images/ceph_nodes.*
:width: 80%
:align: center
Swift API relies on the same HAProxy setup with VIP on controller nodes
as the other REST APIs. If don't expect too much data traffic in Swift,
you can also deploy Swift Storage and Proxy services on controller
nodes. For a larger production environment you'll need dedicated nodes:
two for Swift Proxy and at least three for Swift Storage.
Whether or not you'd want separate Swift nodes depends primarily on how
much data you expect to keep there. A simple test is to fully populate
your Swift object store with data and then fail one controller node. If
replication of the degraded Swift objects between the remaining nodes
controller generates enough network traffic, CPU load, or disk I/O to
impact performance of other OpenStack services running on the same
nodes, you should separate Swift from controllers.
.. image:: /_images/logical-diagram-storage.*
:width: 40%
:align: center
If you select Cinder LVM as the block storage backend for Cinder
volumes, you should have at least one Cinder LVM node. Unlike Swift and
Ceph, Cinder LVM doesn't implement data redundancy across nodes: if a
Cinder node is lost, volumes stored on that node cannot be recovered
from the data stored on other Cinder nodes. If you need your block
storage to be resilient, use Ceph for volumes.
.. include:: /pages/reference-architecture/logical-setup/0100-intro-logical-setup.rst
.. include:: /pages/reference-architecture/logical-setup/0200-controller-nodes.rst
.. include:: /pages/reference-architecture/logical-setup/0300-compute-nodes.rst
.. include:: /pages/reference-architecture/logical-setup/0400-storage-nodes.rst

103
pages/reference-architecture/0040-network-setup.rst
View File

@@ -1,98 +1,5 @@
.. raw:: pdf
PageBreak
.. index:: Network Architecture
Network Architecture
====================
.. contents :local:
For better network performance and manageability, Fuel place different types
of traffic into separate networks. This section describes how to distribute
the network traffic in an OpenStack cluster.
.. index:: Admin (PXE) Network
This network is reserved for communcation with Fuel master for provisioning
and orchestration of OpenStack deployment. It is used during installation to
provide DNS, DHCP, and gateway services to a node if it requires it before
provisioning. Nodes retrieve their network configuration via DHCP from Fuel
Master node. This is why it is important that this network is isolated from
your network and has no DHCP server (except Fuel Master) running on it.
.. index:: Public Network
Public Network
--------------
This network allows inbound connections to VMs from the outside world (allowing
users to connect to VMs from the Internet). It also allows outbound connections
from VMs to the outside world. For security reasons, the public network is usually
isolated from other networks in cluster. The word "Public" means that these addresses
can be used to communicate with cluster and its VMs from outside of cluster.
To enable external access to VMs, the public network provides the address space
for the floating IPs assigned to individual VM instances by the project
administrator. Nova Network or Neutron services can then
configure this address on the public network interface of the Network controller
node. E.g. environments based on Nova Network use iptables to create a
Destination NAT from this address to the private IP of the corresponding VM
instance through the appropriate virtual bridge interface on the Network
controller node.
In the other direction, the public network provides connectivity to the globally
routed address space for VMs. The IP address from the public network that has
been assigned to a compute node is used as the source for the Source NAT
performed for traffic going from VM instances on the compute node to Internet.
The public network also provides Virtual IPs for Endpoint nodes, which are used to
connect to OpenStack services APIs.
.. index:: Internal Network, Management Network
Internal (Management) Network
-----------------------------
The internal network connects all OpenStack nodes in the environment. All
components of an OpenStack environment communicate with each other using this
network. This network must be isolated from both the private and public
networks for security reasons.
The internal network can also be used for serving iSCSI protocol exchanges
between Compute and Storage nodes.
.. index:: Private Network
Private Network
---------------
The private network facilitates communication between each tenant's VMs. Private
network address spaces are not a part of the enterprise network address space. Fixed
IPs of virtual instances are directly unaccessible from the rest of Enterprise network.
NIC usage
---------
The current architecture assumes the presence of 3 NICs, but it can be
customized for two or 4+ network interfaces. Most servers are built with at least
two network interfaces. In this case, let's consider a typical example of three
NIC cards. They're utilized as follows:
**eth0**:
The Admin (PXE) network, used for communication with Fuel Master for
deployment.
**eth1**:
The public network and floating IPs assigned to VMs
**eth2**:
The private network, for communication between OpenStack VMs, and the
bridge interface (VLANs)
The figure below illustrates the relevant nodes and networks in Neutron VLAN mode.
.. image:: /_images/080-networking-diagram.*
:width: 75%
:align: center
.. include:: /pages/reference-architecture/network-setup/0100-network-intro.rst
.. include:: /pages/reference-architecture/network-setup/0200-public-net.rst
.. include:: /pages/reference-architecture/network-setup/0300-internal-mgmt-net.rst
.. include:: /pages/reference-architecture/network-setup/0400-private-network.rst
.. include:: /pages/reference-architecture/network-setup/0500-nic-usage.rst

129
pages/reference-architecture/0045-storage.rst
View File

@@ -1,125 +1,4 @@
.. index:: Storage architecture, Object storage, Cinder, Swift, Glance, Ceph
.. _Storage_Architecture:
Storage Architecture
====================
Object Storage for Images
-------------------------
.. _Object_Storage_for_Images:
Fuel can configure one of these storage backends for the Glance image
service:
* File system backend,
* `Swift object store <http://swift.openstack.org/>`_, the standard
OpenStack object storage component,
* `Ceph RBD <http://ceph.com/docs/master/rbd/rbd-openstack/>`_, the
distributed block device service based on RADOS, the object store
component of the Ceph storage platform.
By default, Glance stores virtual machine images using the **file
system** backend. With it, you can use any of the shared file systems
supported by Glance. Fuel will default to this option in a simple non-HA
deployment with a local file system on a single controller node.
For a production HA environment where you want data resilience for VM
images, the best practice is to use an object store as the backend for
Glance. That way, multiple Glance instances running on controller nodes
can store and retrieve images from the same data set, while the object
store takes care of data protection and HA.
In HA deployment mode, Fuel will default to using **Swift** as the
storage backend for Glance instead of the file system backend. In both
HA and non-HA multi-node configurations, Fuel will also offer you the
option to use Ceph as storage backend.
The primary advantage of using **Ceph RBD** for images is the ability to
unify different classes of data into a single shared pool of storage
nodes that can handle all classes of data important for OpenStack.
Instead of having to copy OS images and volumes between separate Glance,
Cinder, and Nova storage pools, you can have all three services use
copy-on-write clones of the original image. In addition to better
utilization of storage capacity, copy-on-write also significantly speeds
up launching VMs from images.
To make the most out of the copy-on-write capability of the Ceph
backend, you should only use images in **raw** format. Images in other
formats such as qcow2 have to be converted to raw format first and
cannot be cloned without conversion.
As of this writing, vanilla OpenStack Havana release placed several
important limitations on the copy-on-write capability of the Ceph
backend:
* You have to create an RBD backed bootable volume from a raw image for
copy-on-write and live migrations to work;
* Launching an instance directly from image results in a full copy on
the compute node instead of a copy-on-write clone in Ceph;
* Ephemeral drives are stored in local files on compute nodes instead
of Ceph, preventing live migration of instances with ephemenral
drives;
* Non-raw images are not converted to raw automatically when creating a
bootable volume: if you don't convert the image yourself, you will
end up with a clone of a qcow2 image that will not be bootable.
These limitations are removed in Mirantis OpenStack distribution
starting with release 4.0.
Object Storage for Applications
-------------------------------
The objects storage systems supported by Mirantis OpenStack don't have
to be limited to serving Glance. Swift provides a REST API that can be
used by any application that needs to store data in an object store, and
is immediately available whenever you have Swift running.
Ceph includes the optional radosgw_ object gateway component which
allows to access objects in RADOS object store using REST interfaces
compatible with Amazon S3 and Swift APIs.
.. _radosgw: http://ceph.com/docs/master/radosgw/
Ceph RBD backend for Glance doesn't use Swift API and uses RADOS
directly, so it is possible to store Glance images in Ceph and still use
Swift as object store for applications. This does not work the other way
around: when you choose to install the Ceph object gateway, it replaces
Swift as the provider of Swift API, so you can't have both radosgw and
Swift in the same OpenStack environment.
Block Storage for Volumes
-------------------------
When you configure Mirantis OpenStack to use the default **LVM** backend
for Cinder block devices, Cinder will store each volume as a logical
volume in an LVM volume group on one of your Cinder nodes.
If you don't need your volumes to be resilient, you can let Fuel create
a JBOD partition spanning all your storage drives in a node during
deployment, or you can join all drives into a RAID array before
deployment, and have the array appear to Fuel as a single block device.
Even if you use RAID in each Cinder node, that only protects your data
from a hard drive failure. If the whole Cinder node is lost, so are all
volumes that were stored on that node.
Setting Cinder backend to **Ceph RBD** allows to take advantage of
Ceph's object replication capabilities by storing Cinder volumes as
objects in Ceph RADOS object storage system. By default, Ceph simply
ensures that every replica of an object is stored on a different node.
The set of data replication rules (CRUSH map) can be customized
separately for each object pool to change the number of object replicas,
add different types of failure domains, etc. The amount of storage
required to host your volumes will be multiplied by the replication
factor you configure for Ceph, but your volumes will be protected from
node or even data center failure.
If you combine RBD backends for Cinder and Glance, you gain another
important advantage over Cinder LVM: copy-on-write cloning of Glance
images into bootable Ceph volumes.
.. include:: /pages/reference-architecture/storage/0000-storage-intro.rst
.. include:: /pages/reference-architecture/storage/0100-object-store-for-images.rst
.. include:: /pages/reference-architecture/storage/0200-object-storage-for-apps.rst
.. include:: /pages/reference-architecture/storage/0300-block-storage-for-volumes.rst

3
pages/reference-architecture/0050-technical-considerations-DISPLAYSTUB
View File

@@ -1,3 +0,0 @@
.. include:: /pages/reference-architecture/0050-technical-considerations-overview.rst
.. include:: /pages/reference-architecture/0060-quantum-vs-nova-network.rst

5
pages/reference-architecture/0060-neutron-vs-nova-network.rst Normal file
View File

@@ -0,0 +1,5 @@
.. include:: /pages/reference-architecture/neutron-intro/0100-neutron-vs-nova-network-intro.rst
.. include:: /pages/reference-architecture/neutron-intro/0150-mos-networks-list.rst
.. include:: /pages/reference-architecture/neutron-intro/0200-neutron-tech-intro.rst
.. include:: /pages/reference-architecture/neutron-intro/0300-neutron-limits.rst
.. include:: /pages/reference-architecture/neutron-intro/0600-virtualbox-promiscuous-mode.rst

153
pages/reference-architecture/0060-quantum-vs-nova-network.rst
View File

@@ -1,153 +0,0 @@
.. index:: Neutron vs. nova-network, Quantum vs. nova-network
Neutron vs. nova-network
------------------------
Neutron (formerly Quantum) is a service which provides Networking-as-a-Service
functionality in OpenStack. It has a rich tenant-facing API for defining
network connectivity and addressing in the cloud, and gives operators the
ability to leverage different networking technologies to power their cloud
networking.
There are various deployment use cases for Neutron. Fuel supports the most
common of them, called Per-tenant Routers with Private Networks.
Each tenant has a virtual Neutron router with one or more private networks,
which can communicate with the outside world.
This allows full routing isolation for each tenant private network.
Neutron is not, however, required in order to run an OpenStack environment. If
you don't need (or want) this added functionality, it's perfectly acceptable to
continue using nova-network.
In order to deploy Neutron, you need to enable it in the Fuel configuration.
Fuel sets up Neutron components on each of the controllers to act as a router
in HA (if deploying in HA mode).
Terminology
~~~~~~~~~~~
* **Public network** (also known as External network) used for Internet
access for all nodes.
* **Floating IP network** subnet within public network allocated for tenant
Internet access. A Neutron server directly assigns IP addresses for this network.
If a user delegates a floating IP address to an instance, an IP address will
be assigned from this subnet (by SNAT/DNAT).
* **Private network** used for passing tenant private traffic.
* **Admin network** shared network between Fuel Master and all nodes in the
cluster for PXE installation and orchestration of environment for deployment.
* **Storage network** network used for communicating between storage nodes
(using Ceph, swift, or cinder) and compute nodes.
* **Management network** (also known as Internal) used
for necessary communication between controllers and computes for AMQP
messaging, DB queries, other inter-controller traffic required for
supporting services.
* **Router** virtual Neutron router.
* **NIC** network interface card (ethernet adapter).
Overview
~~~~~~~~
OpenStack networking with Neutron (Quantum) has some differences from
Nova-network. Neutron is able to virtualize and manage both layer 2 (logical)
and layer 3 (network) of the OSI network model, as compared to simple layer 3
virtualization provided by nova-network. This is the main difference between
the two networking models for OpenStack. Virtual networks (one or more) can be
created for a single tenant, forming an isolated L2 network called a
"private network". Each private network can support one or many IP subnets.
Private networks can be segmented using two different technologies:
* **VLAN segmentation** "Private network" traffic is managed by
Neutron by the use of a dedicated network adapter. This network adapter must be
attached to a untagged network port. This network segment also must be
reserved only for Neutron on each host (Computes and Controllers). You should
not use any other 802.1q VLANs on this network for other purposes.
Additionally, each private network requires its own dedicated VLAN, selected
from a given range configured in Fuel UI.
* **GRE segmentation** In this mode of operation, Neutron does not
require a dedicated network adapter. Neutron builds a mesh of GRE tunnels from
each compute node and controller nodes to every other node. Private networks
for each tenant make use of this mesh for isolated traffic.
It is important to note:
* if you use tagged networks for your configuration
and combine multiple networks onto one NIC, you should make the Public
network untagged on this NIC.
It is not a requirement, but best for access to the OpenStack Dashboard
and public OpenStack API.
* Is a best if you place the Private, Admin, Public and Management networks on a
separate NIC to ensure that traffic is separated adequately.
* Admin and Private networks must be located together on separate NIC from the
other networks.
A typical network configuration for Neutron with VLAN segmentation might look
like this:
.. image:: /_images/Neutron_32_vlan_v2.png
:align: center
A typical network configuration for Neutron with GRE segmentation might look
like this:
.. image:: /_images/Neutron_32_gre_v2.png
:align: center
The most likely configuration for different number NICs on cluster nodes:
+------+--------------------------------------+--------------------------------------+
| NICs | VLAN | GRE |
+======+======================================+======================================+
| 2 | Not supported | .. image:: /_images/q32_gre_2nic.* |
| | | :align: center |
+------+--------------------------------------+--------------------------------------+
| 3 | .. image:: /_images/q32_vlan_3nic.* | .. image:: /_images/q32_gre_3nic.* |
| | :align: center | :align: center |
+------+--------------------------------------+--------------------------------------+
| 4 | .. image:: /_images/q32_vlan_4nic.* | .. image:: /_images/q32_gre_4nic.* |
| | :align: center | :align: center |
+------+--------------------------------------+--------------------------------------+
Known limitations
~~~~~~~~~~~~~~~~~
* Neutron will not allocate a floating IP range for your tenants. After each
tenant is created, a floating IP range must be created. Note that this does
not prevent Internet connectivity for a tenant's instances, but it would
prevent them from receiving incoming connections. You, the administrator,
should assign a floating IP addresses for the tenant. Below are steps you can
follow to do this:
| get admin credentials:
| # source /root/openrc
| get admin tenant-ID:
| # keystone tenant-list
+----------------------------------+----------+---------+
| id | name | enabled |
+==================================+==========+=========+
| b796f91df6b84860a7cd474148fb2229 | admin | True |
+----------------------------------+----------+---------+
| cba7b0ff68ee4985816ac3585c8e23a9 | services | True |
+----------------------------------+----------+---------+
| create one floating-ip address for admin tenant:
| # neutron floatingip-create --tenant-id=b796f91df6b84860a7cd474148fb2229 net04_ext
* You can't combine Private or Admin network with any other networks on one NIC.
* To deploy OpenStack using Neutron with GRE segmentation, each node requires at
least 2 NICs.
* To deploy OpenStack using Neutron with VLAN segmentation, each node requires
at least 3 NICs.
FAQ
~~~
| Q: I tried to deploy a Fuel OpenStack environment on VirtualBox, but the
deployment fails on Neutron setup. How do I fix this?
| A: You should to choose ”Allow all” promiscuous mode on all network
interfaces in VirtualBox and modify the network cards to use the PCnet
PCI II model network card.

338
pages/reference-architecture/0100-bonding.rst
View File

@@ -1,331 +1,7 @@
.. raw:: pdf
PageBreak
.. index:: Advanced Network Configuration using OVS
Advanced Network Configuration using Open VSwitch
=================================================
The Neutron networking model uses Open VSwitch (OVS) bridges and the Linux
namespaces to create a flexible network setup and to isolate
tenants from each other on L2 and L3 layers. Mirantis OpenStack also
provides a flexible network setup model based on Open VSwitch primitives,
which you can use to customize your nodes. Its most popular feature is
link aggregation. While the FuelWeb UI uses a hardcoded
per-node network model, the Fuel CLI tool allows you to modify it in your own way.
Reference Network Model in Neutron
----------------------------------
The FuelWeb UI uses the following per-node network model:
* Create an OVS bridge for each NIC except for the NIC with Admin network
(for example, **br-eth0** bridge for **eth0** NIC) and put NICs into their bridges
* Create a separate bridge for each OpenStack network:
* **br-ex** for the Public network
* **br-prv** for the Private network
* **br-mgmt** for the Management network
* **br-storage** for the Storage network
* Connect each network's bridge with an apropriate NIC bridge using an OVS patch with
an apropriate VLAN tag.
* Assign network IP addresses to the corresponding bridges.
Note that the Admin network IP address is assigned to its NIC directly.
This network model allows the cluster administrator to manipulate cluster
network entities and NICs separately, easily, and on the fly during the cluster
life-cycle.
Adjust the Network Configuration via CLI
----------------------------------------
On a basic level, this network configuration is part of a data structure that provides
instructions to the Puppet modules to set up a network on the current node.
You can examine and modify this data using the Fuel CLI tool. Just download (then
modify and upload if needed) the environment's 'deployment default' configuration:
::
[root@fuel ~]# fuel --env 1 deployment default
directory /root/deployment_1 was created
Created /root/deployment_1/compute_1.yaml
Created /root/deployment_1/controller_2.yaml
[root@fuel ~]# vi ./deployment_1/compute_1.yaml
[root@fuel ~]# fuel --env 1 deployment --upload
.. note::
Please, make sure you read :ref:`the Fuel CLI documentation <cli_usage>` carefully.
The part of this data structure that describes how to apply the network configuration
is the 'network_scheme' key in the top-level hash of the YAML file. Let's take a
closer look at this substructure. The value of the 'network_scheme' key is a hash with
the following keys:
* **interfaces** - A hash of NICs and their low-level/physical parameters.
You can set an MTU and the 'VLAN splinters' feature here.
* **provider** - Set to 'ovs' for Neutron.
* **endpoints** - A hash of network ports (OVS ports or NICs) and their IP
settings.
* **roles** - A hash that specifies the mappings between the endpoints and
internally-used roles in Puppet manifests ('management', 'storage', and so on).
* **transformations** - An ordered list of OVS network primitives.
The "Transformations" Section
-----------------------------
You can use four OVS primitives:
* **add-br** - To add an OVS bridge to the system
* **add-port** - To add a port to an existent OVS bridge
* **add-bond** - To create a port in OVS bridge and add aggregated NICs to it
* **add-patch** - To create an OVS patch between two existing OVS bridges
The primitives will be applied in the order they are listed.
Here are the the available options:
::
{
"action": "add-br", # type of primitive
"name": "xxx" # unique name of the new bridge
},
{
"action": "add-port", # type of primitive
"name": "xxx-port", # unique name of the new port
"bridge": "xxx", # name of the bridge where the port should be created
"type": "internal", # [optional; default: "internal"] a type of OVS interface
# for the port (see OVS documentation);
# possible values: "system", "internal", "tap", "gre", "null"
"tag": 0, # [optional; default: 0] a 802.1q tag of traffic that
# should be captured from an OVS bridge;
# possible values: 0 (means port is a trunk),
# 1-4094 (means port is an access)
"trunks": [], # [optional; default: []] a set of 802.1q tags
# (integers from 0 to 4095) that are allowed to
# pass through if "tag" option equals 0;
# possible values: an empty list (all traffic passes),
# 0 (untagged traffic only), 1 (strange behaviour;
# shouldn't be used), 2-4095 (traffic with this
# tag passes); e.g. [0,10,20]
"port_properties": [], # [optional; default: []] a list of additional
# OVS port properties to modify them in OVS DB
"interface_properties": [], # [optional; default: []] a list of additional
# OVS interface properties to modify them in OVS DB
"vlan_splinters": false, # [optional; default: false] enable 'vlan splinters'
# feature for this interface
},
{
"action": "add-bond", # type of primitive
"name": "xxx-port", # unique name of the new bond
"interfaces": [], # a set of two or more bonded interfaces' names;
# e.g. ['eth1','eth2']
"bridge": "xxx", # name of the bridge where the bond should be created
"tag": 0, # [optional; default: 0] a 802.1q tag of traffic which
# should be catched from an OVS bridge;
# possible values: 0 (means port is a trunk),
# 1-4094 (means port is an access)
"trunks": [], # [optional; default: []] a set of 802.1q tags
# (integers from 0 to 4095) which are allowed to
# pass through if "tag" option equals 0;
# possible values: an empty list (all traffic passes),
# 0 (untagged traffic only), 1 (strange behaviour;
# shouldn't be used), 2-4095 (traffic with this
# tag passes); e.g. [0,10,20]
"properties": [], # [optional; default: []] a list of additional
# OVS bonded port properties to modify them in OVS DB;
# you can use it to set the aggregation mode and balancing
# strategy, to configure LACP, and so on (see the OVS documentation)
},
{
"action": "add-patch", # type of primitive
"bridges": ["br0", "br1"], # a pair of different bridges that will be connected
"peers": ["p1", "p2"], # [optional] abstract names for each end of the patch
"tags": [0, 0] , # [optional; default: [0,0]] a pair of integers that
# represent an 802.1q tag of traffic that is
# captured from an appropriate OVS bridge; possible
# values: 0 (means port is a trunk), 1-4094 (means
# port is an access)
"trunks": [], # [optional; default: []] a set of 802.1q tags
# (integers from 0 to 4095) which are allowed to
# pass through each bridge if "tag" option equals 0;
# possible values: an empty list (all traffic passes),
# 0 (untagged traffic only), 1 (strange behavior;
# shouldn't be used), 2-4095 (traffic with this
# tag passes); e.g., [0,10,20]
}
A combination of these primitives allows you to make custom and complex
network configurations.
NICs Aggregation
----------------
The NIC bonding allows you to aggregate multiple physical links to one link
to increase speed and provide fault tolerance.
**Documentation**
The Linux kernel documentation about bonding can be found in Linux Ethernet Bonding Driver HOWTO:
https://www.kernel.org/doc/Documentation/networking/bonding.txt.
You can find shorter introduction to bonding and tips on link monitoring here:
http://wiki.mikrotik.com/wiki/Manual:Interface/Bonding
Cisco switches configuration guide:
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/layer2/503_U2_1/b_Cisco_n3k_layer2_config_guide_503_U2_1/b_Cisco_n3k_layer2_config_gd_503_U2_1_chapter_01000.html
Switches configuration tips for Fuel:
https://etherpad.openstack.org/p/LACP_FUEL_bonding
**Types of Bonding**
Open VSwitch supports the same bonding features as the Linux kernel.
Fuel supports bonding via Open VSwitch only.
Linux supports two types of bonding:
* IEEE 802.1AX (formerly known as 802.3ad) Link Aggregation Control Protocol (LACP). Devices on both sides of links must communicate using LACP to set up an aggregated link. So both devices must support LACP, enable and configure it on these links.
* One side bonding does not require any special feature support from the switch side. Linux handles it using a set of traffic balancing algorithms.
One Side Bonding Policies:
* Balance-rr - Round-robin policy. This mode provides load balancing and fault tolerance.
* Active-backup - Active-backup policy: Only one slave in the bond is active.This mode provides fault tolerance.
* Balance-xor - XOR policy: Transmit based on the selected transmit hash policy. This mode provides load balancing and fault tolerance.
* Broadcast - Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.
* balance-tlb - Adaptive transmit load balancing based on a current links' utilisation. This mode provides load balancing and fault tolerance.
* balance-alb - Adaptive transmit and receive load balancing based on the current links' utilisation. This mode provides load balancing and fault tolerance.
* balance-slb - Modification of balance-alb mode. SLB bonding allows a limited form of load balancing without the
remote switch's knowledge or cooperation. SLB assigns each source MAC+VLAN pair to a link and transmits all packets from
that MAC+VLAN through that link. Learning in the remote switch causes it to send packets to that MAC+VLAN through the same link.
* balance-tcp - Adaptive transmit load balancing among interfaces.
LACP Policies:
* Layer2 - Uses XOR of hardware MAC addresses to generate the hash.
* Layer2+3 - uses a combination of layer2 and layer3 protocol information to generate the hash.
* Layer3+4 - uses upper layer protocol information, when available, to generate the hash.
* Encap2+3 - uses the same formula as layer2+3 but it relies on skb_flow_dissect to obtain the header fields which might result in the use of inner headers if an encapsulation protocol is used. For example this will improve the performance for tunnel users because the packets will be distributed according to the encapsulated flows.
* Encap3+4 - Similar to Encap2+3 but uses layer3+4.
**Policies Supported by Fuel**
Fuel supports the following policies: Active-backup, balance-slb, LACP balance-tcp.
**Network Verification in Fuel**
Fuel has limited network verification capabilities when working with bonds.
Network connectivity can be checked for the new cluster only (not for deployed one)
so check is done when nodes are in bootstrap and no bonds are up. Connectivity
between slave interfaces can be checked but not bonds themselves.
An Example of NIC Aggregation
-----------------------------
Suppose you have a node with 4 NICs and you want to bond two of them with LACP
enabled ("eth2" and "eth3" here) and then assign Private and Storage networks
to them. The Admin network uses a dedicated NIC ("eth0"). The Management and
Public networks use the last NIC ("eth1"). To achieve this goal, do the following:
* Create a separate OVS bridge "br-bond0" instead of "br-eth2" and "br-eth3".
* Connect "eth2" and "eth3" to "br-bond0" as a bonded port with property
"lacp=active".
* Connect "br-prv" and "br-storage" bridges to "br-bond0" by OVS patches.
* Leave all of the other things unchanged.
Here is an example of "network_scheme" section in the node configuration:
::
'network_scheme':
'provider': 'ovs'
'version': '1.0'
'interfaces':
'eth0': {}
'eth1': {}
'eth2': {}
'eth3': {}
'endpoints':
'br-ex':
'IP': ['172.16.0.2/24']
'gateway': '172.16.0.1'
'br-mgmt':
'IP': ['192.168.0.2/24']
'br-prv': {'IP': 'none'}
'br-storage':
'IP': ['192.168.1.2/24']
'eth0':
'IP': ['10.20.0.4/24']
'roles':
'ex': 'br-ex'
'fw-admin': 'eth0'
'management': 'br-mgmt'
'private': 'br-prv'
'storage': 'br-storage'
'transformations':
- 'action': 'add-br'
'name': 'br-ex'
- 'action': 'add-br'
'name': 'br-mgmt'
- 'action': 'add-br'
'name': 'br-storage'
- 'action': 'add-br'
'name': 'br-prv'
- 'action': 'add-br'
'name': 'br-bond0'
- 'action': 'add-br'
'name': 'br-eth1'
- 'action': 'add-bond'
'bridge': 'br-bond0'
'interfaces': ['eth2', 'eth3']
'properties': ['lacp=active']
'name': 'bond0'
- 'action': 'add-port'
'bridge': 'br-eth1'
'name': 'eth1'
- 'action': 'add-patch'
'bridges': ['br-bond0', 'br-storage']
'tags': [103, 0]
- 'action': 'add-patch'
'bridges': ['br-eth1', 'br-ex']
'tags': [101, 0]
- 'action': 'add-patch'
'bridges': ['br-eth1', 'br-mgmt']
'tags': [102, 0]
- 'action': 'add-patch'
'bridges': ['br-bond0', 'br-prv']
NIC Aggregation in UI
---------------------
Use "Configure Interfaces" tab to configure interfaces bonding.
1. Select node(s) and click "Configure Interfaces".
Select interfaces to be aggregated. All interfaces except Admin-PXE can be aggregated.
.. image:: /_images/bonding-setup-in-ui-2.*
:width: 45%
:align: center
3. Click "Bond interfaces". Now you can select the appropriate bonding mode from the "Mode" drop-down list.
.. image:: /_images/bonding-setup-in-ui-3.*
:width: 45%
:align: center
4. Reassign networks, create additional bonds, etc. You can make all required changes and
click "Apply" after that.
.. image:: /_images/bonding-setup-in-ui-45.*
:width: 90%
:align: center
You can add one or more interfaces to the bond. Select a bond and the interface(s) to add,
then click "Bond Interfaces". Interface(s) can be removed from the bond when the bond has
3 or more slave interfaces. To remove interface from bond click "Remove"
at the left-bottom from interface icon. To unbond interfaces select bond and click "Unbond Interfaces".
.. include :: /pages/reference-architecture/ovs/0000-ovs-intro.rst
.. include :: /pages/reference-architecture/ovs/0100-neutron-ref-model.rst
.. include :: /pages/reference-architecture/ovs/0200-cli-adjust-network-config.rst
.. include :: /pages/reference-architecture/ovs/0230-cli-transformation.rst
.. include :: /pages/reference-architecture/ovs/0300-bonding-theory.rst
.. include :: /pages/reference-architecture/ovs/0330-bonding-cli-ex.rst
.. include :: /pages/reference-architecture/ovs/0350-bonding-ui.rst

14
pages/reference-architecture/logical-setup/0100-intro-logical-setup.rst Normal file
View File

@@ -0,0 +1,14 @@
.. raw:: pdf
PageBreak
.. index:: Reference Architectures: HA Logical Setup, HA Logical Setup
HA Logical Setup
================
.. contents :local:
An OpenStack Multi-node HA environment involves three types of nodes:
controller nodes, compute nodes, and storage nodes.

42
pages/reference-architecture/logical-setup/0200-controller-nodes.rst Normal file
View File

@@ -0,0 +1,42 @@
Controller Nodes
----------------
The first order of business in achieving high availability (HA) is
redundancy, so the first step is to provide multiple controller nodes.
As you may recall, the database uses Galera to achieve HA, and Galera is
a quorum-based system. That means that you should have at least 3
controller nodes.
.. image:: /_images/logical-diagram-controller.*
:width: 80%
:align: center
Every OpenStack controller runs HAProxy, which manages a single External
Virtual IP (VIP) for all controller nodes and provides HTTP and TCP load
balancing of requests going to OpenStack API services, RabbitMQ, and MySQL.
When an end user accesses the OpenStack cloud using Horizon or makes a
request to the REST API for services such as nova-api, glance-api,
keystone-api, neutron-api, nova-scheduler, MySQL or RabbitMQ, the
request goes to the live controller node currently holding the External VIP,
and the connection gets terminated by HAProxy. When the next request
comes in, HAProxy handles it, and may send it to the original
controller or another in the environment, depending on load conditions.
Each of the services housed on the controller nodes has its own
mechanism for achieving HA:
* nova-api, glance-api, keystone-api, neutron-api and nova-scheduler are
stateless services that do not require any special attention besides load
balancing.
* Horizon, as a typical web application, requires sticky sessions to be enabled
at the load balancer.
* RabbitMQ provides active/active high availability using mirrored queues.
* MySQL high availability is achieved through Galera active/active multi-master
deployment and Pacemaker.
* Neutron agents are managed by Pacemaker.
* Ceph monitors implement their own quorum based HA mechanism and
require time synchronization between all nodes. Clock drift higher
than 50ms may break the quorum or even crash the Ceph service.

15
pages/reference-architecture/logical-setup/0300-compute-nodes.rst Normal file
View File

@@ -0,0 +1,15 @@
Compute Nodes
-------------
OpenStack compute nodes are, in many ways, the foundation of your
environment; they are the servers on which your users will create their
Virtual Machines (VMs) and host their applications. Compute nodes need
to talk to controller nodes and reach out to essential services such
as RabbitMQ and MySQL. They use the same approach that provides
redundancy to the end-users of Horizon and REST APIs, reaching out to
controller nodes using the VIP and going through HAProxy.
.. image:: /_images/logical-diagram-compute.*
:width: 40%
:align: center

44
pages/reference-architecture/logical-setup/0400-storage-nodes.rst Normal file
View File

@@ -0,0 +1,44 @@
Storage Nodes
-------------
Depending on the :ref:`storage options <Storage_Architecture>` you
select for your environment, you may have Ceph, Cinder, and Swift
services running on your storage nodes.
Ceph_ implements its own HA, all you need is enough controller nodes
running Ceph Monitor service to `form a quorum
<http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-mon/>`_,
and enough Ceph OSD nodes to satisfy the `object replication factor
<http://ceph.com/docs/master/rados/operations/pools/>`_.
.. _Ceph: http://ceph.com/docs/master/architecture/
.. image:: /_images/ceph_nodes.*
:width: 80%
:align: center
Swift API relies on the same HAProxy setup with VIP on controller nodes
as the other REST APIs. If don't expect too much data traffic in Swift,
you can also deploy Swift Storage and Proxy services on controller
nodes. For a larger production environment you'll need dedicated nodes:
two for Swift Proxy and at least three for Swift Storage.
Whether or not you'd want separate Swift nodes depends primarily on how
much data you expect to keep there. A simple test is to fully populate
your Swift object store with data and then fail one controller node. If
replication of the degraded Swift objects between the remaining nodes
controller generates enough network traffic, CPU load, or disk I/O to
impact performance of other OpenStack services running on the same
nodes, you should separate Swift from controllers.
.. image:: /_images/logical-diagram-storage.*
:width: 40%
:align: center
If you select Cinder LVM as the block storage backend for Cinder
volumes, you should have at least one Cinder LVM node. Unlike Swift and
Ceph, Cinder LVM doesn't implement data redundancy across nodes: if a
Cinder node is lost, volumes stored on that node cannot be recovered
from the data stored on other Cinder nodes. If you need your block
storage to be resilient, use Ceph for volumes.

24
pages/reference-architecture/network-setup/0100-network-intro.rst Normal file
View File

@@ -0,0 +1,24 @@
.. raw:: pdf
PageBreak
.. index:: Network Architecture
Network Architecture
====================
.. contents :local:
For better network performance and manageability, Fuel place different types
of traffic into separate networks. This section describes how to distribute
the network traffic in an OpenStack cluster.
.. index:: Admin (PXE) Network
This network is reserved for communcation with Fuel master for provisioning
and orchestration of OpenStack deployment. It is used during installation to
provide DNS, DHCP, and gateway services to a node if it requires it before
provisioning. Nodes retrieve their network configuration via DHCP from Fuel
Master node. This is why it is important that this network is isolated from
your network and has no DHCP server (except Fuel Master) running on it.

28
pages/reference-architecture/network-setup/0200-public-net.rst Normal file
View File

@@ -0,0 +1,28 @@
.. index:: Public Network
Public Network
--------------
This network allows inbound connections to VMs from the outside world (allowing
users to connect to VMs from the Internet). It also allows outbound connections
from VMs to the outside world. For security reasons, the public network is usually
isolated from other networks in cluster. The word "Public" means that these addresses
can be used to communicate with cluster and its VMs from outside of cluster.
To enable external access to VMs, the public network provides the address space
for the floating IPs assigned to individual VM instances by the project
administrator. Nova Network or Neutron services can then
configure this address on the public network interface of the Network controller
node. E.g. environments based on Nova Network use iptables to create a
Destination NAT from this address to the private IP of the corresponding VM
instance through the appropriate virtual bridge interface on the Network
controller node.
In the other direction, the public network provides connectivity to the globally
routed address space for VMs. The IP address from the public network that has
been assigned to a compute node is used as the source for the Source NAT
performed for traffic going from VM instances on the compute node to Internet.
The public network also provides Virtual IPs for Endpoint nodes, which are used to
connect to OpenStack services APIs.

13
pages/reference-architecture/network-setup/0300-internal-mgmt-net.rst Normal file
View File

@@ -0,0 +1,13 @@
.. index:: Internal Network, Management Network
Internal (Management) Network
-----------------------------
The internal network connects all OpenStack nodes in the environment. All
components of an OpenStack environment communicate with each other using this
network. This network must be isolated from both the private and public
networks for security reasons.
The internal network can also be used for serving iSCSI protocol exchanges
between Compute and Storage nodes.

9
pages/reference-architecture/network-setup/0400-private-network.rst Normal file
View File

@@ -0,0 +1,9 @@
.. index:: Private Network
Private Network
---------------
The private network facilitates communication between each tenant's VMs. Private
network address spaces are not a part of the enterprise network address space. Fixed
IPs of virtual instances are directly unaccessible from the rest of Enterprise network.

25
pages/reference-architecture/network-setup/0500-nic-usage.rst Normal file
View File

@@ -0,0 +1,25 @@
NIC usage
---------
The current architecture assumes the presence of 3 NICs, but it can be
customized for two or 4+ network interfaces. Most servers are built with at least
two network interfaces. In this case, let's consider a typical example of three
NIC cards. They're utilized as follows:
**eth0**:
The Admin (PXE) network, used for communication with Fuel Master for
deployment.
**eth1**:
The public network and floating IPs assigned to VMs
**eth2**:
The private network, for communication between OpenStack VMs, and the
bridge interface (VLANs)
The figure below illustrates the relevant nodes and networks in Neutron VLAN mode.
.. image:: /_images/080-networking-diagram.*
:width: 75%
:align: center

25
pages/reference-architecture/neutron-intro/0100-neutron-vs-nova-network-intro.rst Normal file
View File

@@ -0,0 +1,25 @@
.. index:: Neutron vs. nova-network, Quantum vs. nova-network
Neutron vs. nova-network
------------------------
Neutron (formerly Quantum) is a service which provides Networking-as-a-Service
functionality in OpenStack. It has a rich tenant-facing API for defining
network connectivity and addressing in the cloud, and gives operators the
ability to leverage different networking technologies to power their cloud
networking.
There are various deployment use cases for Neutron. Fuel supports the most
common of them, called Per-tenant Routers with Private Networks.
Each tenant has a virtual Neutron router with one or more private networks,
which can communicate with the outside world.
This allows full routing isolation for each tenant private network.
Neutron is not, however, required in order to run an OpenStack environment. If
you don't need (or want) this added functionality, it's perfectly acceptable to
continue using nova-network.
In order to deploy Neutron, you need to enable it in the Fuel configuration.
Fuel sets up Neutron components on each of the controllers to act as a router
in HA (if deploying in HA mode).

21
pages/reference-architecture/neutron-intro/0150-mos-networks-list.rst Normal file
View File

@@ -0,0 +1,21 @@
Terminology
~~~~~~~~~~~
* **Public network** (also known as External network) used for Internet
access for all nodes.
* **Floating IP network** subnet within public network allocated for tenant
Internet access. A Neutron server directly assigns IP addresses for this network.
If a user delegates a floating IP address to an instance, an IP address will
be assigned from this subnet (by SNAT/DNAT).
* **Private network** used for passing tenant private traffic.
* **Admin network** shared network between Fuel Master and all nodes in the
cluster for PXE installation and orchestration of environment for deployment.
* **Storage network** network used for communicating between storage nodes
(using Ceph, swift, or cinder) and compute nodes.
* **Management network** (also known as Internal) used
for necessary communication between controllers and computes for AMQP
messaging, DB queries, other inter-controller traffic required for
supporting services.
* **Router** virtual Neutron router.
* **NIC** network interface card (ethernet adapter).

64
pages/reference-architecture/neutron-intro/0200-neutron-tech-intro.rst Normal file
View File

@@ -0,0 +1,64 @@
Overview
~~~~~~~~
OpenStack networking with Neutron (Quantum) has some differences from
Nova-network. Neutron is able to virtualize and manage both layer 2 (logical)
and layer 3 (network) of the OSI network model, as compared to simple layer 3
virtualization provided by nova-network. This is the main difference between
the two networking models for OpenStack. Virtual networks (one or more) can be
created for a single tenant, forming an isolated L2 network called a
"private network". Each private network can support one or many IP subnets.
Private networks can be segmented using two different technologies:
* **VLAN segmentation** "Private network" traffic is managed by
Neutron by the use of a dedicated network adapter. This network adapter must be
attached to a untagged network port. This network segment also must be
reserved only for Neutron on each host (Computes and Controllers). You should
not use any other 802.1q VLANs on this network for other purposes.
Additionally, each private network requires its own dedicated VLAN, selected
from a given range configured in Fuel UI.
* **GRE segmentation** In this mode of operation, Neutron does not
require a dedicated network adapter. Neutron builds a mesh of GRE tunnels from
each compute node and controller nodes to every other node. Private networks
for each tenant make use of this mesh for isolated traffic.
It is important to note:
* if you use tagged networks for your configuration
and combine multiple networks onto one NIC, you should make the Public
network untagged on this NIC.
It is not a requirement, but best for access to the OpenStack Dashboard
and public OpenStack API.
* Is a best if you place the Private, Admin, Public and Management networks on a
separate NIC to ensure that traffic is separated adequately.
* Admin and Private networks must be located together on separate NIC from the
other networks.
A typical network configuration for Neutron with VLAN segmentation might look
like this:
.. image:: /_images/Neutron_32_vlan_v2.png
:align: center
A typical network configuration for Neutron with GRE segmentation might look
like this:
.. image:: /_images/Neutron_32_gre_v2.png
:align: center
The most likely configuration for different number NICs on cluster nodes:
+------+--------------------------------------+--------------------------------------+
| NICs | VLAN | GRE |
+======+======================================+======================================+
| 2 | Not supported | .. image:: /_images/q32_gre_2nic.* |
| | | :align: center |
+------+--------------------------------------+--------------------------------------+
| 3 | .. image:: /_images/q32_vlan_3nic.* | .. image:: /_images/q32_gre_3nic.* |
| | :align: center | :align: center |
+------+--------------------------------------+--------------------------------------+
| 4 | .. image:: /_images/q32_vlan_4nic.* | .. image:: /_images/q32_gre_4nic.* |
| | :align: center | :align: center |
+------+--------------------------------------+--------------------------------------+

32
pages/reference-architecture/neutron-intro/0300-neutron-limits.rst Normal file
View File

@@ -0,0 +1,32 @@
Known limitations
~~~~~~~~~~~~~~~~~
* Neutron will not allocate a floating IP range for your tenants. After each
tenant is created, a floating IP range must be created. Note that this does
not prevent Internet connectivity for a tenant's instances, but it would
prevent them from receiving incoming connections. You, the administrator,
should assign a floating IP addresses for the tenant. Below are steps you can
follow to do this:
| get admin credentials:
| # source /root/openrc
| get admin tenant-ID:
| # keystone tenant-list
+----------------------------------+----------+---------+
| id | name | enabled |
+==================================+==========+=========+
| b796f91df6b84860a7cd474148fb2229 | admin | True |
+----------------------------------+----------+---------+
| cba7b0ff68ee4985816ac3585c8e23a9 | services | True |
+----------------------------------+----------+---------+
| create one floating-ip address for admin tenant:
| # neutron floatingip-create --tenant-id=b796f91df6b84860a7cd474148fb2229 net04_ext
* You can't combine Private or Admin network with any other networks on one NIC.
* To deploy OpenStack using Neutron with GRE segmentation, each node requires at
least 2 NICs.
* To deploy OpenStack using Neutron with VLAN segmentation, each node requires
at least 3 NICs.

10
pages/reference-architecture/neutron-intro/0600-virtualbox-promiscuous-mode.rst Normal file
View File

@@ -0,0 +1,10 @@
FAQ
~~~
| Q: I tried to deploy a Fuel OpenStack environment on VirtualBox, but the
deployment fails on Neutron setup. How do I fix this?
| A: You should to choose ”Allow all” promiscuous mode on all network
interfaces in VirtualBox and modify the network cards to use the PCnet
PCI II model network card.

17
pages/reference-architecture/ovs/0000-ovs-intro.rst Normal file
View File

@@ -0,0 +1,17 @@
.. raw:: pdf
PageBreak
.. index:: Advanced Network Configuration using OVS
Advanced Network Configuration using Open VSwitch
=================================================
The Neutron networking model uses Open VSwitch (OVS) bridges and the Linux
namespaces to create a flexible network setup and to isolate
tenants from each other on L2 and L3 layers. Mirantis OpenStack also
provides a flexible network setup model based on Open VSwitch primitives,
which you can use to customize your nodes. Its most popular feature is
link aggregation. While the FuelWeb UI uses a hardcoded
per-node network model, the Fuel CLI tool allows you to modify it in your own way.

24
pages/reference-architecture/ovs/0100-neutron-ref-model.rst Normal file
View File

@@ -0,0 +1,24 @@
Reference Network Model in Neutron
----------------------------------
The FuelWeb UI uses the following per-node network model:
* Create an OVS bridge for each NIC except for the NIC with Admin network
(for example, **br-eth0** bridge for **eth0** NIC) and put NICs into their bridges
* Create a separate bridge for each OpenStack network:
* **br-ex** for the Public network
* **br-prv** for the Private network
* **br-mgmt** for the Management network
* **br-storage** for the Storage network
* Connect each network's bridge with an apropriate NIC bridge using an OVS patch with
an apropriate VLAN tag.
* Assign network IP addresses to the corresponding bridges.
Note that the Admin network IP address is assigned to its NIC directly.
This network model allows the cluster administrator to manipulate cluster
network entities and NICs separately, easily, and on the fly during the cluster
life-cycle.

35
pages/reference-architecture/ovs/0200-cli-adjust-network-config.rst Normal file
View File

@@ -0,0 +1,35 @@
Adjust the Network Configuration via CLI
----------------------------------------
On a basic level, this network configuration is part of a data structure that provides
instructions to the Puppet modules to set up a network on the current node.
You can examine and modify this data using the Fuel CLI tool. Just download (then
modify and upload if needed) the environment's 'deployment default' configuration:
::
[root@fuel ~]# fuel --env 1 deployment default
directory /root/deployment_1 was created
Created /root/deployment_1/compute_1.yaml
Created /root/deployment_1/controller_2.yaml
[root@fuel ~]# vi ./deployment_1/compute_1.yaml
[root@fuel ~]# fuel --env 1 deployment --upload
.. note::
Please, make sure you read :ref:`the Fuel CLI documentation <cli_usage>` carefully.
The part of this data structure that describes how to apply the network configuration
is the 'network_scheme' key in the top-level hash of the YAML file. Let's take a
closer look at this substructure. The value of the 'network_scheme' key is a hash with
the following keys:
* **interfaces** - A hash of NICs and their low-level/physical parameters.
You can set an MTU and the 'VLAN splinters' feature here.
* **provider** - Set to 'ovs' for Neutron.
* **endpoints** - A hash of network ports (OVS ports or NICs) and their IP
settings.
* **roles** - A hash that specifies the mappings between the endpoints and
internally-used roles in Puppet manifests ('management', 'storage', and so on).
* **transformations** - An ordered list of OVS network primitives.

88
pages/reference-architecture/ovs/0230-cli-transformation.rst Normal file
View File

@@ -0,0 +1,88 @@
The "Transformations" Section
-----------------------------
You can use four OVS primitives:
* **add-br** - To add an OVS bridge to the system
* **add-port** - To add a port to an existent OVS bridge
* **add-bond** - To create a port in OVS bridge and add aggregated NICs to it
* **add-patch** - To create an OVS patch between two existing OVS bridges
The primitives will be applied in the order they are listed.
Here are the the available options:
::
{
"action": "add-br", # type of primitive
"name": "xxx" # unique name of the new bridge
},
{
"action": "add-port", # type of primitive
"name": "xxx-port", # unique name of the new port
"bridge": "xxx", # name of the bridge where the port should be created
"type": "internal", # [optional; default: "internal"] a type of OVS interface
# for the port (see OVS documentation);
# possible values: "system", "internal", "tap", "gre", "null"
"tag": 0, # [optional; default: 0] a 802.1q tag of traffic that
# should be captured from an OVS bridge;
# possible values: 0 (means port is a trunk),
# 1-4094 (means port is an access)
"trunks": [], # [optional; default: []] a set of 802.1q tags
# (integers from 0 to 4095) that are allowed to
# pass through if "tag" option equals 0;
# possible values: an empty list (all traffic passes),
# 0 (untagged traffic only), 1 (strange behaviour;
# shouldn't be used), 2-4095 (traffic with this
# tag passes); e.g. [0,10,20]
"port_properties": [], # [optional; default: []] a list of additional
# OVS port properties to modify them in OVS DB
"interface_properties": [], # [optional; default: []] a list of additional
# OVS interface properties to modify them in OVS DB
"vlan_splinters": false, # [optional; default: false] enable 'vlan splinters'
# feature for this interface
},
{
"action": "add-bond", # type of primitive
"name": "xxx-port", # unique name of the new bond
"interfaces": [], # a set of two or more bonded interfaces' names;
# e.g. ['eth1','eth2']
"bridge": "xxx", # name of the bridge where the bond should be created
"tag": 0, # [optional; default: 0] a 802.1q tag of traffic which
# should be catched from an OVS bridge;
# possible values: 0 (means port is a trunk),
# 1-4094 (means port is an access)
"trunks": [], # [optional; default: []] a set of 802.1q tags
# (integers from 0 to 4095) which are allowed to
# pass through if "tag" option equals 0;
# possible values: an empty list (all traffic passes),
# 0 (untagged traffic only), 1 (strange behaviour;
# shouldn't be used), 2-4095 (traffic with this
# tag passes); e.g. [0,10,20]
"properties": [], # [optional; default: []] a list of additional
# OVS bonded port properties to modify them in OVS DB;
# you can use it to set the aggregation mode and balancing
# strategy, to configure LACP, and so on (see the OVS documentation)
},
{
"action": "add-patch", # type of primitive
"bridges": ["br0", "br1"], # a pair of different bridges that will be connected
"peers": ["p1", "p2"], # [optional] abstract names for each end of the patch
"tags": [0, 0] , # [optional; default: [0,0]] a pair of integers that
# represent an 802.1q tag of traffic that is
# captured from an appropriate OVS bridge; possible
# values: 0 (means port is a trunk), 1-4094 (means
# port is an access)
"trunks": [], # [optional; default: []] a set of 802.1q tags
# (integers from 0 to 4095) which are allowed to
# pass through each bridge if "tag" option equals 0;
# possible values: an empty list (all traffic passes),
# 0 (untagged traffic only), 1 (strange behavior;
# shouldn't be used), 2-4095 (traffic with this
# tag passes); e.g., [0,10,20]
}
A combination of these primitives allows you to make custom and complex
network configurations.

57
pages/reference-architecture/ovs/0300-bonding-theory.rst Normal file
View File

@@ -0,0 +1,57 @@
NICs Aggregation
----------------
The NIC bonding allows you to aggregate multiple physical links to one link
to increase speed and provide fault tolerance.
**Documentation**
The Linux kernel documentation about bonding can be found in Linux Ethernet Bonding Driver HOWTO:
https://www.kernel.org/doc/Documentation/networking/bonding.txt.
You can find shorter introduction to bonding and tips on link monitoring here:
http://wiki.mikrotik.com/wiki/Manual:Interface/Bonding
Cisco switches configuration guide:
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/layer2/503_U2_1/b_Cisco_n3k_layer2_config_guide_503_U2_1/b_Cisco_n3k_layer2_config_gd_503_U2_1_chapter_01000.html
Switches configuration tips for Fuel:
https://etherpad.openstack.org/p/LACP_FUEL_bonding
**Types of Bonding**
Open VSwitch supports the same bonding features as the Linux kernel.
Fuel supports bonding via Open VSwitch only.
Linux supports two types of bonding:
* IEEE 802.1AX (formerly known as 802.3ad) Link Aggregation Control Protocol (LACP). Devices on both sides of links must communicate using LACP to set up an aggregated link. So both devices must support LACP, enable and configure it on these links.
* One side bonding does not require any special feature support from the switch side. Linux handles it using a set of traffic balancing algorithms.
One Side Bonding Policies:
* Balance-rr - Round-robin policy. This mode provides load balancing and fault tolerance.
* Active-backup - Active-backup policy: Only one slave in the bond is active.This mode provides fault tolerance.
* Balance-xor - XOR policy: Transmit based on the selected transmit hash policy. This mode provides load balancing and fault tolerance.
* Broadcast - Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.
* balance-tlb - Adaptive transmit load balancing based on a current links' utilisation. This mode provides load balancing and fault tolerance.
* balance-alb - Adaptive transmit and receive load balancing based on the current links' utilisation. This mode provides load balancing and fault tolerance.
* balance-slb - Modification of balance-alb mode. SLB bonding allows a limited form of load balancing without the
remote switch's knowledge or cooperation. SLB assigns each source MAC+VLAN pair to a link and transmits all packets from
that MAC+VLAN through that link. Learning in the remote switch causes it to send packets to that MAC+VLAN through the same link.
* balance-tcp - Adaptive transmit load balancing among interfaces.
LACP Policies:
* Layer2 - Uses XOR of hardware MAC addresses to generate the hash.
* Layer2+3 - uses a combination of layer2 and layer3 protocol information to generate the hash.
* Layer3+4 - uses upper layer protocol information, when available, to generate the hash.
* Encap2+3 - uses the same formula as layer2+3 but it relies on skb_flow_dissect to obtain the header fields which might result in the use of inner headers if an encapsulation protocol is used. For example this will improve the performance for tunnel users because the packets will be distributed according to the encapsulated flows.
* Encap3+4 - Similar to Encap2+3 but uses layer3+4.
**Policies Supported by Fuel**
Fuel supports the following policies: Active-backup, balance-slb, LACP balance-tcp.
**Network Verification in Fuel**
Fuel has limited network verification capabilities when working with bonds.
Network connectivity can be checked for the new cluster only (not for deployed one)
so check is done when nodes are in bootstrap and no bonds are up. Connectivity
between slave interfaces can be checked but not bonds themselves.

76
pages/reference-architecture/ovs/0330-bonding-cli-ex.rst Normal file
View File

@@ -0,0 +1,76 @@
An Example of NIC Aggregation
-----------------------------
Suppose you have a node with 4 NICs and you want to bond two of them with LACP
enabled ("eth2" and "eth3" here) and then assign Private and Storage networks
to them. The Admin network uses a dedicated NIC ("eth0"). The Management and
Public networks use the last NIC ("eth1"). To achieve this goal, do the following:
* Create a separate OVS bridge "br-bond0" instead of "br-eth2" and "br-eth3".
* Connect "eth2" and "eth3" to "br-bond0" as a bonded port with property
"lacp=active".
* Connect "br-prv" and "br-storage" bridges to "br-bond0" by OVS patches.
* Leave all of the other things unchanged.
Here is an example of "network_scheme" section in the node configuration:
::
'network_scheme':
'provider': 'ovs'
'version': '1.0'
'interfaces':
'eth0': {}
'eth1': {}
'eth2': {}
'eth3': {}
'endpoints':
'br-ex':
'IP': ['172.16.0.2/24']
'gateway': '172.16.0.1'
'br-mgmt':
'IP': ['192.168.0.2/24']
'br-prv': {'IP': 'none'}
'br-storage':
'IP': ['192.168.1.2/24']
'eth0':
'IP': ['10.20.0.4/24']
'roles':
'ex': 'br-ex'
'fw-admin': 'eth0'
'management': 'br-mgmt'
'private': 'br-prv'
'storage': 'br-storage'
'transformations':
- 'action': 'add-br'
'name': 'br-ex'
- 'action': 'add-br'
'name': 'br-mgmt'
- 'action': 'add-br'
'name': 'br-storage'
- 'action': 'add-br'
'name': 'br-prv'
- 'action': 'add-br'
'name': 'br-bond0'
- 'action': 'add-br'
'name': 'br-eth1'
- 'action': 'add-bond'
'bridge': 'br-bond0'
'interfaces': ['eth2', 'eth3']
'properties': ['lacp=active']
'name': 'bond0'
- 'action': 'add-port'
'bridge': 'br-eth1'
'name': 'eth1'
- 'action': 'add-patch'
'bridges': ['br-bond0', 'br-storage']
'tags': [103, 0]
- 'action': 'add-patch'
'bridges': ['br-eth1', 'br-ex']
'tags': [101, 0]
- 'action': 'add-patch'
'bridges': ['br-eth1', 'br-mgmt']
'tags': [102, 0]
- 'action': 'add-patch'
'bridges': ['br-bond0', 'br-prv']

30
pages/reference-architecture/ovs/0350-bonding-ui.rst Normal file
View File

@@ -0,0 +1,30 @@
NIC Aggregation in UI
---------------------
Use "Configure Interfaces" tab to configure interfaces bonding.
1. Select node(s) and click "Configure Interfaces".
Select interfaces to be aggregated. All interfaces except Admin-PXE can be aggregated.
.. image:: /_images/bonding-setup-in-ui-2.*
:width: 45%
:align: center
3. Click "Bond interfaces". Now you can select the appropriate bonding mode from the "Mode" drop-down list.
.. image:: /_images/bonding-setup-in-ui-3.*
:width: 45%
:align: center
4. Reassign networks, create additional bonds, etc. You can make all required changes and
click "Apply" after that.
.. image:: /_images/bonding-setup-in-ui-45.*
:width: 90%
:align: center
You can add one or more interfaces to the bond. Select a bond and the interface(s) to add,
then click "Bond Interfaces". Interface(s) can be removed from the bond when the bond has
3 or more slave interfaces. To remove interface from bond click "Remove"
at the left-bottom from interface icon. To unbond interfaces select bond and click "Unbond Interfaces".

7
pages/reference-architecture/storage/0000-storage-intro.rst Normal file
View File

@@ -0,0 +1,7 @@
.. index:: Storage architecture, Object storage, Cinder, Swift, Glance, Ceph
.. _Storage_Architecture:
Storage Architecture
====================

68
pages/reference-architecture/storage/0100-object-store-for-images.rst Normal file
View File

@@ -0,0 +1,68 @@
Object Storage for Images
-------------------------
.. _Object_Storage_for_Images:
Fuel can configure one of these storage backends for the Glance image
service:
* File system backend,
* `Swift object store <http://swift.openstack.org/>`_, the standard
OpenStack object storage component,
* `Ceph RBD <http://ceph.com/docs/master/rbd/rbd-openstack/>`_, the
distributed block device service based on RADOS, the object store
component of the Ceph storage platform.
By default, Glance stores virtual machine images using the **file
system** backend. With it, you can use any of the shared file systems
supported by Glance. Fuel will default to this option in a simple non-HA
deployment with a local file system on a single controller node.
For a production HA environment where you want data resilience for VM
images, the best practice is to use an object store as the backend for
Glance. That way, multiple Glance instances running on controller nodes
can store and retrieve images from the same data set, while the object
store takes care of data protection and HA.
In HA deployment mode, Fuel will default to using **Swift** as the
storage backend for Glance instead of the file system backend. In both
HA and non-HA multi-node configurations, Fuel will also offer you the
option to use Ceph as storage backend.
The primary advantage of using **Ceph RBD** for images is the ability to
unify different classes of data into a single shared pool of storage
nodes that can handle all classes of data important for OpenStack.
Instead of having to copy OS images and volumes between separate Glance,
Cinder, and Nova storage pools, you can have all three services use
copy-on-write clones of the original image. In addition to better
utilization of storage capacity, copy-on-write also significantly speeds
up launching VMs from images.
To make the most out of the copy-on-write capability of the Ceph
backend, you should only use images in **raw** format. Images in other
formats such as qcow2 have to be converted to raw format first and
cannot be cloned without conversion.
As of this writing, vanilla OpenStack Havana release placed several
important limitations on the copy-on-write capability of the Ceph
backend:
* You have to create an RBD backed bootable volume from a raw image for
copy-on-write and live migrations to work;
* Launching an instance directly from image results in a full copy on
the compute node instead of a copy-on-write clone in Ceph;
* Ephemeral drives are stored in local files on compute nodes instead
of Ceph, preventing live migration of instances with ephemenral
drives;
* Non-raw images are not converted to raw automatically when creating a
bootable volume: if you don't convert the image yourself, you will
end up with a clone of a qcow2 image that will not be bootable.
These limitations are removed in Mirantis OpenStack distribution
starting with release 4.0.

21
pages/reference-architecture/storage/0200-object-storage-for-apps.rst Normal file
View File

@@ -0,0 +1,21 @@
Object Storage for Applications
-------------------------------
The objects storage systems supported by Mirantis OpenStack don't have
to be limited to serving Glance. Swift provides a REST API that can be
used by any application that needs to store data in an object store, and
is immediately available whenever you have Swift running.
Ceph includes the optional radosgw_ object gateway component which
allows to access objects in RADOS object store using REST interfaces
compatible with Amazon S3 and Swift APIs.
.. _radosgw: http://ceph.com/docs/master/radosgw/
Ceph RBD backend for Glance doesn't use Swift API and uses RADOS
directly, so it is possible to store Glance images in Ceph and still use
Swift as object store for applications. This does not work the other way
around: when you choose to install the Ceph object gateway, it replaces
Swift as the provider of Swift API, so you can't have both radosgw and
Swift in the same OpenStack environment.

30
pages/reference-architecture/storage/0300-block-storage-for-volumes.rst Normal file
View File

@@ -0,0 +1,30 @@
Block Storage for Volumes
-------------------------
When you configure Mirantis OpenStack to use the default **LVM** backend
for Cinder block devices, Cinder will store each volume as a logical
volume in an LVM volume group on one of your Cinder nodes.
If you don't need your volumes to be resilient, you can let Fuel create
a JBOD partition spanning all your storage drives in a node during
deployment, or you can join all drives into a RAID array before
deployment, and have the array appear to Fuel as a single block device.
Even if you use RAID in each Cinder node, that only protects your data
from a hard drive failure. If the whole Cinder node is lost, so are all
volumes that were stored on that node.
Setting Cinder backend to **Ceph RBD** allows to take advantage of
Ceph's object replication capabilities by storing Cinder volumes as
objects in Ceph RADOS object storage system. By default, Ceph simply
ensures that every replica of an object is stored on a different node.
The set of data replication rules (CRUSH map) can be customized
separately for each object pool to change the number of object replicas,
add different types of failure domains, etc. The amount of storage
required to host your volumes will be multiplied by the replication
factor you configure for Ceph, but your volumes will be protected from
node or even data center failure.
If you combine RBD backends for Cinder and Glance, you gain another
important advantage over Cinder LVM: copy-on-write cloning of Glance
images into bootable Ceph volumes.

2
reference-architecture.rst
View File

@@ -18,6 +18,6 @@ Reference Architectures
.. include:: /pages/reference-architecture/0040-network-setup.rst
.. include:: /pages/reference-architecture/0045-storage.rst
.. include:: /pages/reference-architecture/0050-technical-considerations-overview.rst
.. include:: /pages/reference-architecture/0060-quantum-vs-nova-network.rst
.. include:: /pages/reference-architecture/0060-neutron-vs-nova-network.rst
.. include:: /pages/reference-architecture/0090-savanna.rst
.. include:: /pages/reference-architecture/0100-bonding.rst