General arch, added Ceph and Galera info
A quick fix to reflect that we no longer have separate multi-node and multi-node HA models in 5.0. Also moved information about Ceph and MySQL/Galera from FAQ doc and augmented it with information from Miroslav Anashkin and Dmitriy Novakovskiy. This removes the last bits of the FAQ, which is now completely removed from the landing page. The "bigpicture" architectural description needs additional work but this package is reasonable accurate as far as it goes. Co-Authored-By: Dmitry Borodaenko <dborodaenko@mirantis.com> Change-Id: Ia29d906a8d1377c08c822262f4d3759b9fd84593
This commit is contained in:
Meg McRoberts
committed by
Dmitry Borodaenko
Dmitry Borodaenko
parent
2146224e53
commit
341cf42c1d
@@ -18,6 +18,5 @@ Documentation
|
||||
operations
|
||||
reference-architecture
|
||||
release-notes
|
||||
frequently-asked-questions
|
||||
terminology
|
||||
eula
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
.. include:: /pages/frequently-asked-questions/0100-ha-notes.rst
|
||||
@@ -1,8 +1,11 @@
|
||||
|
||||
.. include:: /pages/reference-architecture/0010-overview.rst
|
||||
.. include:: /pages/reference-architecture/0012-simple.rst
|
||||
.. include:: /pages/reference-architecture/0014-compact.rst
|
||||
.. include:: /pages/reference-architecture/0015-closer-look.rst
|
||||
.. include:: /pages/reference-architecture/0020-logical-setup.rst
|
||||
.. include:: /pages/reference-architecture/1000-logical-setup.rst
|
||||
.. include:: /pages/reference-architecture/1100-ha-notes.rst
|
||||
.. include:: /pages/reference-architecture/1200-mysql-galera.rst
|
||||
.. include:: /pages/reference-architecture/1700-ceph.rst
|
||||
.. include:: /pages/reference-architecture/0030-cluster-sizing.rst
|
||||
.. include:: /pages/reference-architecture/0040-network-setup.rst
|
||||
.. include:: /pages/reference-architecture/0045-storage.rst
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
.. index:: F.A.Q.
|
||||
|
||||
.. _F.A.Q.:
|
||||
|
||||
===========================================
|
||||
F.A.Q
|
||||
===========================================
|
||||
|
||||
.. contents:: :local:
|
||||
:depth: 3
|
||||
|
||||
.. include:: /contents/contents-faq.rst
|
||||
@@ -9,7 +9,4 @@
|
||||
How HA with Pacemaker and Corosync Works
|
||||
========================================
|
||||
|
||||
.. include:: /pages/frequently-asked-questions/0110-corosync-settings.rst
|
||||
.. include:: /pages/frequently-asked-questions/0120-pacemaker-settings.rst
|
||||
.. include:: /pages/frequently-asked-questions/0130-how-fuel-deploys-ha.rst
|
||||
|
||||
|
||||
@@ -30,45 +30,6 @@ Other Questions
|
||||
this would not be possible, as that repository only keeps the latest version
|
||||
for OpenStack packages.
|
||||
|
||||
2. **[Q]** Is MySQL with Galera an active/active HA? Does it support
|
||||
multi-master writes? A simple workflow example would be helpful.
|
||||
|
||||
**[A]** Yes, MySQL+Galera is a true multi-master solution. Although MySQL+Galera
|
||||
supports multi-master topology, Mirantis OpenStack configures MySQL+Galera to
|
||||
have only a single active node (via HAProxy) to receive writes and serve
|
||||
reads, and uses the remaining cluster nodes as standby masters.
|
||||
It is important to note, however, that unlike regular MySQL master/slave
|
||||
topologies, these standby masters do not have "slave lag", as Galera employs
|
||||
synchronous replication and ensures each cluster node is identical.
|
||||
Previous Fuel versions used only HAProxy as a MySQL management solution,
|
||||
but version 3.0 and later of Mirantis OpenStack uses Pacemaker and HAProxy
|
||||
to manage MySQL+Galera.
|
||||
|
||||
Pacemaker manages the individual MySQL+Galera nodes, HAProxy, and the
|
||||
Virtual IP Address (VIP). HAProxy manages connections between MySQL+Galera
|
||||
active master, backup masters, and the MySQL Clients connecting to the VIP.
|
||||
Only one MySQL+Galera master is active in the VIP as the single direction
|
||||
synchronous replication performs better in most cases.
|
||||
|
||||
The Workflow is simple: One node tied to the VIP serves new data updates and
|
||||
increases its global transaction ID number (GTID). The rest of the Galera
|
||||
cluster must then synchronize the data from the nodes with GTID greater than
|
||||
their current value. If the status of any node falls too far behind the
|
||||
Galera cache, an entire replica is distributed to that node. This will cause
|
||||
a master to switch to the Donor role allowing an out-of-sync node to catch
|
||||
up.
|
||||
|
||||
3. **[Q]** Are the Ceph monitors on the controllers in active/active HA?
|
||||
|
||||
**[A]** Yes, the Ceph Monitors (MON) use the Paxos algorithm to determine
|
||||
all updates to the data they manage. All monitors that are in quorum will
|
||||
have consistant up-to-date data because of this.
|
||||
|
||||
Ceph monitors manage various maps like MON map, CRUSH map, and others. The
|
||||
CRUSH map is used by clients to deterministically select the storage devices (OSDs)
|
||||
to receive copies of the data.
|
||||
|
||||
You can read .. _more about ceph: http://ceph.com/docs/master/architecture
|
||||
|
||||
4. **[Q]** Is Neutron an active/standy HA? I got this understanding from the docs
|
||||
and I want to understand why. I was told that Grizzly and Havanna support multiple
|
||||
|
||||
@@ -4,51 +4,57 @@
|
||||
|
||||
.. index Reference Architectures
|
||||
|
||||
Overview
|
||||
========
|
||||
OpenStack Environment Architecture
|
||||
==================================
|
||||
|
||||
.. contents :local:
|
||||
|
||||
Before you install any hardware or software, you must know what
|
||||
you're trying to achieve. This section looks at the basic components of
|
||||
an OpenStack infrastructure and organizes them into one of the more
|
||||
common reference architectures. You'll then use that architecture as a
|
||||
basis for installing OpenStack in the next section.
|
||||
Fuel deploys an OpenStack Environment
|
||||
with nodes that provide a specific set of functionality.
|
||||
Beginning with Fuel 5.0,
|
||||
a single architecture model can support HA (High Availability)
|
||||
and non-HA deployments;
|
||||
you can deploy a non-HA environment
|
||||
and then add additional nodes to implement HA
|
||||
rather than needing to redeploy the environment from scratch.
|
||||
|
||||
As you know, OpenStack provides the following basic services:
|
||||
The OpenStack environment consists of multiple physical server nodes
|
||||
(or an equivalent VM),
|
||||
each of which is one of the following node types:
|
||||
|
||||
**Controller:**
|
||||
The Controller manages all activities in the environment.
|
||||
`nova-controller` maintains the life cycle of the Controller.
|
||||
along with RabbitMQ, HAProxy, MySQL/Galera,
|
||||
the Pacemaker Cluster (Corosync and Pacemaker),
|
||||
Keystone, Glance, and Cinder.
|
||||
Other services that may optionally run on the Controller include
|
||||
Heat, Neutron, Swift, Ceph Monitor, Ceilometer,
|
||||
Sahara, and Murano.
|
||||
|
||||
**Compute:**
|
||||
Compute servers are the workhorses of your installation; they're
|
||||
the servers on which your users' virtual machines are created.
|
||||
`nova-compute` controls the life cycle of these VMs.
|
||||
|
||||
**Networking:**
|
||||
Typically, an OpenStack environment includes multiple servers that
|
||||
need to communicate to each other and to outside world. Fuel supports
|
||||
both old `nova-network` and new `neutron` based OpenStack Networking
|
||||
implementations:
|
||||
|
||||
* With `nova-network`, Flat-DHCP and VLAN modes are available.
|
||||
|
||||
* With `neutron`, GRE tunnels or VLANs can be used for network
|
||||
segmentation.
|
||||
Compute servers are the workhorses of your installation;
|
||||
they are the servers on which your users' virtual machines are created.
|
||||
`nova-compute` controls the life cycle of these VMs;
|
||||
Neutron Agent and Ceilometer Compute may also run on Compute nodes.
|
||||
|
||||
**Storage:**
|
||||
OpenStack requires block and object storage to be provisioned. Fuel
|
||||
provides the following storage options out of the box:
|
||||
OpenStack requires block and object storage to be provisioned.
|
||||
These can be provisioned as Storage nodes
|
||||
or as roles that run on Compute nodes.
|
||||
Fuel provides the following storage options out of the box:
|
||||
|
||||
* Cinder LVM provides persistent block storage to virtual machines
|
||||
over iSCSI protocol
|
||||
over iSCSI protocol. The Cinder Storage node runs a Cinder Volume.
|
||||
|
||||
* Swift object store can be used by Glance to store VM images and
|
||||
snapshots, it may also be used directly by applications
|
||||
* Swift object store can be used by Glance to store VM images and snapshots;
|
||||
it may also be used directly by applications
|
||||
Swift is the default storage provider that is provisioned
|
||||
if another storage option is not chosen when the environment is deployed.
|
||||
|
||||
* Ceph combines object and block storage and can replace either one or
|
||||
both of the above.
|
||||
The Ceph Storage node runs Ceph OSD.
|
||||
|
||||
Compute, Networking, and Storage services can be combined in many
|
||||
different ways. Out of the box, Fuel supports the following deployment
|
||||
configurations:
|
||||
|
||||
- :ref:`Multi-node <Multi-node>`
|
||||
- :ref:`Multi-node with HA <Multi-node_HA>`
|
||||
The key principle is that your controller(s) are separate from
|
||||
the compute servers on which your user's VMs run.
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
.. raw:: pdf
|
||||
|
||||
PageBreak
|
||||
|
||||
.. index:: Reference Architectures: Multi-node
|
||||
|
||||
.. _Multi-node:
|
||||
|
||||
Multi-node Deployment
|
||||
========================================
|
||||
|
||||
For a production environment, you will typically create a high-availability deployment of OpenStack. However, the Multi-node (non-HA) deployment is extremely useful if you just want to see how OpenStack works from a user's point of view.
|
||||
|
||||
.. image:: /_images/deployment-simple.*
|
||||
:width: 60%
|
||||
:align: center
|
||||
|
||||
More commonly, your OpenStack installation will consist of multiple
|
||||
servers. Exactly how many is up to you, of course, but the main idea
|
||||
is that your controller(s) are separate from your compute servers, on
|
||||
which your users' VMs will actually run. One arrangement that will
|
||||
enable you to achieve this separation while still keeping your
|
||||
hardware investment relatively modest is to house your storage on your
|
||||
controller nodes.
|
||||
@@ -5,17 +5,15 @@
|
||||
Multi-node with HA Deployment
|
||||
=============================
|
||||
|
||||
Production environments typically require high availability, which
|
||||
involves several architectural requirements. Specifically, you will
|
||||
need at least three controllers, and
|
||||
certain components will be deployed in multiple locations to prevent
|
||||
single points of failure. That's not to say, however, that you can't
|
||||
reduce hardware requirements by combining your storage, network, and controller
|
||||
nodes:
|
||||
High availability is recommended for production environments.
|
||||
This provides replicated servers to prevent single points of failure.
|
||||
An HA deployment must have at least three controllers
|
||||
as well as replicas of other servers.
|
||||
You can combine compute, storage, and network nodes
|
||||
to reduce the hardware requirements for the environment,
|
||||
although this may degrade the performance and robustness of the environment.
|
||||
|
||||
.. image:: /_images/deployment-ha-compact.*
|
||||
:width: 80%
|
||||
:align: center
|
||||
|
||||
We'll take a closer look at the details of this deployment configuration in
|
||||
:ref:`Close_look_Multi-node_HA` section.
|
||||
|
||||
13
pages/reference-architecture/1100-ha-notes.rst
Normal file
13
pages/reference-architecture/1100-ha-notes.rst
Normal file
@@ -0,0 +1,13 @@
|
||||
.. raw:: pdf
|
||||
|
||||
PageBreak
|
||||
|
||||
.. index:: HA with Pacemaker and Corosync
|
||||
|
||||
How HA with Pacemaker and Corosync Works
|
||||
========================================
|
||||
|
||||
.. include:: /pages/reference-architecture/ha-notes/0110-corosync-settings.rst
|
||||
.. include:: /pages/reference-architecture/ha-notes/0120-pacemaker-settings.rst
|
||||
.. include:: /pages/reference-architecture/ha-notes/0130-how-fuel-deploys-ha.rst
|
||||
|
||||
41
pages/reference-architecture/1200-mysql-galera.rst
Normal file
41
pages/reference-architecture/1200-mysql-galera.rst
Normal file
@@ -0,0 +1,41 @@
|
||||
|
||||
.. _mysql-galera-arch:
|
||||
|
||||
MySQL and Galera
|
||||
================
|
||||
|
||||
My SQL with Galera implements true active/active HA.
|
||||
Fuel configures MySQL/Galera to have a single active node
|
||||
that receives write operations and serves read operations.
|
||||
You can add one or two Galera slave nodes;
|
||||
this is recommended for environments that have six or more nodes:
|
||||
|
||||
- Only one MySQL/Galera node is considered active at a time;
|
||||
the remaining cluster nodes are standby masters.
|
||||
- The standby masters do not have the "slave lag"
|
||||
that is typical for MySQL master/slave topologies
|
||||
because Galera employs synchronous replication
|
||||
and ensures that each cluster node is identical.
|
||||
- Mirantis OpenStack uses Pacemaker and HAProxy to manage MySQL/Galera:
|
||||
|
||||
* Pacemaker manages the individual MySQL+Galera nodes, HAProxy,
|
||||
and the Virtual IP Address (VIP).
|
||||
* HAProxy runs in the dedicated network namespace
|
||||
and manages connections between the MySQL/Galera active master,
|
||||
backup masters, and the MySQL Clients connecting to the VIP.
|
||||
|
||||
- Only one MySQL/Galera master is active in the VIP;
|
||||
this single direction synchronous replication
|
||||
usually provides better preformance than other implementations.
|
||||
|
||||
The workflow is:
|
||||
|
||||
- The node that is tied to the VIP serves new data updates
|
||||
and increases its global transaction ID number (GTID).
|
||||
- Each other node in the Galera cluster then synchronizes its data
|
||||
with the node that has a GTID greater than its current values.
|
||||
- If the status of any node falls too far behind the Galera cache,
|
||||
an entire replica is distributed to that node.
|
||||
This causes a master to switch to the Donor role,
|
||||
allowing an out-of-sync node to catch up.
|
||||
|
||||
31
pages/reference-architecture/1700-ceph.rst
Normal file
31
pages/reference-architecture/1700-ceph.rst
Normal file
@@ -0,0 +1,31 @@
|
||||
.. _ceph-mon-arch:
|
||||
|
||||
Ceph Monitors
|
||||
=============
|
||||
|
||||
Ceph monitors (MON) manage various maps like MON map, CRUSH map, and others.
|
||||
The CRUSH map is used by clients
|
||||
to deterministically select the storage devices (OSDs)
|
||||
to receive copies of the data.
|
||||
Ceph monitor nodes manage where the data should be stored
|
||||
and maintain data consistency with the Ceph OSD nodes
|
||||
that store the actual data.
|
||||
|
||||
Ceph monitors implement HA using a master-master model:
|
||||
|
||||
- One Ceph monitor node is designated the "leader."
|
||||
This is the node that first received
|
||||
the most recent cluster map replica.
|
||||
- Each other monitor node must sync its cluster map with the current leader.
|
||||
- Each monitor node that is already sync'ed with the leader becomes a provider;
|
||||
the leader knows which nodes are currently providers.
|
||||
The leader tells the other nodes which provider
|
||||
they should use to sync their data.
|
||||
|
||||
Ceph Monitors use the Paxos algorithm
|
||||
to determine all updates to the data they manage.
|
||||
All monitors that are in quorum
|
||||
have consistant up-to-date data because of this.
|
||||
|
||||
You can read more in `Ceph documentation <http://ceph.com/docs/master/architecture>`_.
|
||||
|
||||
@@ -3,10 +3,11 @@
|
||||
How Fuel Deploys HA
|
||||
-------------------
|
||||
|
||||
Fuel installs Corosync service, configures ``corosync.conf`` and includes Pacemaker
|
||||
service plugin into ``/etc/corosync/service.d``. Then Corosync service starts and
|
||||
spawns corresponding Pacemaker processes. Fuel configures cluster properties of
|
||||
Pacemaker and then injects resources configuration for virtual IPs, HAProxy,
|
||||
Fuel installs Corosync service, configures ``corosync.conf``,
|
||||
and includes the Pacemaker service plugin into ``/etc/corosync/service.d``.
|
||||
Then Corosync service starts and spawns corresponding Pacemaker processes.
|
||||
Fuel configures the cluster properties of Pacemaker
|
||||
and then injects resource configurations for virtual IPs, HAProxy,
|
||||
MySQL and Neutron agent resources::
|
||||
|
||||
primitive p_haproxy ocf:pacemaker:haproxy \
|
||||
@@ -5,6 +5,6 @@ Galera Cluster for MySQL
|
||||
Galera is a synchronous multi-master cluster
|
||||
for the MySQL database.
|
||||
Mirantis OpenStack uses MySQL/Galera for HA deployments;
|
||||
see the `FAQ <http://docs.mirantis.com/fuel/fuel-4.1/frequently-asked-questions.html#other-questions>`_
|
||||
see :ref:`mysql-galera-arch` in Reference Architecture
|
||||
for more information.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user