Merge "Add spec for ceph-rbd-mirror charm"
This commit is contained in:
commit
ffdcafda5e
|
@ -0,0 +1,188 @@
|
||||||
|
..
|
||||||
|
Copyright 2018 Canonical Ltd.
|
||||||
|
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0
|
||||||
|
Unported License.
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
..
|
||||||
|
This template should be in ReSTructured text. Please do not delete
|
||||||
|
any of the sections in this template. If you have nothing to say
|
||||||
|
for a whole section, just write: "None". For help with syntax, see
|
||||||
|
http://sphinx-doc.org/rest.html To test out your formatting, see
|
||||||
|
http://www.tele3.cz/jbar/rest/rest.html
|
||||||
|
|
||||||
|
=====================
|
||||||
|
Ceph RBD Mirror Charm
|
||||||
|
=====================
|
||||||
|
|
||||||
|
Problem Description
|
||||||
|
===================
|
||||||
|
|
||||||
|
RBD image mirroring can be used to provide a solution for Ceph cluster disaster
|
||||||
|
recovery. Ceph has a daemon called
|
||||||
|
`rbd-mirror <http://docs.ceph.com/docs/mimic/rbd/rbd-mirroring/>`__ which can
|
||||||
|
be placed on a primary and a backup cluster and provide asynchronous
|
||||||
|
replication of RBD images in a given pool.
|
||||||
|
|
||||||
|
rbd-mirror can work in two modes:
|
||||||
|
|
||||||
|
* pool (all images in a given pool are synchronized);
|
||||||
|
* image (per-image synchronization).
|
||||||
|
|
||||||
|
The scenario targeted by this spec involves an operator when it comes to
|
||||||
|
performing promote/demote actions and the DR procedure is operator-driven
|
||||||
|
as opposed to a full automatic failover in the event of a outage at the
|
||||||
|
primary site.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Promotion/demotion of pools will be operator driven
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
RADOS objects are not mirrored so for mirroring radosgw objects which
|
||||||
|
use RADOS objects or gnocchi metrics stored in RADOS objects different
|
||||||
|
backup mechanisms are required - this spec covers RBD images only.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
RBD mirroring relies on the use of the exclusive-lock and journaling
|
||||||
|
features of RBD; these are only supported in the userspace integration
|
||||||
|
libraries as used by libvirt and qemu for native KVM virtualization.
|
||||||
|
This requirement excludes the use of this feature with LXD based clouds
|
||||||
|
which disable the majority of RBD features for compatibility with the
|
||||||
|
Linux kernel RBD driver.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
The initial RBD mirror charm will only support mirroring of whole
|
||||||
|
pools.
|
||||||
|
|
||||||
|
Proposed Change
|
||||||
|
===============
|
||||||
|
|
||||||
|
High Level Design
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
As rbd-mirror is a separate package and the service itself acts as an RBD
|
||||||
|
client it makes sense to implement the target functionality in a separate
|
||||||
|
principle charm (ceph-rbd-mirror). The charm will accept parameters on which
|
||||||
|
pools to replicate and be able to be related to multiple ceph-mon applications
|
||||||
|
in separate clusters.
|
||||||
|
|
||||||
|
The charm will relate to a local ceph cluster and a remote ceph cluster
|
||||||
|
typically using a cross model relation.
|
||||||
|
|
||||||
|
A new interface type ('rbd-mirror') will be created to support this
|
||||||
|
integration; this will be provided by the ceph-mon charm, and consumed by
|
||||||
|
the new ceph-rbd-mirror charm for both local and remote cluster connections.
|
||||||
|
|
||||||
|
Each rbd-mirror daemon requires a key for connectivity to the local cluster
|
||||||
|
(named uniquely for the daemon) and a key for connectivity to the remote
|
||||||
|
cluster (named globally for all rbd-mirror daemons). Multiple ceph
|
||||||
|
configurations will also be maintained on the ceph-rbd-mirror units -
|
||||||
|
'ceph' to reference the local cluster and 'remote' to reference the
|
||||||
|
remote cluster. Configuration files and keys will be prefixed inline
|
||||||
|
with this naming - for example:
|
||||||
|
|
||||||
|
.. code::
|
||||||
|
|
||||||
|
$ ls /etc/ceph
|
||||||
|
ceph.conf
|
||||||
|
ceph.client.rbd-mirror.<hostname>.keyring
|
||||||
|
remote.conf
|
||||||
|
remote.client.rbd-mirror.keyring
|
||||||
|
|
||||||
|
In order to support resilience and scale-out of rbd mirroring, multiple
|
||||||
|
units of the charm may be deployed; as a result this feature will only
|
||||||
|
be supported with Ceph Luminous or later (which support multiple instances
|
||||||
|
of the rbd-mirror service).
|
||||||
|
|
||||||
|
Deployment and Scalability Considerations
|
||||||
|
-----------------------------------------
|
||||||
|
|
||||||
|
From the deployment perspective the charm units should have high-bandwidth
|
||||||
|
and low-latency L3 connectivity to access and replication networks of both
|
||||||
|
clusters to be able to keep up with the changes to Ceph pools it tries to
|
||||||
|
replicate. At minimum, static routes will need to be configured on the node
|
||||||
|
running rbd-mirror daemon but that is outside of the scope of this spec.
|
||||||
|
|
||||||
|
Multiple units of the ceph-rbd-mirror charm may be used to scale out
|
||||||
|
replication traffic.
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
|
||||||
|
No alternative solutions have been considered.
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
This feature relies on use of a Juju version which supports cross model
|
||||||
|
relations.
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
|
||||||
|
<tbd>
|
||||||
|
|
||||||
|
Gerrit Topic
|
||||||
|
------------
|
||||||
|
|
||||||
|
Use Gerrit topic "rbd-mirror" for all patches related to this spec.
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
git-review -t rbd-mirror
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
* Implement a new reactive charm called ceph-rbd-mirror.
|
||||||
|
* Implement the following relation:
|
||||||
|
* rbd-mirror - ceph-mon (local and cross model).
|
||||||
|
* Add "cluster" endpoint to extra-bindings in metadata.yaml to allow binding
|
||||||
|
the "cluster" endpoint to a ceph replication space.
|
||||||
|
* ceph-mon relations should retrieve cluster details and cephx keys via the
|
||||||
|
broker protocol implemented in Ceph charms (code reuse).
|
||||||
|
* Add config options to specify pool names for replication.
|
||||||
|
* Automate creation of pools on a backup cluster if they are not present.
|
||||||
|
* Add actions to promote and demote pools.
|
||||||
|
* Enable RBD journaling feature as documented in rbd-mirror docs.
|
||||||
|
* Write unit tests.
|
||||||
|
* Write functional tests via zaza framework.
|
||||||
|
|
||||||
|
Repositories
|
||||||
|
------------
|
||||||
|
|
||||||
|
A new git repository will be required for the ``ceph-rbd-mirror`` charm:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
git://git.openstack.org/openstack/charm-ceph-rbd-mirror
|
||||||
|
|
||||||
|
Documentation
|
||||||
|
-------------
|
||||||
|
|
||||||
|
The ``ceph-rbd-mirror`` charm should contain a README with instructions on
|
||||||
|
deploying the charm and on limitations related to scalability and networking.
|
||||||
|
|
||||||
|
Security
|
||||||
|
--------
|
||||||
|
|
||||||
|
- Users created for replication must not have admin privileges - they only
|
||||||
|
need to be able to write to the pools they require on the target cluster.
|
||||||
|
This is supported through the existing group based permissions system
|
||||||
|
in the ceph-mon broker using the 'rbd' profile for mon and osd permissions.
|
||||||
|
|
||||||
|
Testing
|
||||||
|
-------
|
||||||
|
|
||||||
|
Code written or changed will be covered by unit tests; functional testing will
|
||||||
|
be done using ``Zaza``.
|
Loading…
Reference in New Issue