8c5012e940
* Register Swift-compatible endpoints in Keystone * Load balance across RadosGW API servers using HAProxy The support is exercised in the cephadm CI jobs, but since RGW is not currently enabled via cephadm, it is not yet tested. https://docs.ceph.com/en/latest/radosgw/keystone/ Implements: blueprint ceph-rgw Change-Id: I891c3ed4ed93512607afe65a42dd99596fd4dbf9
285 lines
9.5 KiB
ReStructuredText
285 lines
9.5 KiB
ReStructuredText
.. _external-ceph-guide:
|
|
|
|
=============
|
|
External Ceph
|
|
=============
|
|
|
|
Kolla Ansible does not provide support for provisioning and configuring a
|
|
Ceph cluster directly. Instead, administrators should use a tool dedicated
|
|
to this purpose, such as:
|
|
|
|
* `ceph-ansible <https://docs.ceph.com/projects/ceph-ansible/en/latest/>`_
|
|
* `cephadm <https://docs.ceph.com/en/latest/cephadm/install/>`_
|
|
|
|
The desired pool(s) and keyrings should then be created via the Ceph CLI
|
|
or similar.
|
|
|
|
Requirements
|
|
~~~~~~~~~~~~
|
|
|
|
* An existing installation of Ceph
|
|
* Existing Ceph storage pools
|
|
* Existing credentials in Ceph for OpenStack services to connect to Ceph
|
|
(Glance, Cinder, Nova, Gnocchi, Manila)
|
|
|
|
Refer to https://docs.ceph.com/en/latest/rbd/rbd-openstack/ for details on
|
|
creating the pool and keyrings with appropriate permissions for each service.
|
|
|
|
Configuring External Ceph
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Ceph integration is configured for different OpenStack services independently.
|
|
|
|
Glance
|
|
------
|
|
|
|
Ceph RBD can be used as a storage backend for Glance images. Configuring Glance
|
|
for Ceph includes the following steps:
|
|
|
|
#. Enable Glance Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
glance_backend_ceph: "yes"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_glance_keyring`` (default: ``ceph.client.glance.keyring``)
|
|
* ``ceph_glance_user`` (default: ``glance``)
|
|
* ``ceph_glance_pool_name`` (default: ``images``)
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/glance/ceph.conf``
|
|
|
|
.. path /etc/kolla/config/glance/ceph.conf
|
|
.. code-block:: ini
|
|
|
|
[global]
|
|
fsid = 1d89fec3-325a-4963-a950-c4afedd37fe3
|
|
mon_initial_members = ceph-0
|
|
mon_host = 192.168.0.56
|
|
auth_cluster_required = cephx
|
|
auth_service_required = cephx
|
|
auth_client_required = cephx
|
|
|
|
#. Copy Ceph keyring to ``/etc/kolla/config/glance/<ceph_glance_keyring>``
|
|
|
|
Cinder
|
|
------
|
|
|
|
Ceph RBD can be used as a storage backend for Cinder volumes. Configuring
|
|
Cinder for Ceph includes following steps:
|
|
|
|
#. When using external Ceph, there may be no nodes defined in the storage
|
|
group. This will cause Cinder and related services relying on this group to
|
|
fail. In this case, operator should add some nodes to the storage group,
|
|
all the nodes where ``cinder-volume`` and ``cinder-backup`` will run:
|
|
|
|
.. code-block:: ini
|
|
|
|
[storage]
|
|
control01
|
|
|
|
#. Enable Cinder Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
cinder_backend_ceph: "yes"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_cinder_keyring`` (default: ``ceph.client.cinder.keyring``)
|
|
* ``ceph_cinder_user`` (default: ``cinder``)
|
|
* ``ceph_cinder_pool_name`` (default: ``volumes``)
|
|
* ``ceph_cinder_backup_keyring``
|
|
(default: ``ceph.client.cinder-backup.keyring``)
|
|
* ``ceph_cinder_backup_user`` (default: ``cinder-backup``)
|
|
* ``ceph_cinder_backup_pool_name`` (default: ``backups``)
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/cinder/ceph.conf``
|
|
|
|
Separate configuration options can be configured for
|
|
cinder-volume and cinder-backup by adding ceph.conf files to
|
|
``/etc/kolla/config/cinder/cinder-volume`` and
|
|
``/etc/kolla/config/cinder/cinder-backup`` respectively. They
|
|
will be merged with ``/etc/kolla/config/cinder/ceph.conf``.
|
|
|
|
#. Copy Ceph keyring files to:
|
|
|
|
* ``/etc/kolla/config/cinder/cinder-volume/<ceph_cinder_keyring>``
|
|
* ``/etc/kolla/config/cinder/cinder-backup/<ceph_cinder_keyring>``
|
|
* ``/etc/kolla/config/cinder/cinder-backup/<ceph_cinder_backup_keyring>``
|
|
|
|
.. note::
|
|
|
|
``cinder-backup`` requires two keyrings for accessing volumes
|
|
and backup pool.
|
|
|
|
Nova must also be configured to allow access to Cinder volumes:
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_cinder_keyring`` (default: ``ceph.client.cinder.keyring``)
|
|
|
|
#. Copy Ceph keyring file(s) to:
|
|
|
|
* ``/etc/kolla/config/nova/<ceph_cinder_keyring>``
|
|
|
|
Nova
|
|
----
|
|
|
|
Ceph RBD can be used as a storage backend for Nova instance ephemeral disks.
|
|
This avoids the requirement for local storage for instances on compute nodes.
|
|
It improves the performance of migration, since instances' ephemeral disks do
|
|
not need to be copied between hypervisors.
|
|
|
|
Configuring Nova for Ceph includes following steps:
|
|
|
|
#. Enable Nova Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
nova_backend_ceph: "yes"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_nova_keyring`` (by default it's the same as
|
|
``ceph_cinder_keyring``)
|
|
* ``ceph_nova_user`` (by default it's the same as ``ceph_cinder_user``)
|
|
* ``ceph_nova_pool_name`` (default: ``vms``)
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/nova/ceph.conf``
|
|
#. Copy Ceph keyring file(s) to:
|
|
|
|
* ``/etc/kolla/config/nova/<ceph_nova_keyring>``
|
|
|
|
.. note::
|
|
|
|
If you are using a Ceph deployment tool that generates separate Ceph
|
|
keys for Cinder and Nova, you will need to override
|
|
``ceph_nova_keyring`` and ``ceph_nova_user`` to match.
|
|
|
|
Gnocchi
|
|
-------
|
|
|
|
Ceph object storage can be used as a storage backend for Gnocchi metrics.
|
|
Configuring Gnocchi for Ceph includes following steps:
|
|
|
|
#. Enable Gnocchi Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
gnocchi_backend_storage: "ceph"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_gnocchi_keyring``
|
|
(default: ``ceph.client.gnocchi.keyring``)
|
|
* ``ceph_gnocchi_user`` (default: ``gnocchi``)
|
|
* ``ceph_gnocchi_pool_name`` (default: ``gnocchi``)
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/gnocchi/ceph.conf``
|
|
#. Copy Ceph keyring to ``/etc/kolla/config/gnocchi/<ceph_gnocchi_keyring>``
|
|
|
|
Manila
|
|
------
|
|
|
|
CephFS can be used as a storage backend for Manila shares. Configuring Manila
|
|
for Ceph includes following steps:
|
|
|
|
#. Enable Manila Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
enable_manila_backend_cephfs_native: "yes"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_manila_keyring`` (default: ``ceph.client.manila.keyring``)
|
|
* ``ceph_manila_user`` (default: ``manila``)
|
|
|
|
.. note::
|
|
|
|
Required Ceph identity caps for manila user are documented in
|
|
:manila-doc:`CephFS Native driver <admin/cephfs_driver.html#authorizing-the-driver-to-communicate-with-ceph>`.
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/manila/ceph.conf``
|
|
#. Copy Ceph keyring to ``/etc/kolla/config/manila/<ceph_manila_keyring>``
|
|
#. Setup Manila in the usual way
|
|
|
|
For more details on the rest of the Manila setup, such as creating the share
|
|
type ``default_share_type``, please see :doc:`Manila in Kolla <manila-guide>`.
|
|
|
|
For more details on the CephFS Native driver, please see
|
|
:manila-doc:`CephFS Native driver <admin/cephfs_driver.html>`.
|
|
|
|
RadosGW
|
|
-------
|
|
|
|
As of the Xena 13.0.0 release, Kolla Ansible supports integration with Ceph
|
|
RadosGW. This includes:
|
|
|
|
* Registration of Swift-compatible endpoints in Keystone
|
|
* Load balancing across RadosGW API servers using HAProxy
|
|
|
|
See the `Ceph documentation
|
|
<https://docs.ceph.com/en/latest/radosgw/keystone/>`__ for further information,
|
|
including changes that must be applied to the Ceph cluster configuration.
|
|
|
|
Enable Ceph RadosGW integration:
|
|
|
|
.. code-block:: yaml
|
|
|
|
enable_ceph_rgw: true
|
|
|
|
Keystone integration
|
|
====================
|
|
|
|
A Keystone user and endpoints are registered by default, however this may be
|
|
avoided by setting ``enable_ceph_rgw_keystone`` to ``false``. If registration
|
|
is enabled, the username is defined via ``ceph_rgw_keystone_user``, and this
|
|
defaults to ``ceph_rgw``. The hostnames used by the endpoints default to
|
|
``ceph_rgw_external_fqdn`` and ``ceph_rgw_internal_fqdn`` for the public and
|
|
internal endpoints respectively. These default to ``kolla_external_fqdn`` and
|
|
``kolla_internal_fqdn`` respectively. The port used by the endpoints is defined
|
|
via ``ceph_rgw_port``, and defaults to 6780.
|
|
|
|
By default RadosGW supports both Swift and S3 API, and it is not completely
|
|
compatible with Swift API. The option ``ceph_rgw_swift_compatibility`` can
|
|
enable/disable complete RadosGW compatibility with Swift API. This should
|
|
match the configuration used by Ceph RadosGW. After changing the value, run
|
|
the ``kolla-ansible deploy`` command to enable.
|
|
|
|
By default, the RadosGW endpoint URL does not include the project (account) ID.
|
|
This prevents cross-project and public object access. This can be resolved by
|
|
setting ``ceph_rgw_swift_account_in_url`` to ``true``. This should match the
|
|
``rgw_swift_account_in_url`` configuration option in Ceph RadosGW.
|
|
|
|
Load balancing
|
|
==============
|
|
|
|
.. warning::
|
|
|
|
Users of Ceph RadosGW can generate very high volumes of traffic. It is
|
|
advisable to use a separate load balancer for RadosGW for anything other
|
|
than small or lightly utilised RadosGW deployments, however this is
|
|
currently out of scope for Kolla Ansible.
|
|
|
|
Load balancing is enabled by default, however this may be avoided by setting
|
|
``enable_ceph_rgw_loadbalancer`` to ``false``. If using load balancing, the
|
|
RadosGW hosts and ports must be configured. Each item should contain
|
|
``host`` and ``port`` keys. The ``ip`` and ``port`` keys are optional. If
|
|
``ip`` is not specified, the ``host`` values should be resolvable from the host
|
|
running HAProxy. If the ``port`` is not specified, the default HTTP (80) or
|
|
HTTPS (443) port will be used. For example:
|
|
|
|
.. code-block:: yaml
|
|
|
|
ceph_rgw_hosts:
|
|
- host: rgw-host-1
|
|
- host: rgw-host-2
|
|
ip: 10.0.0.42
|
|
port: 8080
|
|
|
|
The HAProxy frontend port is defined via ``ceph_rgw_port``, and defaults to
|
|
6780.
|