Update Pike storage documentation for ceph-ansible

The user is required to install ceph-ansible in pike. The package can not be made a dependency of tripleoclient [1] Updated install, update/upgrade, and backup and restore The installation section was updated to cover deployment of Ceph via TripleO with either puppet-ceph (Ceph not in containers) or ceph-ansible (Ceph in containers). [1] http://eavesdrop.openstack.org/meetings/rdo_meeting___2017_06_21/2017/rdo_meeting___2017_06_21.2017-06-21-15.01.log.html#l-29 Co-Authored-By: John Fulton <fulton@redhat.com> Change-Id: I8e9def9577dc84b63895e10924a01a4949d02aef
2017-07-25 13:49:29 -04:00 · 2017-07-25 13:49:29 -04:00 · 69b6c94dba
parent 2a5073ab3c
commit 69b6c94dba
5 changed files with 184 additions and 9 deletions
--- a/doc/source/install/advanced_deployment/ceph_config.rst
+++ b/doc/source/install/advanced_deployment/ceph_config.rst
@ -1,11 +1,62 @@
 Configuring Ceph with Custom Config Settings
 ============================================

-This guide assumes that your undercloud is already installed and ready to
-deploy an overcloud.
+This guide assumes that the undercloud is already installed and ready
+to deploy an overcloud and that the appropriate repositories
+containing Ceph packages, including ceph-ansible if applicable, have
+been enabled and installed as described in
+:doc:`../installation/installing`.

-Customizing ceph.conf
---------------------
+Deploying an Overcloud with Ceph
+--------------------------------
+
+TripleO can deploy and configure Ceph as if it was a composable
+OpenStack service and configure OpenStack services like Nova, Glance,
+Cinder, Cinder Backup, and Gnocchi to use it as a storage backend.
+
+Prior to Pike, TripleO deployed Ceph with `puppet-ceph`_. With the
+Pike release it is possible to use TripleO to deploy Ceph with
+either `ceph-ansible`_ or puppet-ceph, though puppet-ceph is
+deprecated. To deploy Ceph in containers use `ceph-ansible`, for which
+only a containerized Ceph deployment is possible. It is not possible
+to deploy a containerized Ceph with `puppet-ceph`.
+
+To deploy with Ceph include either of the appropriate environment
+files. For puppet-ceph use "environments/puppet-ceph.yaml"
+like the following::
+
+    openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-ceph.yaml
+
+For ceph-ansible use "environments/ceph-ansible/ceph-ansible.yaml"
+like the following::
+
+    openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml
+
+In both the puppet-ceph and ceph-ansible examples above, at least one
+Ceph storage node is required. The following example will configure
+one Ceph storage nodes on servers matching the `ceph-storage`
+profile. It will also set the default pool size, the number of times
+that an object should be written for data protection, to one. These
+`parameter_defaults` may be saved in an environment file
+"~/my-ceph-settings.yaml" and added to the deploy commandline::
+
+  parameter_defaults:
+    OvercloudCephStorageFlavor: ceph-storage
+    CephStorageCount: 1
+    CephDefaultPoolSize: 1
+
+The values above are only appropriate for a development or POC
+deployment. The default pool size is three but if there are less
+than three Ceph OSDs, then the cluster will never reach status
+`HEALTH_OK` because it has no place to make additional copies.
+Thus, a POC deployment with less than three OSDs should override the
+default default pool size. However, a production deployment should
+replace both of the ones above with threes, or greater, in order to
+have at least three storage nodes and at least three back up copies of
+each object at minimum.
+
+Customizing ceph.conf with puppet-ceph
+--------------------------------------

 Ceph demands for more careful configuration when deployed at scale.

@ -19,10 +70,6 @@ It is possible to override any of the configuration parameters supported by
 will customize the default `osd_journal_size` overriding any default
 provided in the `ceph.yaml static hieradata`_.

-Remember to add such an environment file to the deploy commandline::
-
-    openstack overcloud deploy --templates --ceph-storage-scale <number of nodes> -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e ~/my-ceph-settings.yaml
-
 It is also possible to provide arbitrary stanza/key/value lines for `ceph.conf`
 using the special `ceph::conf` configuration class. For example by using::

@ -40,10 +87,95 @@ the resulting `ceph.conf` file should be populated with the following::
  max_open_files: 131072
  my_setting: my_value

+To specify a set of dedicated block devices to use as Ceph OSDs use
+the following::
+
+  parameter_defaults:
+    ExtraConfig:
+      ceph::profile::params::osds:
+        '/dev/sdb':
+          journal: '/dev/sde'
+        '/dev/sdc':
+          journal: '/dev/sde'
+        '/dev/sdd':
+          journal: '/dev/sde'
+
+The above will produce three OSDs which run on `/dev/sdb`, `/dev/sdc`,
+and `/dev/sdd` which all journal to `/dev/sde`. This same setup will
+be duplicated per Ceph storage node and assumes uniform hardware.
+
+The `parameter_defaults` like the above may be saved in an environment
+file "~/my-ceph-settings.yaml" and added to the deploy commandline::
+
+    openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-ceph.yaml -e ~/my-ceph-settings.yaml
+
+Customizing ceph.conf with ceph-ansible
+---------------------------------------
+
+The playbooks provided by `ceph-ansible` are triggered by Mistral
+workflow. A new `CephAnsibleExtraConfig` parameter has been added to
+the templates and can be used to provide arbitrary config variables
+consumed by `ceph-ansible`. The pre-existing template params consumed
+by the TripleO Pike release to drive `puppet-ceph` continue to work
+and are translated, when possible, into their equivalent
+`ceph-ansible` variable.
+
+Global settings in the `ceph.conf` may be set using
+`ceph_conf_overrides` like the following::
+
+  CephAnsibleExtraConfig:
+    ceph_conf_overrides:
+      global:
+        journal_size: 2048
+        max_open_files: 131072
+        osd_pool_default_size: 3
+        osd_pool_default_pg_num: 256
+
+`CephAnsibleExtraConfig` isn't just for `ceph.conf` overrides. For
+example, to encrypt the data stored on OSDs use the following::
+
+  CephAnsibleExtraConfig:
+    dmcrypt: true
+
+The above overrides the defaults found in the
+`ceph-ansible/group_vars`_.
+
+To specify a set of dedicated block devices to use as Ceph OSDs use
+a variation of the following::
+
+  parameter_defaults:
+    CephAnsibleDisksConfig:
+      devices:
+        - /dev/sdb
+        - /dev/sdc
+        - /dev/sdd
+      raw_journal_devices:
+        - /dev/sde
+        - /dev/sde
+        - /dev/sde
+      journal_collocation: false
+      raw_multi_journal: true
+
+The above will produce three OSDs which run on `/dev/sdb`, `/dev/sdc`,
+and `/dev/sdd` which all journal to `/dev/sde`. This same setup will
+be duplicated per Ceph storage node and assumes uniform hardware.
+
+The `parameter_defaults` like the above may be saved in an environment
+file "~/my-ceph-settings.yaml" and added to the deploy commandline::
+
+    openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml -e ~/my-ceph-settings.yaml
+
 .. note::

    The :doc:`extra_config` doc has a more details on the usage of the different
    ExtraConfig interfaces.

+.. note::
+
+    Deployment with `ceph-ansible` requires that OSDs run on dedicated
+    block devices.
+
 .. _`puppet-ceph`: https://github.com/openstack/puppet-ceph
+.. _`ceph-ansible`: https://github.com/ceph/ceph-ansible
 .. _`ceph.yaml static hieradata`: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/hieradata/ceph.yaml
+.. _`ceph-ansible/group_vars`: https://github.com/ceph/ceph-ansible/tree/master/group_vars
--- a/doc/source/install/installation/installing.rst
+++ b/doc/source/install/installation/installing.rst
@ -69,6 +69,13 @@ Installing the Undercloud

    sudo yum install -y python-tripleoclient

+   .. admonition:: Ceph
+      :class: ceph
+
+      If you intend to deploy Ceph in the overcloud and are running Pike or newer, then install ceph-ansible on the undercloud::
+
+          sudo yum install -y ceph-ansible
+
 #. Copy in the sample configuration file and edit it to reflect your environment::

    cp /usr/share/instack-undercloud/undercloud.conf.sample ~/undercloud.conf
--- a/doc/source/install/installation/updating.rst
+++ b/doc/source/install/installation/updating.rst
@ -31,6 +31,14 @@ You can upgrade any packages that are installed on the undercloud machine.

    sudo yum -y update python-tripleoclient

+   .. admonition:: Ceph
+      :class: ceph
+
+      If you are using Pike or newer and Ceph was deployed in the
+      overcloud, update ceph-ansible on the undercloud::
+
+          sudo yum -y update ceph-ansible
+
 #. Run the undercloud upgrade command. This command will upgrade all packages
   and use puppet to apply new configuration and restart all OpenStack
   services.::
--- a/doc/source/install/post_deployment/backup_restore_undercloud.rst
+++ b/doc/source/install/post_deployment/backup_restore_undercloud.rst
@ -66,6 +66,12 @@ Finally we rerun the undercloud installation from the stack user, making sure to
  # Double check hostname is correctly set in /etc/hosts
  openstack install undercloud

+If you are using Pike and Ceph will be used in the overcloud, install
+ceph-ansible on the undercloud::
+
+  sudo yum install -y ceph-ansible
+
+
 Reconnect the restored undercloud to the overcloud
 --------------------------------------------------
 Having completed the steps above, the undercloud can be expected to automatically
--- a/doc/source/install/post_deployment/upgrade.rst
+++ b/doc/source/install/post_deployment/upgrade.rst
@ -86,12 +86,29 @@ Upgrading the Undercloud
       you are using a later newton release you don't need to explicitly enable
       this option.

+   .. admonition:: Ocata to Pike
+      :class: mton
+
+       Prior to Pike, TripleO deployed Ceph with puppet-ceph. With the
+       Pike release it is possible to use TripleO to deploy Ceph with
+       either ceph-ansible or puppet-ceph, though puppet-ceph is
+       deprecated. To use ceph-ansible, the CentOS Storage SIG Ceph
+       repository must be enabled on the undercloud and the
+       ceph-ansible package must then be installed::
+
+          sudo yum -y install --enablerepo=extras centos-release-ceph-jewel
+          sudo yum -y install ceph-ansible
+
+       It is not yet possible to migrate an existing puppet-ceph
+       deployment to a ceph-ansible deployment. Only new deployments
+       are currently possible with ceph-ansible.
+
   The following command will upgrade the undercloud::

      sudo systemctl stop openstack-*
      sudo systemctl stop neutron-*
      sudo systemctl stop httpd
-      sudo yum -y update instack-undercloud openstack-puppet-modules openstack-tripleo-common python-tripleoclient
+      sudo yum -y update instack-undercloud openstack-puppet-modules openstack-tripleo-common python-tripleoclient ceph-ansible
      openstack undercloud upgrade

   Once the undercloud upgrade is fully completed you may
@ -111,6 +128,11 @@ Upgrading the Undercloud
 .. _validations: ../validations/validations.html#running-a-group-of-validations
 .. _tripleo-validations: https://github.com/openstack/tripleo-validations/tree/master/validations

+.. note::
+
+   It is not necessary to update ceph-ansible if Ceph is not used in
+   the overcloud.
+
 Upgrading the Overcloud to Ocata and beyond
 -------------------------------------------