Merge "Document "deployed ceph""
This commit is contained in:
commit
2d1a4a1c43
deploy-guide/source
@ -95,6 +95,12 @@ that it can run all the steps:
|
|||||||
As an external deploy step the neutron ports for Service Virtual IPs are
|
As an external deploy step the neutron ports for Service Virtual IPs are
|
||||||
created, and the properties of the Virtual IPs are included in hieradata.
|
created, and the properties of the Virtual IPs are included in hieradata.
|
||||||
|
|
||||||
|
.. admonition:: Ceph
|
||||||
|
:class: ceph
|
||||||
|
|
||||||
|
Optionally Ceph may be deployed after the baremetal instances
|
||||||
|
are provisioned but before the ephemeral Heat stack is created
|
||||||
|
as described in :doc:`../features/deployed_ceph`.
|
||||||
|
|
||||||
Using
|
Using
|
||||||
-----
|
-----
|
||||||
|
@ -11,6 +11,7 @@ OpenStack projects.
|
|||||||
cinder_custom_backend
|
cinder_custom_backend
|
||||||
cinder_netapp
|
cinder_netapp
|
||||||
cephadm
|
cephadm
|
||||||
|
deployed_ceph
|
||||||
ceph_config
|
ceph_config
|
||||||
ceph_external
|
ceph_external
|
||||||
domain_specific_ldap_backends
|
domain_specific_ldap_backends
|
||||||
|
@ -15,13 +15,9 @@ Limitations
|
|||||||
-----------
|
-----------
|
||||||
|
|
||||||
TripleO deployments of Ceph with cephadm_ are only supported in Wallaby
|
TripleO deployments of Ceph with cephadm_ are only supported in Wallaby
|
||||||
or newer and are currently limited to only Ceph RBD and RGW
|
or newer. The default version of Ceph deployed by TripleO in Wallaby
|
||||||
services. Support for deployment of CephFS with Manila and Ceph
|
is Pacific, regardless of if cephadm or ceph-ansible is used to deploy
|
||||||
Dashboard via TripleO's cephadm integration are not yet available in
|
it.
|
||||||
Wallaby though these services may be deployed with ceph-ansible as
|
|
||||||
described in the :doc:`ceph_config` documentation. The default version
|
|
||||||
of Ceph deployed by TripleO in Wallaby is Pacific, regardless of if
|
|
||||||
cephadm or ceph-ansible is used to deploy it.
|
|
||||||
|
|
||||||
TripleO can only deploy one Ceph cluster in the overcloud per Heat
|
TripleO can only deploy one Ceph cluster in the overcloud per Heat
|
||||||
stack. However, within that Heat stack it's possible to configure
|
stack. However, within that Heat stack it's possible to configure
|
||||||
@ -71,8 +67,8 @@ for some target nodes, see `cleaning instructions in the Ironic documentation`_.
|
|||||||
Deploying Ceph During Overcloud Deployment
|
Deploying Ceph During Overcloud Deployment
|
||||||
------------------------------------------
|
------------------------------------------
|
||||||
|
|
||||||
To deploy an overcloud with a Ceph RBD and RGW server include the
|
To deploy an overcloud with a Ceph include the appropriate environment
|
||||||
appropriate environment file as in the example below::
|
file as in the example below::
|
||||||
|
|
||||||
openstack overcloud deploy --templates \
|
openstack overcloud deploy --templates \
|
||||||
-e /usr/share/openstack-tripleo-heat-templates/environments/cephadm/cephadm.yaml
|
-e /usr/share/openstack-tripleo-heat-templates/environments/cephadm/cephadm.yaml
|
||||||
@ -93,12 +89,23 @@ deploy like this::
|
|||||||
-e /usr/share/openstack-tripleo-heat-templates/environments/cephadm/cephadm.yaml \
|
-e /usr/share/openstack-tripleo-heat-templates/environments/cephadm/cephadm.yaml \
|
||||||
-e cephadm-overrides.yaml
|
-e cephadm-overrides.yaml
|
||||||
|
|
||||||
Deploying with the commands above will result in the process described
|
Deploying with the commands above will result in the processes described
|
||||||
in the next section.
|
in the rest of this document.
|
||||||
|
|
||||||
|
Deploying Ceph Before Overcloud Deployment
|
||||||
|
------------------------------------------
|
||||||
|
|
||||||
|
In Wallaby and newer it is possible to provision hardware and deploy
|
||||||
|
Ceph before deploying the overcloud on the same hardware. This feature
|
||||||
|
is called "deployed ceph" and it uses the command `openstack overcloud
|
||||||
|
ceph deploy` which executes the same Ansible roles described
|
||||||
|
below. For more details see :doc:`deployed_ceph`.
|
||||||
|
|
||||||
Overview of Ceph Deployment with TripleO and cephadm
|
Overview of Ceph Deployment with TripleO and cephadm
|
||||||
----------------------------------------------------
|
----------------------------------------------------
|
||||||
|
|
||||||
|
When Ceph is deployed during overcloud configuration or when Ceph is
|
||||||
|
deployed before overcloud configuration with :doc:`deployed_ceph`,
|
||||||
TripleO will use Ansible automate the process described in the
|
TripleO will use Ansible automate the process described in the
|
||||||
`cephadm`_ documentation to bootstrap a new cluster. It will
|
`cephadm`_ documentation to bootstrap a new cluster. It will
|
||||||
bootstrap a single Ceph monitor and manager on one server
|
bootstrap a single Ceph monitor and manager on one server
|
||||||
@ -132,13 +139,22 @@ e.g. which servers run which services according to composable
|
|||||||
roles, will be converted by the tripleo-ansible `ceph_spec_bootstrap`_
|
roles, will be converted by the tripleo-ansible `ceph_spec_bootstrap`_
|
||||||
module into a `Ceph Service Specification`_ file. The module has the
|
module into a `Ceph Service Specification`_ file. The module has the
|
||||||
ability to do this based on the Ansible inventory generated by the
|
ability to do this based on the Ansible inventory generated by the
|
||||||
`tripleo-ansible-inventory` but it can also generate the Ceph Service
|
`tripleo-ansible-inventory`. When Ceph is deployed *during* overcloud
|
||||||
Specification file from a combination of a TripleO roles data file
|
configuration by including the cephadm.yaml environment file, the
|
||||||
|
module uses the Ansible inventory to create the `Ceph Service
|
||||||
|
Specification`_. In this scenario the default location of the
|
||||||
|
generated Ceph Service Specification file is
|
||||||
|
`config-download/<STACK>/cephadm/ceph_spec.yaml`.
|
||||||
|
|
||||||
|
The same `ceph_spec_bootstrap`_ module can also generate the Ceph
|
||||||
|
Service Specification file from a combination of a TripleO roles data
|
||||||
|
file
|
||||||
(e.g. /usr/share/openstack-tripleo-heat-templates/roles_data.yaml)
|
(e.g. /usr/share/openstack-tripleo-heat-templates/roles_data.yaml)
|
||||||
and the output of the command
|
and the output of the command
|
||||||
`openstack overcloud node provision --output deployed_metal.yaml`.
|
`openstack overcloud node provision --output deployed_metal.yaml`.
|
||||||
The default location of the generated Ceph Service Specification
|
When Ceph is deployed *before* overcloud configuration as described in
|
||||||
file is in `config-download/<STACK>/cephadm/ceph_spec.yaml`.
|
:doc:`deployed_ceph`, the module uses the deployed_metal.yaml and
|
||||||
|
roles_data.yaml to create the `Ceph Service Specification`_.
|
||||||
|
|
||||||
After the `ceph-admin` user is created, `ceph_spec.yaml` is copied
|
After the `ceph-admin` user is created, `ceph_spec.yaml` is copied
|
||||||
to the bootstrap host. The bootstrap host will be the first host
|
to the bootstrap host. The bootstrap host will be the first host
|
||||||
@ -152,14 +168,16 @@ the bootstrap node and then run `ceph orch apply -i ceph_spec.yaml`
|
|||||||
and `cephadm` will use the `ceph-admin` account and SSH keys to add
|
and `cephadm` will use the `ceph-admin` account and SSH keys to add
|
||||||
the other nodes.
|
the other nodes.
|
||||||
|
|
||||||
After the full Ceph cluster is running the Ceph pools and the cephx
|
After the full Ceph cluster is running, either as a result of
|
||||||
keys to access the pools will be created as defined or overridden as
|
:doc:`deployed_ceph` or by cephadm being triggered during the
|
||||||
described in the Heat environment examples below. The information
|
overcloud deployment via the `cephadm.yaml` environment file, the
|
||||||
necessary to configure Ceph clients will then be extracted to
|
Ceph pools and the cephx keys to access the pools will be created as
|
||||||
`/home/stack/ceph_client.yml` on the undercloud and passed to the
|
defined or overridden as described in the Heat environment examples
|
||||||
as input to the tripleo-ansible role tripleo_ceph_client which will
|
below. The information necessary to configure Ceph clients will then
|
||||||
then configure the rest of the overcloud to use the new Ceph cluster
|
be extracted to `/home/stack/ceph_client.yml` on the undercloud and
|
||||||
as described in the :doc:`ceph_external` documentation.
|
passed to the as input to the tripleo-ansible role tripleo_ceph_client
|
||||||
|
which will then configure the rest of the overcloud to use the new
|
||||||
|
Ceph cluster as described in the :doc:`ceph_external` documentation.
|
||||||
|
|
||||||
When `openstack overcloud deploy` is re-run in order to update
|
When `openstack overcloud deploy` is re-run in order to update
|
||||||
the stack, the cephadm bootstrap process is not repeated because
|
the stack, the cephadm bootstrap process is not repeated because
|
||||||
@ -199,6 +217,8 @@ monitor. The supported configuration groups are 'global', 'mon',
|
|||||||
'mgr', 'osd', 'mds', and 'client'. If no group is provided, then the
|
'mgr', 'osd', 'mds', and 'client'. If no group is provided, then the
|
||||||
default configuration group is 'global'.
|
default configuration group is 'global'.
|
||||||
|
|
||||||
|
The above does not apply to :doc:`deployed_ceph`.
|
||||||
|
|
||||||
Overriding Server Configuration after deployment
|
Overriding Server Configuration after deployment
|
||||||
------------------------------------------------
|
------------------------------------------------
|
||||||
|
|
||||||
@ -262,6 +282,11 @@ If `CephDynamicSpec` is true and `CephSpecPath` is set to a valid
|
|||||||
path, then the spec will be created at that path before it is used to
|
path, then the spec will be created at that path before it is used to
|
||||||
deploy Ceph.
|
deploy Ceph.
|
||||||
|
|
||||||
|
The `CephDynamicSpec` and `CephSpecPath` parameters are not available
|
||||||
|
when using "deployed ceph", but the functionality is available via
|
||||||
|
the `--ceph-spec` command line option as described in
|
||||||
|
:doc:`deployed_ceph`.
|
||||||
|
|
||||||
Overriding which disks should be OSDs
|
Overriding which disks should be OSDs
|
||||||
-------------------------------------
|
-------------------------------------
|
||||||
|
|
||||||
@ -307,8 +332,12 @@ available but with the new syntax. When using this option consider
|
|||||||
setting `CephDynamicSpec` to false and defining a custom specification
|
setting `CephDynamicSpec` to false and defining a custom specification
|
||||||
which is passed to TripleO by setting the `CephSpecPath`.
|
which is passed to TripleO by setting the `CephSpecPath`.
|
||||||
|
|
||||||
Overriding Ceph placement group values during deployment
|
The `CephOsdSpec` parameter is not available when using "deployed
|
||||||
--------------------------------------------------------
|
ceph", but the same functionality is available via `--osd-spec`
|
||||||
|
command line option as described in :doc:`deployed_ceph`.
|
||||||
|
|
||||||
|
Overriding Ceph Pools and Placement Group values during deployment
|
||||||
|
------------------------------------------------------------------
|
||||||
|
|
||||||
The default cephadm deployment as triggered by TripleO has
|
The default cephadm deployment as triggered by TripleO has
|
||||||
`Autoscaling Placement Groups`_ enabled. Thus, it is not necessary to
|
`Autoscaling Placement Groups`_ enabled. Thus, it is not necessary to
|
||||||
@ -335,6 +364,12 @@ the following::
|
|||||||
- {"name": vms, "pg_num": 512, "pgp_num": 512, "application": rbd}
|
- {"name": vms, "pg_num": 512, "pgp_num": 512, "application": rbd}
|
||||||
- {"name": images, "pg_num": 128, "pgp_num": 128, "application": rbd}
|
- {"name": images, "pg_num": 128, "pgp_num": 128, "application": rbd}
|
||||||
|
|
||||||
|
|
||||||
|
Regardless of if the :doc:`deployed_ceph` feature is used, pools will
|
||||||
|
always be created during overcloud deployment as documented above.
|
||||||
|
Additional pools may also be created directly via the Ceph command
|
||||||
|
line tools.
|
||||||
|
|
||||||
Overriding CRUSH rules
|
Overriding CRUSH rules
|
||||||
----------------------
|
----------------------
|
||||||
|
|
||||||
@ -359,6 +394,10 @@ parameter::
|
|||||||
- {'name': 'slow_pool', 'rule_name': 'HDD', 'application': 'rbd'}
|
- {'name': 'slow_pool', 'rule_name': 'HDD', 'application': 'rbd'}
|
||||||
- {'name': 'fast_pool', 'rule_name': 'SSD', 'application': 'rbd'}
|
- {'name': 'fast_pool', 'rule_name': 'SSD', 'application': 'rbd'}
|
||||||
|
|
||||||
|
Regardless of if the :doc:`deployed_ceph` feature is used, custom
|
||||||
|
CRUSH rules may be created during overcloud deployment as documented
|
||||||
|
above. CRUSH rules may also be created directly via the Ceph command
|
||||||
|
line tools.
|
||||||
|
|
||||||
Overriding CephX Keys
|
Overriding CephX Keys
|
||||||
---------------------
|
---------------------
|
||||||
@ -420,6 +459,11 @@ pool used by Glance. The default Glance deployment defined in the Heat
|
|||||||
stack will continue to use the `ceph.client.openstack.keyring` file
|
stack will continue to use the `ceph.client.openstack.keyring` file
|
||||||
unless that Glance configuration itself is overridden.
|
unless that Glance configuration itself is overridden.
|
||||||
|
|
||||||
|
Regardless of if the :doc:`deployed_ceph` feature is used, CephX keys
|
||||||
|
may be created during overcloud deployment as documented above.
|
||||||
|
Additional CephX keys may also be created directly via the Ceph
|
||||||
|
command line tools.
|
||||||
|
|
||||||
Enabling cephadm debug mode
|
Enabling cephadm debug mode
|
||||||
---------------------------
|
---------------------------
|
||||||
|
|
||||||
@ -512,7 +556,10 @@ which is passed to the following command::
|
|||||||
--output ~/overcloud-baremetal-deployed.yaml \
|
--output ~/overcloud-baremetal-deployed.yaml \
|
||||||
~/overcloud_baremetal_deploy.yaml
|
~/overcloud_baremetal_deploy.yaml
|
||||||
|
|
||||||
As described in :doc:`../provisioning/baremetal_provision`, pass
|
If desired at this stage, then Ceph may be deployed early as described
|
||||||
|
in :doc:`deployed_ceph`. Otherwise Ceph may be deployed during the
|
||||||
|
overcloud deployment. Either way, as described in
|
||||||
|
:doc:`../provisioning/baremetal_provision`, pass
|
||||||
~/overcloud_baremetal_deploy.yaml as input, along with
|
~/overcloud_baremetal_deploy.yaml as input, along with
|
||||||
/usr/share/openstack-tripleo-heat-templates/environments/cephadm/cephadm.yaml
|
/usr/share/openstack-tripleo-heat-templates/environments/cephadm/cephadm.yaml
|
||||||
and cephadm-overrides.yaml described above, to the `openstack overcloud
|
and cephadm-overrides.yaml described above, to the `openstack overcloud
|
||||||
@ -772,6 +819,7 @@ example OSD, use commands like this::
|
|||||||
10
|
10
|
||||||
[ceph: root@oc0-controller-0 /]#
|
[ceph: root@oc0-controller-0 /]#
|
||||||
|
|
||||||
|
The above example assumes that :doc:`deployed_ceph` is not used.
|
||||||
|
|
||||||
Add the Ceph Dashboard to a Overcloud deployment
|
Add the Ceph Dashboard to a Overcloud deployment
|
||||||
------------------------------------------------
|
------------------------------------------------
|
||||||
|
356
deploy-guide/source/features/deployed_ceph.rst
Normal file
356
deploy-guide/source/features/deployed_ceph.rst
Normal file
@ -0,0 +1,356 @@
|
|||||||
|
Deployed Ceph
|
||||||
|
=============
|
||||||
|
|
||||||
|
In Wallaby and newer it is possible to provision hardware and deploy
|
||||||
|
Ceph before deploying the overcloud on the same hardware.
|
||||||
|
|
||||||
|
Deployed Ceph Workflow
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
As described in the :doc:`../deployment/network_v2` the ``overcloud
|
||||||
|
deploy`` command was extended so that it can run all of the following
|
||||||
|
as separate steps:
|
||||||
|
|
||||||
|
#. Create Networks
|
||||||
|
#. Create Virtual IPs
|
||||||
|
#. Provision Baremetal Instances
|
||||||
|
#. Deploy Ceph
|
||||||
|
#. Create the overcloud Ephemeral Heat stack
|
||||||
|
#. Run Config-Download and the deploy-steps playbook
|
||||||
|
|
||||||
|
This document covers the "Deploy Ceph" step above. For details on the
|
||||||
|
other steps see :doc:`../deployment/network_v2`.
|
||||||
|
|
||||||
|
The "Provision Baremetal Instances" step outputs a YAML file
|
||||||
|
describing the deployed baremetal, for example::
|
||||||
|
|
||||||
|
openstack overcloud node provision \
|
||||||
|
-o ~/deployed_metal.yaml \
|
||||||
|
...
|
||||||
|
|
||||||
|
The deployed_metal.yaml file can be passed as input to the ``openstack
|
||||||
|
overcloud ceph deploy`` command, which in turn outputs a YAML file
|
||||||
|
describing the deployed Ceph cluster, for example::
|
||||||
|
|
||||||
|
openstack overcloud ceph deploy \
|
||||||
|
~/deployed_metal.yaml \
|
||||||
|
-o ~/deployed_ceph.yaml \
|
||||||
|
...
|
||||||
|
|
||||||
|
Both the deployed_metal.yaml and deployed_ceph.yaml files may then be
|
||||||
|
passed as input to the step to "Create the overcloud Ephemeral Heat
|
||||||
|
stack", for example::
|
||||||
|
|
||||||
|
openstack overcloud deploy --templates \
|
||||||
|
-e ~/deployed_metal.yaml \
|
||||||
|
-e ~/deployed_ceph.yaml \
|
||||||
|
...
|
||||||
|
|
||||||
|
While the overcloud is being deployed the data in the
|
||||||
|
deployed_ceph.yaml file will be used to configure the OpenStack
|
||||||
|
clients to connect to the Ceph cluster as well as configure the Ceph
|
||||||
|
cluster to host OpenStack.
|
||||||
|
|
||||||
|
The above workflow is called "Deployed Ceph" because Ceph is already
|
||||||
|
deployed when the overcloud is configured.
|
||||||
|
|
||||||
|
Deployed Ceph Scope
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
The "Deployed Ceph" feature deploys a Ceph cluster ready to serve RBD
|
||||||
|
by calling the same TripleO Ansible roles described in :doc:`cephadm`.
|
||||||
|
When the "Deployed Ceph" process is over you should expect to find the
|
||||||
|
following:
|
||||||
|
|
||||||
|
- The CephMon, CephMgr, and CephOSD services are running on all nodes
|
||||||
|
which should have those services
|
||||||
|
- It's possible to SSH into a node with the CephMon service and run
|
||||||
|
`sudo cepham shell`
|
||||||
|
- All OSDs should be running unless there were environmental issues
|
||||||
|
(e.g. disks were not cleaned)
|
||||||
|
- A ceph configuration file and client admin keyring file in /etc/ceph
|
||||||
|
of overcloud nodes with the CephMon service
|
||||||
|
- The Ceph cluster is ready to serve RBD
|
||||||
|
|
||||||
|
You should not expect the following after "Deployed Ceph" has run:
|
||||||
|
|
||||||
|
- No pools or cephx keys for OpenStack will be created yet
|
||||||
|
- No CephDashboard, CephRGW or CephMds services will be running yet
|
||||||
|
|
||||||
|
The above will be configured during overcloud deployment by the
|
||||||
|
`openstack overcloud deploy` command as they were prior to the
|
||||||
|
"Deployed Ceph" feature. The reasons for this are the following:
|
||||||
|
|
||||||
|
- The Dashboard and RGW services need to integrate with haproxy which
|
||||||
|
is deployed with the overcloud
|
||||||
|
- The list of pools to create and their respective cephx keys are a
|
||||||
|
function of which OpenStack clients (e.g. Nova, Cinder, etc) will be
|
||||||
|
used so they must be in the overcloud definition. Thus, they are
|
||||||
|
created during overcloud deployment
|
||||||
|
|
||||||
|
During the overcloud deployment the above resources will be
|
||||||
|
created in Ceph by the TripleO Ansible roles described in
|
||||||
|
:doc:`cephadm` using the client admin keyring file and the
|
||||||
|
``~/deployed_ceph.yaml`` file output by `openstack overcloud ceph
|
||||||
|
deploy`. Because these resources are created directly on the Ceph
|
||||||
|
cluster with admin level access, "Deployed Ceph" is different from
|
||||||
|
the "External Ceph" feature described in :doc:`ceph_external`.
|
||||||
|
|
||||||
|
The main benefits of using "Deployed Ceph" are the following:
|
||||||
|
|
||||||
|
- Use cephadm to deploy Ceph on the hardware managed by TripleO
|
||||||
|
without having to write your own cephadm spec file (though you may
|
||||||
|
provide your own if you wish)
|
||||||
|
- Focus on debugging the basic Ceph deployment without debugging the
|
||||||
|
overcloud deployment at the same time
|
||||||
|
- Fix any Ceph deployment problems directly using either Ansible or
|
||||||
|
the Ceph orchestrator tools before starting the overcloud deployment
|
||||||
|
- Have the benefits above while maintaining hyperconverged support by
|
||||||
|
using a tested workflow
|
||||||
|
|
||||||
|
In summary, `openstack overcloud ceph deploy` deploys the Ceph cluster
|
||||||
|
while `openstack overcloud deploy` (and the commands that follow)
|
||||||
|
deploy OpenStack and configure that Ceph cluster to be used by
|
||||||
|
OpenStack.
|
||||||
|
|
||||||
|
Deployed Ceph Command Line Interface
|
||||||
|
------------------------------------
|
||||||
|
|
||||||
|
The command line interface supports the following options::
|
||||||
|
|
||||||
|
$ openstack overcloud ceph deploy --help
|
||||||
|
usage: openstack overcloud ceph deploy [-h] -o <deployed_ceph.yaml> [-y]
|
||||||
|
[--stack STACK]
|
||||||
|
[--working-dir WORKING_DIR]
|
||||||
|
[--roles-data ROLES_DATA]
|
||||||
|
[--ceph-spec CEPH_SPEC | --osd-spec OSD_SPEC]
|
||||||
|
[--container-image-prepare CONTAINER_IMAGE_PREPARE]
|
||||||
|
[--container-namespace CONTAINER_NAMESPACE]
|
||||||
|
[--container-image CONTAINER_IMAGE]
|
||||||
|
[--container-tag CONTAINER_TAG]
|
||||||
|
[--registry-url REGISTRY_URL]
|
||||||
|
[--registry-username REGISTRY_USERNAME]
|
||||||
|
[--registry-password REGISTRY_PASSWORD]
|
||||||
|
<deployed_baremetal.yaml>
|
||||||
|
|
||||||
|
positional arguments:
|
||||||
|
<deployed_baremetal.yaml>
|
||||||
|
Path to the environment file output from "openstack
|
||||||
|
overcloud node provision".
|
||||||
|
|
||||||
|
optional arguments:
|
||||||
|
-h, --help show this help message and exit
|
||||||
|
-o <deployed_ceph.yaml>, --output <deployed_ceph.yaml>
|
||||||
|
The path to the output environment file describing the
|
||||||
|
Ceph deployment to pass to the overcloud deployment.
|
||||||
|
-y, --yes Skip yes/no prompt before overwriting an existing
|
||||||
|
<deployed_ceph.yaml> output file (assume yes).
|
||||||
|
--stack STACK Name or ID of heat stack (default=Env:
|
||||||
|
OVERCLOUD_STACK_NAME)
|
||||||
|
--working-dir WORKING_DIR
|
||||||
|
The working directory for the deployment where all
|
||||||
|
input, output, and generated files will be stored.
|
||||||
|
Defaults to "$HOME/overcloud-deploy/<stack>"
|
||||||
|
--roles-data ROLES_DATA
|
||||||
|
Path to an alternative roles_data.yaml. Used to decide
|
||||||
|
which node gets which Ceph mon, mgr, or osd service
|
||||||
|
based on the node's role in <deployed_baremetal.yaml>.
|
||||||
|
--ceph-spec CEPH_SPEC
|
||||||
|
Path to an existing Ceph spec file. If not provided a
|
||||||
|
spec will be generated automatically based on --roles-
|
||||||
|
data and <deployed_baremetal.yaml>
|
||||||
|
--osd-spec OSD_SPEC Path to an existing OSD spec file. Mutually exclusive
|
||||||
|
with --ceph-spec. If the Ceph spec file is generated
|
||||||
|
automatically, then the OSD spec in the Ceph spec file
|
||||||
|
defaults to {data_devices: {all: true}} for all
|
||||||
|
service_type osd. Use --osd-spec to override the
|
||||||
|
data_devices value inside the Ceph spec file.
|
||||||
|
--container-image-prepare CONTAINER_IMAGE_PREPARE
|
||||||
|
Path to an alternative
|
||||||
|
container_image_prepare_defaults.yaml. Used to control
|
||||||
|
which Ceph container is pulled by cephadm via the
|
||||||
|
ceph_namespace, ceph_image, and ceph_tag variables in
|
||||||
|
addition to registry authentication via
|
||||||
|
ContainerImageRegistryCredentials.
|
||||||
|
|
||||||
|
container-image-prepare overrides:
|
||||||
|
The following options may be used to override individual values set via
|
||||||
|
--container-image-prepare. If the example variables below were set the
|
||||||
|
image would be concatenated into quay.io/ceph/ceph:latest and a custom
|
||||||
|
registry login would be used.
|
||||||
|
|
||||||
|
--container-namespace CONTAINER_NAMESPACE
|
||||||
|
e.g. quay.io/ceph
|
||||||
|
--container-image CONTAINER_IMAGE
|
||||||
|
e.g. ceph
|
||||||
|
--container-tag CONTAINER_TAG
|
||||||
|
e.g. latest
|
||||||
|
--registry-url REGISTRY_URL
|
||||||
|
--registry-username REGISTRY_USERNAME
|
||||||
|
--registry-password REGISTRY_PASSWORD
|
||||||
|
|
||||||
|
This command is provided by the python-tripleoclient plugin.
|
||||||
|
$
|
||||||
|
|
||||||
|
Run `openstack overcloud ceph deploy --help` in your own environment
|
||||||
|
to see the latest options which you have available.
|
||||||
|
|
||||||
|
Ceph Spec Options
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
The roles file, described in the next section, and the output of
|
||||||
|
`openstack overcloud node provision` are passed to the
|
||||||
|
`ceph_spec_bootstrap`_ Ansible module to create a `Ceph Service
|
||||||
|
Specification`_. The `openstack overcloud ceph deploy` command does
|
||||||
|
this automatically so it is not necessary to use the options described
|
||||||
|
in this section unless desired.
|
||||||
|
|
||||||
|
It's possible to generate a Ceph Spec on the undercloud before
|
||||||
|
deployment by using the `ceph_spec_bootstrap`_ Ansible module
|
||||||
|
directly, for example::
|
||||||
|
|
||||||
|
ansible localhost -m ceph_spec_bootstrap \
|
||||||
|
-a deployed_metalsmith=deployed_metal.yaml
|
||||||
|
|
||||||
|
By default the above creates the file ``~/ceph_spec.yaml``. For more
|
||||||
|
information on the ``ceph_spec_bootstrap`` module run `ansible-doc
|
||||||
|
ceph_spec_bootstrap`. The spec file may then be edited if desired and
|
||||||
|
passed directly like this::
|
||||||
|
|
||||||
|
openstack overcloud ceph deploy \
|
||||||
|
deployed_metal.yaml \
|
||||||
|
-o deployed_ceph.yaml \
|
||||||
|
--ceph-spec ~/ceph_spec.yaml
|
||||||
|
|
||||||
|
All available disks (excluding the disk where the operating system is
|
||||||
|
installed) are used as OSDs as per the following default inside the
|
||||||
|
ceph spec::
|
||||||
|
|
||||||
|
data_devices:
|
||||||
|
all: true
|
||||||
|
|
||||||
|
In the above example, the `data_devices` key is valid for any `Ceph
|
||||||
|
Service Specification`_ whose `service_type` is "osd". Other OSD
|
||||||
|
service types, as found in the `Advanced OSD Service
|
||||||
|
Specifications`_, may be set by using the ``--osd-spec`` option.
|
||||||
|
|
||||||
|
If the file ``osd_spec.yaml`` contains the following::
|
||||||
|
|
||||||
|
data_devices:
|
||||||
|
rotational: 1
|
||||||
|
db_devices:
|
||||||
|
rotational: 0
|
||||||
|
|
||||||
|
and the following command is run::
|
||||||
|
|
||||||
|
openstack overcloud ceph deploy \
|
||||||
|
deployed_metal.yaml \
|
||||||
|
-o deployed_ceph.yaml \
|
||||||
|
--osd-spec osd_spec.yaml
|
||||||
|
|
||||||
|
Then all rotating devices will be data devices and all non-rotating
|
||||||
|
devices will be used as shared devices (wal, db). This is because when
|
||||||
|
the dynamic Ceph service specification is built whatever is in the
|
||||||
|
file referenced by ``--osd-spec`` will be appended to the section of
|
||||||
|
the specification if the `service_type` is "osd".
|
||||||
|
|
||||||
|
Service Placement Options
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
The Ceph services defined in the roles_data.yaml file as described in
|
||||||
|
:doc:`composable_services` determine which baremetal node runs which
|
||||||
|
service. By default the Controller role has the CephMon and CephMgr
|
||||||
|
service while the CephStorage role has the CephOSD service. Most
|
||||||
|
composable services require Heat output in order to determine how
|
||||||
|
services are configured, but not the Ceph services. Thus, the
|
||||||
|
roles_data.yaml file remains authoritative for Ceph service placement
|
||||||
|
even though the "Deployed Ceph" process happens before Heat is run.
|
||||||
|
|
||||||
|
It is only necessary to use the `--roles-file` option if the default
|
||||||
|
roles_data.yaml file is not being used. For example if you intend to
|
||||||
|
deploy hyperconverged nodes, then you want the predeployed compute
|
||||||
|
nodes to be in the ceph spec with the "osd" label and for the
|
||||||
|
`service_type` "osd" to have a placement list containing a list of the
|
||||||
|
compute nodes. To do this generate a custom roles file as described in
|
||||||
|
:doc:`composable_services` like this::
|
||||||
|
|
||||||
|
openstack overcloud roles generate Controller ComputeHCI > custom_roles.yaml
|
||||||
|
|
||||||
|
and then pass that roles file like this::
|
||||||
|
|
||||||
|
openstack overcloud ceph deploy \
|
||||||
|
deployed_metal.yaml \
|
||||||
|
-o deployed_ceph.yaml \
|
||||||
|
--roles-data custom_roles.yaml
|
||||||
|
|
||||||
|
After running the above the compute nodes should have running OSD
|
||||||
|
containers and when the overcloud is deployed Nova compute services
|
||||||
|
will then be set up on the same hosts.
|
||||||
|
|
||||||
|
If you wish to generate the ceph spec with the modified placement
|
||||||
|
described above before the ceph deployment, then the same file may be
|
||||||
|
passed to a direct call of the ceph_spec_bootstrap ansible module::
|
||||||
|
|
||||||
|
ansible localhost -m ceph_spec_bootstrap \
|
||||||
|
-a "deployed_metalsmith=deployed_metal.yaml tripleo_roles=custom_roles.yaml"
|
||||||
|
|
||||||
|
Container Options
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
As described in :doc:`../deployment/container_image_prepare` the
|
||||||
|
undercloud may be used as a container registry for ceph containers
|
||||||
|
and there is a supported syntax to download containers from
|
||||||
|
authenticated registries. By default `openstack overcloud ceph deploy`
|
||||||
|
will pull the Ceph container in the default
|
||||||
|
``container_image_prepare_defaults.yaml`` file. The version of the
|
||||||
|
Ceph used in each OpenStack release changes per release and can be
|
||||||
|
seen by running a command like this::
|
||||||
|
|
||||||
|
egrep "ceph_namespace|ceph_image|ceph_tag" \
|
||||||
|
/usr/share/tripleo-common/container-images/container_image_prepare_defaults.yaml
|
||||||
|
|
||||||
|
The `--container-image-prepare` option can be used to override which
|
||||||
|
``container_image_prepare_defaults.yaml`` file is used. If a version
|
||||||
|
of this file called ``custom_container_image_prepare.yaml`` is
|
||||||
|
modified to contain syntax like the following::
|
||||||
|
|
||||||
|
ContainerImageRegistryCredentials:
|
||||||
|
quay.io/ceph-ci:
|
||||||
|
quay_username: quay_password
|
||||||
|
|
||||||
|
Then when a command like the following is run::
|
||||||
|
|
||||||
|
openstack overcloud ceph deploy \
|
||||||
|
deployed_metal.yaml \
|
||||||
|
-o deployed_ceph.yaml \
|
||||||
|
--container-image-prepare custom_container_image_prepare.yaml
|
||||||
|
|
||||||
|
The credentials will be extracted from the file and the tripleo
|
||||||
|
ansible role to bootstrap Ceph will be executed like this::
|
||||||
|
|
||||||
|
cephadm bootstrap
|
||||||
|
--registry-url quay.io/ceph-ci
|
||||||
|
--registry-username quay_username
|
||||||
|
--registry-password quay_password
|
||||||
|
...
|
||||||
|
|
||||||
|
The syntax of the container image prepare file can also be ignored and
|
||||||
|
instead the following command line options may be used instead::
|
||||||
|
|
||||||
|
--container-namespace CONTAINER_NAMESPACE
|
||||||
|
e.g. quay.io/ceph
|
||||||
|
--container-image CONTAINER_IMAGE
|
||||||
|
e.g. ceph
|
||||||
|
--container-tag CONTAINER_TAG
|
||||||
|
e.g. latest
|
||||||
|
--registry-url REGISTRY_URL
|
||||||
|
--registry-username REGISTRY_USERNAME
|
||||||
|
--registry-password REGISTRY_PASSWORD
|
||||||
|
|
||||||
|
If a variable above is unused, then it defaults to the ones found in
|
||||||
|
the default ``container_image_prepare_defaults.yaml`` file. In other
|
||||||
|
words, the above options are overrides.
|
||||||
|
|
||||||
|
.. _`ceph_spec_bootstrap`: https://docs.openstack.org/tripleo-ansible/latest/modules/modules-ceph_spec_bootstrap.html
|
||||||
|
.. _`Ceph Service Specification`: https://docs.ceph.com/en/latest/cephadm/service-management/#orchestrator-cli-service-spec
|
||||||
|
.. _`Advanced OSD Service Specifications`: https://docs.ceph.com/en/latest/cephadm/osd/#drivegroups
|
Loading…
x
Reference in New Issue
Block a user