be95faee43
Like documented at https://wiki.openstack.org/wiki/Documentation/Conventions#Client_arguments:_.22--option_ARGUMENT.22 we prefer to use '--option ARGUMENT'. Change-Id: Iea99bbd253ee4ced00ab9983c00ebc805e6e7568
323 lines
17 KiB
XML
323 lines
17 KiB
XML
<section xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
version="5.0"
|
|
xml:id="section_compute-cells">
|
|
<title>Cells</title>
|
|
<?dbhtml stop-chunking?>
|
|
<para><emphasis role="italic">Cells</emphasis> functionality
|
|
enables you to scale an OpenStack Compute cloud in a more
|
|
distributed fashion without having to use complicated
|
|
technologies like database and message queue clustering. It
|
|
supports very large deployments.</para>
|
|
<para>When this functionality is enabled, the hosts in an
|
|
OpenStack Compute cloud are partitioned into groups called
|
|
cells. Cells are configured as a tree. The top-level cell
|
|
should have a host that runs a <systemitem class="service"
|
|
>nova-api</systemitem> service, but no <systemitem
|
|
class="service">nova-compute</systemitem> services. Each
|
|
child cell should run all of the typical <systemitem
|
|
class="service">nova-*</systemitem> services in a regular
|
|
Compute cloud except for <systemitem class="service"
|
|
>nova-api</systemitem>. You can think of cells as a normal
|
|
Compute deployment in that each cell has its own database
|
|
server and message queue broker.</para>
|
|
<para>The <systemitem class="service">nova-cells</systemitem>
|
|
service handles communication between cells and selects cells
|
|
for new instances. This service is required for every cell.
|
|
Communication between cells is pluggable, and currently the
|
|
only option is communication through RPC.</para>
|
|
<para>Cells scheduling is separate from host scheduling.
|
|
<systemitem class="service">nova-cells</systemitem> first
|
|
picks a cell. Once a cell is selected and the new build
|
|
request reaches its <systemitem class="service"
|
|
>nova-cells</systemitem> service, it is sent over to the
|
|
host scheduler in that cell and the build proceeds as it would
|
|
have without cells.</para>
|
|
<warning>
|
|
<para>Cell functionality is currently considered
|
|
experimental.</para>
|
|
</warning>
|
|
<section xml:id="cell-config-options">
|
|
<title>Cell configuration options</title>
|
|
<para>Cells are disabled by default. All cell-related
|
|
configuration options appear in the
|
|
<literal>[cells]</literal> section in
|
|
<filename>nova.conf</filename>. The following
|
|
cell-related options are currently supported:<variablelist>
|
|
<varlistentry>
|
|
<term><option>enable</option></term>
|
|
<listitem>
|
|
<para>Set to <literal>True</literal> to turn
|
|
on cell functionality. Default is
|
|
<literal>false</literal>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><option>name</option></term>
|
|
<listitem>
|
|
<para>Name of the current cell. Must be unique
|
|
for each cell.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><option>capabilities</option></term>
|
|
<listitem>
|
|
<para>List of arbitrary
|
|
<literal><replaceable>key</replaceable>=<replaceable>value</replaceable></literal>
|
|
pairs defining capabilities of the current
|
|
cell. Values include
|
|
<literal>hypervisor=xenserver;kvm,os=linux;windows</literal>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><option>call_timeout</option></term>
|
|
<listitem>
|
|
<para>How long in seconds to wait for replies
|
|
from calls between cells.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><option>scheduler_filter_classes</option></term>
|
|
<listitem>
|
|
<para>Filter classes that the cells scheduler
|
|
should use. By default, uses
|
|
"<literal>nova.cells.filters.all_filters</literal>"
|
|
to map to all cells filters included with
|
|
Compute.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><option>scheduler_weight_classes</option></term>
|
|
<listitem>
|
|
<para>Weight classes that the scheduler for cells uses. By default, uses
|
|
<literal>nova.cells.weights.all_weighers</literal>
|
|
to map to all cells weight algorithms
|
|
included with Compute.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><option>ram_weight_multiplier</option></term>
|
|
<listitem>
|
|
<para>Multiplier used to weight RAM.
|
|
Negative numbers indicate that Compute should
|
|
stack VMs on one host instead of spreading
|
|
out new VMs to more hosts in the cell. The
|
|
default value is 10.0.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist></para>
|
|
</section>
|
|
<section xml:id="config-API-cell">
|
|
<title>Configure the API (top-level) cell</title>
|
|
<para>The compute API class must be changed in the API cell so
|
|
that requests can be proxied through nova-cells down to
|
|
the correct cell properly. Add the following line to
|
|
<filename>nova.conf</filename> in the API
|
|
cell:<programlisting language="ini">[DEFAULT]
|
|
compute_api_class=nova.compute.cells_api.ComputeCellsAPI
|
|
...
|
|
|
|
[cells]
|
|
enable=True
|
|
name=api</programlisting></para>
|
|
</section>
|
|
<section xml:id="config-child-cell">
|
|
<title>Configure the child cells</title>
|
|
<para>Add the following lines to
|
|
<filename>nova.conf</filename> in the child cells,
|
|
replacing <replaceable>cell1</replaceable> with the name
|
|
of each
|
|
cell:<programlisting language="ini">[DEFAULT]
|
|
# Disable quota checking in child cells. Let API cell do it exclusively.
|
|
quota_driver=nova.quota.NoopQuotaDriver
|
|
|
|
[cells]
|
|
enable=True
|
|
name=<replaceable>cell1</replaceable></programlisting></para>
|
|
</section>
|
|
<section xml:id="config-cell-db">
|
|
<title>Configure the database in each cell</title>
|
|
<para>Before bringing the services online, the database in
|
|
each cell needs to be configured with information about
|
|
related cells. In particular, the API cell needs to know
|
|
about its immediate children, and the child cells must
|
|
know about their immediate agents. The information needed
|
|
is the <application>RabbitMQ</application> server
|
|
credentials for the particular cell.</para>
|
|
<para>Use the <command>nova-manage cell create</command>
|
|
command to add this information to the database in each
|
|
cell:<screen><prompt>#</prompt> <userinput>nova-manage cell create -h</userinput>
|
|
<computeroutput>Options:
|
|
-h, --help show this help message and exit
|
|
--name=<name> Name for the new cell
|
|
--cell_type=<parent|child>
|
|
Whether the cell is a parent or child
|
|
--username=<username>
|
|
Username for the message broker in this cell
|
|
--password=<password>
|
|
Password for the message broker in this cell
|
|
--hostname=<hostname>
|
|
Address of the message broker in this cell
|
|
--port=<number> Port number of the message broker in this cell
|
|
--virtual_host=<virtual_host>
|
|
The virtual host of the message broker in this cell
|
|
--woffset=<float>
|
|
(weight offset) It might be used by some cell scheduling code in the future
|
|
--wscale=<float>
|
|
(weight scale) It might be used by some cell scheduling code in the future</computeroutput></screen></para>
|
|
<para>As an example, assume an API cell named
|
|
<literal>api</literal> and a child cell named
|
|
<literal>cell1</literal>.</para>
|
|
<para>Within the <literal>api</literal> cell, specify the
|
|
following RabbitMQ server
|
|
information:<programlisting language="ini">rabbit_host=10.0.0.10
|
|
rabbit_port=5672
|
|
rabbit_username=api_user
|
|
rabbit_password=api_passwd
|
|
rabbit_virtual_host=api_vhost</programlisting></para>
|
|
<para>Within the <literal>cell1</literal> child cell, specify
|
|
the following RabbitMQ server
|
|
information:<programlisting language="ini">rabbit_host=10.0.1.10
|
|
rabbit_port=5673
|
|
rabbit_username=cell1_user
|
|
rabbit_password=cell1_passwd
|
|
rabbit_virtual_host=cell1_vhost</programlisting></para>
|
|
<para>You can run this in the API cell as
|
|
root:<screen><prompt>#</prompt> <userinput>nova-manage cell create --name cell1 --cell_type child \
|
|
--username cell1_user --password cell1_passwd --hostname 10.0.1.10 \
|
|
--port 5673 --virtual_host cell1_vhost --woffset 1.0 --wscale 1.0</userinput></screen></para>
|
|
<para>Repeat the previous steps for all child cells.</para>
|
|
<para>In the child cell, run the following, as
|
|
root:<screen><prompt>#</prompt> <userinput>nova-manage cell create --name api --cell_type parent \
|
|
--username api_user --password api_passwd --hostname 10.0.0.10 \
|
|
--port 5672 --virtual_host api_vhost --woffset 1.0 --wscale 1.0</userinput></screen></para>
|
|
<para>To customize the Compute cells, use the configuration
|
|
option settings documented in <xref
|
|
linkend="config_table_nova_cells"/>.</para>
|
|
</section>
|
|
<section xml:id="cell-weights-filters">
|
|
<title>Cell scheduling configuration</title>
|
|
<para>To determine the best cell to use to launch a new
|
|
instance, Compute uses a set of filters and weights
|
|
defined in the <filename>/etc/nova/nova.conf</filename>
|
|
file. The following options are available to prioritize
|
|
cells for scheduling:</para>
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><option>scheduler_filter_classes</option></term>
|
|
<listitem>
|
|
<para>List of filter classes. By
|
|
default
|
|
<option>nova.cells.weights.all_filters</option>
|
|
is specified, which maps to all cells filters
|
|
included with Compute (see <xref
|
|
linkend="scheduler-filters"/>).</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><option>scheduler_weight_classes</option></term>
|
|
<listitem>
|
|
<para>List of weight classes. By
|
|
default
|
|
<option>nova.cells.weights.all_weighers</option>
|
|
is specified, which maps to all cell weight
|
|
algorithms included with Compute.
|
|
The following modules are available:</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><literal>mute_child</literal>.
|
|
Downgrades the likelihood of child
|
|
cells being chosen for scheduling
|
|
requests, which haven't sent capacity
|
|
or capability updates in a while.
|
|
Options include
|
|
<option>mute_weight_multiplier</option>
|
|
(multiplier for mute children; value
|
|
should be negative) and
|
|
<option>mute_weight_value</option>
|
|
(assigned to mute children; should be
|
|
a positive value).</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>ram_by_instance_type</literal>.
|
|
Select cells with the most RAM
|
|
capacity for the instance type being
|
|
requested. Because higher weights win,
|
|
Compute returns the number of
|
|
available units for the instance type
|
|
requested. The
|
|
<option>ram_weight_multiplier</option>
|
|
option defaults to 10.0 that adds to
|
|
the weight by a factor of 10. Use a
|
|
negative number to stack VMs on one
|
|
host instead of spreading out new VMs
|
|
to more hosts in the cell.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><literal>weight_offset</literal>.
|
|
Allows modifying the database to
|
|
weight a particular cell. You can use
|
|
this when you want to disable a cell
|
|
(for example, '0'), or to set a
|
|
default cell by making its
|
|
weight_offset very high (for example,
|
|
'999999999999999'). The highest weight
|
|
will be the first cell to be scheduled
|
|
for launching an instance.</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
<para>Additionally, the following options are available for
|
|
the cell scheduler:</para>
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><option>scheduler_retries</option></term>
|
|
<listitem>
|
|
<para>Specifies how many times the scheduler tries
|
|
to launch a new instance when no cells are
|
|
available (default=10).</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><option>scheduler_retry_delay</option></term>
|
|
<listitem>
|
|
<para>Specifies the delay (in seconds) between
|
|
retries (default=2).</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
<para>As an admin user, you can also add a filter that directs
|
|
builds to a particular cell. The
|
|
<filename>policy.json</filename> file must have a line
|
|
with <literal>"cells_scheduler_filter:TargetCellFilter" :
|
|
"is_admin:True"</literal> to let an admin user specify
|
|
a scheduler hint to direct a build to a particular
|
|
cell.</para>
|
|
</section>
|
|
<section xml:id="cell-config-optional-json">
|
|
<title>Optional cell configuration</title>
|
|
<para>Cells store all inter-cell communication data, including
|
|
user names and passwords, in the database. Because the
|
|
cells data is not updated very frequently, use the
|
|
<option>[cells]cells_config</option> option to specify
|
|
a JSON file to store cells data. With this configuration,
|
|
the database is no longer consulted when reloading the
|
|
cells data. The file must have columns present in the Cell
|
|
model (excluding common database fields and the
|
|
<option>id</option> column). You must specify the
|
|
queue connection information through a
|
|
<option>transport_url</option> field, instead of
|
|
<option>username</option>, <option>password</option>,
|
|
and so on. The <option>transport_url</option> has the
|
|
following form:</para>
|
|
<programlisting>rabbit://<replaceable>USERNAME</replaceable>:<replaceable>PASSWORD</replaceable>@<replaceable>HOSTNAME</replaceable>:<replaceable>PORT</replaceable>/<replaceable>VIRTUAL_HOST</replaceable></programlisting>
|
|
<para>The scheme can be either <literal>qpid</literal> or
|
|
<literal>rabbit</literal>, as shown previously. The
|
|
following sample shows this optional configuration:</para>
|
|
<programlisting language="json"><xi:include href="samples/cells-resp.json" parse="text"/></programlisting>
|
|
</section>
|
|
</section>
|