590bb27b37
Removed docker references (the driver has been removed from nova). Update confilcting pages including tables to use xrefs instead. Add a rpc_all section to handle conflicts in the training-guides. Closes-Bug: #1291138 Closes-Bug: #1292612 Change-Id: I67ad31c3913cae4c2e9e3d2ce41302717af5d839
261 lines
16 KiB
XML
261 lines
16 KiB
XML
<section xml:id="section_compute-cells"
|
|
xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
|
xmlns:ns5="http://www.w3.org/1999/xhtml"
|
|
xmlns:ns4="http://www.w3.org/2000/svg"
|
|
xmlns:ns3="http://www.w3.org/1998/Math/MathML"
|
|
xmlns:ns="http://docbook.org/ns/docbook"
|
|
version="5.0">
|
|
<title>Cells</title>
|
|
<?dbhtml stop-chunking?>
|
|
<para><emphasis role="italic">Cells</emphasis> functionality allows you to scale an OpenStack
|
|
Compute cloud in a more distributed fashion without having to use complicated technologies
|
|
like database and message queue clustering. It is intended to support very large
|
|
deployments.</para>
|
|
<para>When this functionality is enabled, the hosts in an OpenStack Compute cloud are
|
|
partitioned into groups called cells. Cells are configured as a tree. The top-level cell
|
|
should have a host that runs a <systemitem class="service">nova-api</systemitem> service,
|
|
but no <systemitem class="service">nova-compute</systemitem> services. Each child cell
|
|
should run all of the typical <systemitem class="service">nova-*</systemitem> services in a
|
|
regular Compute cloud except for <systemitem class="service">nova-api</systemitem>. You can
|
|
think of cells as a normal Compute deployment in that each cell has its own database server
|
|
and message queue broker.</para>
|
|
<para>The <systemitem class="service">nova-cells</systemitem> service handles communication
|
|
between cells and selects cells for new instances. This service is required for every cell.
|
|
Communication between cells is pluggable, and currently the only option is communication through
|
|
RPC.</para>
|
|
<para>Cells scheduling is separate from host scheduling. <systemitem class="service">nova-cells</systemitem> first picks a
|
|
cell (now randomly, but future releases plan to add filtering/weighing functionality, and
|
|
decisions will be based on broadcasts of capacity/capabilities). Once a cell is
|
|
selected and the new build request reaches its <systemitem class="service">nova-cells</systemitem> service, it is sent
|
|
over to the host scheduler in that cell and the build proceeds as it would have without
|
|
cells.</para>
|
|
<warning>
|
|
<para>Cell functionality is currently considered experimental.</para>
|
|
</warning>
|
|
<section xml:id="cell-config-options">
|
|
<title>Cell configuration options</title>
|
|
<para>Cells are disabled by default. All cell-related configuration options go under a
|
|
<literal>[cells]</literal> section in <filename>nova.conf</filename>. The following
|
|
cell-related options are currently supported:<variablelist>
|
|
<varlistentry>
|
|
<term><literal>enable</literal></term>
|
|
<listitem>
|
|
<para>Set this is <literal>True</literal> to turn on cell functionality,
|
|
which is off by default.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><literal>name</literal></term>
|
|
<listitem>
|
|
<para>Name of the current cell. This must be unique for each cell.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><literal>capabilities</literal></term>
|
|
<listitem>
|
|
<para>List of arbitrary
|
|
<literal><replaceable>key</replaceable>=<replaceable>value</replaceable></literal>
|
|
pairs defining capabilities of the current cell.
|
|
Values include
|
|
<literal>hypervisor=xenserver;kvm,os=linux;windows</literal>.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term><literal>call_timeout</literal></term>
|
|
<listitem>
|
|
<para>How long in seconds to wait for replies from calls
|
|
between cells.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term>scheduler_filter_classes</term>
|
|
<listitem>
|
|
<para>Filter classes that the cells scheduler should use. By default, uses
|
|
"<literal>nova.cells.filters.all_filters</literal>" to map to all
|
|
cells filters included with Compute.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term>scheduler_weight_classes</term>
|
|
<listitem><para>Weight classes the cells scheduler should use. By default, uses
|
|
"<literal>nova.cells.weights.all_weighers</literal>"
|
|
to map to all cells weight algorithms (weighers)
|
|
included with Compute.</para></listitem>
|
|
</varlistentry>
|
|
<varlistentry>
|
|
<term>ram_weight_multiplier</term>
|
|
<listitem>
|
|
<para>Multiplier used for weighing ram. Negative numbers
|
|
mean you want Compute to stack VMs on one host
|
|
instead of spreading out new VMs to more hosts in
|
|
the cell. Default value is 10.0.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist></para>
|
|
</section>
|
|
<section xml:id="config-API-cell">
|
|
<title>Configure the API (top-level) cell</title>
|
|
<para>The compute API class must be changed in the API cell so that requests can be proxied
|
|
through nova-cells down to the correct cell properly. Add the following to
|
|
<filename>nova.conf</filename> in the API
|
|
cell:<programlisting language="ini">[DEFAULT]
|
|
compute_api_class=nova.compute.cells_api.ComputeCellsAPI
|
|
...
|
|
|
|
[cells]
|
|
enable=True
|
|
name=api</programlisting></para>
|
|
</section>
|
|
<section xml:id="config-child-cell">
|
|
<title>Configure the child cells</title>
|
|
<para>Add the following to <filename>nova.conf</filename> in the child cells, replacing
|
|
<replaceable>cell1</replaceable> with the name of each
|
|
cell:<programlisting language="ini">[DEFAULT]
|
|
# Disable quota checking in child cells. Let API cell do it exclusively.
|
|
quota_driver=nova.quota.NoopQuotaDriver
|
|
|
|
[cells]
|
|
enable=True
|
|
name=<replaceable>cell1</replaceable></programlisting></para>
|
|
</section>
|
|
<section xml:id="config-cell-db">
|
|
<title>Configure the database in each cell</title>
|
|
<para>Before bringing the services online, the database in each cell needs to be configured
|
|
with information about related cells. In particular, the API cell needs to know about
|
|
its immediate children, and the child cells must know about their immediate agents.
|
|
The information needed is the <application>RabbitMQ</application> server credentials
|
|
for the particular cell.</para>
|
|
<para>Use the <command>nova-manage cell create</command> command to add this information to
|
|
the database in each
|
|
cell:<screen><prompt>#</prompt> <userinput>nova-manage cell create -h</userinput>
|
|
<computeroutput>Options:
|
|
-h, --help show this help message and exit
|
|
--name=<name> Name for the new cell
|
|
--cell_type=<parent|child>
|
|
Whether the cell is a parent or child
|
|
--username=<username>
|
|
Username for the message broker in this cell
|
|
--password=<password>
|
|
Password for the message broker in this cell
|
|
--hostname=<hostname>
|
|
Address of the message broker in this cell
|
|
--port=<number> Port number of the message broker in this cell
|
|
--virtual_host=<virtual_host>
|
|
The virtual host of the message broker in this cell
|
|
--woffset=<float>
|
|
(weight offset) It might be used by some cell scheduling code in the future
|
|
--wscale=<float>
|
|
(weight scale) It might be used by some cell scheduling code in the future
|
|
</computeroutput></screen></para>
|
|
<para>As an example, assume we have an API cell named <literal>api</literal> and a child
|
|
cell named <literal>cell1</literal>. Within the api cell, we have the following RabbitMQ
|
|
server
|
|
info:<programlisting language="ini">rabbit_host=10.0.0.10
|
|
rabbit_port=5672
|
|
rabbit_username=api_user
|
|
rabbit_password=api_passwd
|
|
rabbit_virtual_host=api_vhost</programlisting></para>
|
|
<para>And in the child cell named <literal>cell1</literal> we have the following RabbitMQ
|
|
server
|
|
info:<programlisting language="ini">rabbit_host=10.0.1.10
|
|
rabbit_port=5673
|
|
rabbit_username=cell1_user
|
|
rabbit_password=cell1_passwd
|
|
rabbit_virtual_host=cell1_vhost</programlisting></para>
|
|
<para>We would run this in the API cell, as
|
|
root.<screen><prompt>#</prompt> <userinput>nova-manage cell create --name=cell1 --cell_type=child --username=cell1_user --password=cell1_passwd --hostname=10.0.1.10 --port=5673 --virtual_host=cell1_vhost --woffset=1.0 --wscale=1.0</userinput></screen></para>
|
|
<para>Repeat the above for all child cells.</para>
|
|
<para>In the child cell, we would run the following, as
|
|
root:<screen><prompt>#</prompt> <userinput>nova-manage cell create --name=api --cell_type=parent --username=api1_user --password=api1_passwd --hostname=10.0.0.10 --port=5672 --virtual_host=api_vhost --woffset=1.0 --wscale=1.0
|
|
</userinput></screen></para>
|
|
<para>To customize the Compute cells, use the configuration option
|
|
settings documented in <xref
|
|
linkend="config_table_nova_cells"/>.</para>
|
|
</section>
|
|
<section xml:id="cell-weights-filters">
|
|
<title>Cell scheduling configuration</title>
|
|
<para>To determine the best cell for launching a new instance, Compute uses a set of
|
|
filters and weights configured in <filename>/etc/nova/nova.conf</filename>. The
|
|
following options are available to prioritize cells for scheduling:</para>
|
|
<para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><code>scheduler_filter_classes</code> - Specifies the list of filter
|
|
classes. By default <code>nova.cells.weights.all_filters</code> is
|
|
specified, which maps to all cells filters included with Compute (see
|
|
<xref linkend="scheduler-filters"/>.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><code>scheduler_weight_classes</code> - Specifies the list of weight
|
|
classes. By default <code>nova.cells.weights.all_weighers</code> is
|
|
specified, which maps to all cell weight algorithms (weighers) included with
|
|
Compute. The following modules are available:</para>
|
|
<itemizedlist><listitem>
|
|
<para><code>mute_child</code>: Downgrades the likelihood of child cells
|
|
being chosen for scheduling requests, which haven't sent capacity or
|
|
capability updates in a while. Options include
|
|
<code>mute_weight_multiplier</code> (multiplier for mute
|
|
children; value should be negative) and
|
|
<code>mute_weight_value</code> (assigned to mute children;
|
|
should be a positive value).</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><code>ram_by_instance_type</code>: Select cells with the most RAM
|
|
capacity for the instance type being requested. Because higher
|
|
weights win, Compute returns the number of available units for the
|
|
instance type requested. The <code>ram_weight_multiplier</code>
|
|
option defaults to 10.0 that adds to the weight by a factor of 10.
|
|
Use a negative number to stack VMs on one host instead of spreading
|
|
out new VMs to more hosts in the cell.</para>
|
|
</listitem>
|
|
<listitem><para><code>weight_offset</code>: Allows modifying the database to weight a particular cell. You can
|
|
use this when you want to disable a cell (for example, '0'), or to
|
|
set a default cell by making its weight_offset very high (for
|
|
example, '999999999999999'). The highest weight will be the first
|
|
cell to be scheduled for launching an instance.</para></listitem></itemizedlist>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
<para>Additionally, the following options are available for the cell scheduler:</para>
|
|
<para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para><code>scheduler_retries</code> - Specifies how many times the scheduler
|
|
tries to launch a new instance when no cells are available (default=10).</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><code>scheduler_retry_delay</code> - Specifies the delay (in seconds)
|
|
between retries (default=2).</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
<para>As an admin user, you can also add a filter that directs builds to
|
|
a particular cell. The <filename>policy.json</filename> file must
|
|
have a line with <literal>"cells_scheduler_filter:TargetCellFilter"
|
|
: "is_admin:True"</literal> to let an admin user specify a
|
|
scheduler hint to direct a build to a particular cell.</para>
|
|
</section>
|
|
<section xml:id="cell-config-optional-json">
|
|
<title>Optional cell configuration</title>
|
|
<para>Cells currently keeps all inter-cell communication data, including
|
|
user names and passwords, in the database. This is undesirable and
|
|
unnecessary since cells data isn't updated very frequently. Instead,
|
|
create a JSON file to input cells data specified via a
|
|
<code>[cells]cells_config</code> option. When specified, the
|
|
database is no longer consulted when reloading cells data. The file
|
|
will need the columns present in the Cell model (excluding common
|
|
database fields and the <code>id</code> column). The queue
|
|
connection information must be specified through a
|
|
<code>transport_url</code> field, instead of
|
|
<code>username</code>, <code>password</code>, and so on. The
|
|
<code>transport_url</code> has the following form:</para>
|
|
<programlisting>
|
|
rabbit://<username>:<password>@<hostname>:<port>/<virtual_host>
|
|
</programlisting>
|
|
<para>The scheme can be either <literal>qpid</literal> or <literal>rabbit</literal>, as shown previously. The following sample shows this optional configuration:</para>
|
|
<programlisting language="json"><xi:include href="samples/cells-resp.json" parse="text"/></programlisting>
|
|
</section>
|
|
</section>
|