@@ -1,5 +1,8 @@
<?xml version="1.0" encoding="utf-8"?>
<chapter xmlns= "http://docbook.org/ns/docbook" xmlns:xi= "http://www.w3.org/2001/XInclude" xmlns:xlink= "http://www.w3.org/1999/xlink" version= "5.0" xml:id= "module001-ch008-queues-messaging" >
<chapter xmlns= "http://docbook.org/ns/docbook"
xmlns:xi= "http://www.w3.org/2001/XInclude"
xmlns:xlink= "http://www.w3.org/1999/xlink" version= "5.0"
xml:id= "module001-ch008-queues-messaging" >
<title > OpenStack Messaging and Queues</title>
<figure >
<title > Messaging in OpenStack</title>
@@ -20,7 +23,7 @@
<itemizedlist >
<listitem >
<para > Decoupling between client and servant (such as the client
does not need to know where the servant’ s reference
does not need to know where the servant reference
is).</para>
</listitem>
<listitem >
@@ -48,55 +51,53 @@
<para > Nova implements RPC (both request+response, and one-way,
respectively nicknamed ‘ rpc.call’ and ‘ rpc.cast’ ) over AMQP by
providing an adapter class which take cares of marshaling and
unmarshaling of messages into function calls. Each Nova service
(for example Compute, Scheduler, etc.) create two queues at the
un- marshall ing of messages into function calls. Each Nova service,
such as Compute, Scheduler, and so on, creates two queues at the
initialization time, one which accepts messages with routing keys
‘ NODE-TYPE.NODE-ID’ ( for example compute.hostname) and another,
which accepts messages with routing keys as generic ‘ NODE-TYPE’
(for example compute). The former is used specifically when
‘ NODE-TYPE.NODE-ID’ , for example, compute.hostname, and another,
which accepts messages with routing keys as generic ‘ NODE-TYPE’ , for example compute. The former is used specifically when
Nova-API needs to redirect commands to a specific node like
‘ euca-terminate instance’ . In this case, only the compute node
whose host’ s hypervisor is running the virtual machine can kill
the instance. The API acts as a consumer when RPC calls are
request/response, otherwise is acts as publisher only.</para>
<para > <guilabel > Nova RPC Mappings</guilabel> </para>
<para > The figure below shows the internals of a message broker
node (referred to as a RabbitMQ node in the diagrams) when a
single instance is deployed and shared in an OpenStack cloud.
Every Nova component connects to the message broker and,
depending on its personality (for example a compute node or a
network node), may use the queue either as an Invoker (such as
API or Scheduler) or a Worker (such as Compute or Network).
Invokers and Workers do not actually exist in the Nova object
model, but we are going to use them as an abstraction for sake
of clarity. An Invoker is a component that sends messages in the
queuing system via two operations: 1) rpc.call and ii) rpc.cast;
a Worker is a component that receives messages from the queuing
system and reply accordingly to rcp.call operations.</para>
<para > Figure 2 shows the following internal elements:</para>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Topic Publisher:</emphasis> a Topic
<para > <guilabel > Nova RPC Mappings</guilabel> </para>
<para > The figure below shows the internals of a message broker node
(referred to as a RabbitMQ node in the diagrams) when a single
instance is deployed and shared in an OpenStack cloud. Every Nova
component connects to the message broker and, depending on its
personality, such as a compute node or a network node, may
use the queue either as an Invoker (such as API or Scheduler) or a
Worker (such as Compute or Network). Invokers and Workers do not
actually exist in the Nova object model, but we are going to use
them as an abstraction for sake of clarity. An Invoker is a
component that sends messages in the queuing system via two
operations: 1) rpc.call and ii) rpc.cast; a Worker is a component
that receives messages from the queuing system and reply
accordingly to rcp.call operations.</para>
<para > Figure 2 shows the following internal elements:</para>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Topic Publisher:</emphasis> a Topic
Publisher comes to life when an rpc.call or an rpc.cast
operation is executed; this object is instantiated and used to
push a message to the queuing system. Every publisher connects
always to the same topic-based exchange; its life-cycle is
limited to the message delivery.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Direct Consumer:</emphasis> a
Direct Consumer comes to life if (an only if) a rpc.call
operation is executed; this object is instantiated and used to
receive a response message from the queuing system; Every
consumer connects to a unique direct-based exchange via a
unique exclusive queue; its life-cycle is limited to the
message delivery; the exchange and queue identifiers are
determined by a UUID generator, and are marshaled in the
message sent by the Topic Publisher (only rpc.call
operations).</para >
</ listitem>
<listitem >
<para > <emphasis role= "bold" > Topic Consumer:</emphasis> a Topic
</listitem>
<listitem >
<para > <emphasis role= "bold" > Direct Consumer:</emphasis> a Direct
Consumer comes to life if (an only if) a rpc.call operation is
executed; this object is instantiated and used to receive a
response message from the queuing system; Every consumer
connects to a unique direct-based exchange via a unique
exclusive queue; its life-cycle is limited to the message
delivery; the exchange and queue identifiers are determined by
a UUID generator, and are marshaled in the message sent by the
Topic Publisher (only rpc.call operations).</para>
</listitem >
<listitem >
<para > <emphasis role= "bold" > Topic Consumer:</emphasis> a Topic
Consumer comes to life as soon as a Worker is instantiated and
exists throughout its life-cycle; this object is used to
receive messages from the queue and it invokes the appropriate
@@ -108,39 +109,39 @@
key is ‘ topic’ ) and the other that is addressed only during
rpc.call operations (and it connects to a unique queue whose
exchange key is ‘ topic.host’ ).</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Direct Publisher:</emphasis> a
Direct Publisher comes to life only during rpc.call operations
and it is instantiated to return the message required by the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Direct Publisher:</emphasis> a Direct
Publisher comes to life only during rpc.call operations and it
is instantiated to return the message required by the
request/response operation. The object connects to a
direct-based exchange whose identity is dictated by the
incoming message.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Topic Exchange:</emphasis> The
</listitem>
<listitem >
<para > <emphasis role= "bold" > Topic Exchange:</emphasis> The
Exchange is a routing table that exists in the context of a
virtual host (the multi-tenancy mechanism provided by Qpid or
RabbitMQ); its type (such as topic vs. direct) determines the
routing policy; a message broker node will have only one
topic-based exchange for every topic in Nova.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Direct Exchange:</emphasis> this is
a routing table that is created during rpc.call operations;
</listitem>
<listitem >
<para > <emphasis role= "bold" > Direct Exchange:</emphasis> this is a
routing table that is created during rpc.call operations;
there are many instances of this kind of exchange throughout
the life-cycle of a message broker node, one for each rpc.call
invoked.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Queue Element:</emphasis> A Queue
is a message bucket. Messages are kept in the queue until a
</listitem>
<listitem >
<para > <emphasis role= "bold" > Queue Element:</emphasis> A Queue is
a message bucket. Messages are kept in the queue until a
Consumer (either Topic or Direct Consumer) connects to the
queue and fetch it. Queues can be shared or can be exclusive.
Queues whose routing key is ‘ topic’ are shared amongst Workers
of the same personality.</para>
</listitem>
</itemizedlist>
</listitem>
</itemizedlist>
<figure >
<title > RabbitMQ</title>
<mediaobject >
@@ -149,33 +150,33 @@
</imageobject>
</mediaobject>
</figure>
<para > <guilabel > RPC Calls</guilabel> </para>
<para > The diagram below shows the message flow during an rp.call
operation:</para>
<orderedlist >
<listitem >
<para > a Topic Publisher is instantiated to send the message
request to the queuing system; immediately before the
publishing operation, a Direct Consumer is instantiated to
wait for the response message.</para>
</listitem>
<listitem >
<para > once the message is dispatched by the exchange, it is
fetched by the Topic Consumer dictated by the routing key
(such as ‘ topic.host’ ) and passed to the Worker in charge of
the task.</para>
</listitem>
<listitem >
<para > once the task is completed, a Direct Publisher is
allocated to send the response message to the queuing
system.</para>
</listitem>
<listitem >
<para > once the message is dispatched by the exchange, it is
fetched by the Direct Consumer dictated by the routing key
(such as ‘ msg_id’ ) and passed to the Invoker.</para>
</listitem>
</orderedlist>
<para > <guilabel > RPC Calls</guilabel> </para>
<para > The diagram below shows the message flow during an rp.call
operation:</para>
<orderedlist >
<listitem >
<para > a Topic Publisher is instantiated to send the message
request to the queuing system; immediately before the
publishing operation, a Direct Consumer is instantiated to
wait for the response message.</para>
</listitem>
<listitem >
<para > once the message is dispatched by the exchange, it is
fetched by the Topic Consumer dictated by the routing key
(such as ‘ topic.host’ ) and passed to the Worker in charge of
the task.</para>
</listitem>
<listitem >
<para > once the task is completed, a Direct Publisher is
allocated to send the response message to the queuing
system.</para>
</listitem>
<listitem >
<para > once the message is dispatched by the exchange, it is
fetched by the Direct Consumer dictated by the routing key
(such as ‘ msg_id’ ) and passed to the Invoker.</para>
</listitem>
</orderedlist>
<figure >
<title > RabbitMQ</title>
<mediaobject >
@@ -184,21 +185,21 @@
</imageobject>
</mediaobject>
</figure>
<para > <guilabel > RPC Casts</guilabel> </para>
<para > The diagram below the message flow during an rp.cast
operation:</para>
<orderedlist >
<listitem >
<para > A Topic Publisher is instantiated to send the message
request to the queuing system.</para>
</listitem>
<listitem >
<para > Once the message is dispatched by the exchange, it is
fetched by the Topic Consumer dictated by the routing key
(such as ‘ topic’ ) and passed to the Worker in charge of the
task.</para>
</listitem>
</orderedlist>
<para > <guilabel > RPC Casts</guilabel> </para>
<para > The diagram below the message flow during an rp.cast
operation:</para>
<orderedlist >
<listitem >
<para > A Topic Publisher is instantiated to send the message
request to the queuing system.</para>
</listitem>
<listitem >
<para > Once the message is dispatched by the exchange, it is
fetched by the Topic Consumer dictated by the routing key
(such as ‘ topic’ ) and passed to the Worker in charge of the
task.</para>
</listitem>
</orderedlist>
<figure >
<title > RabbitMQ</title>
<mediaobject >
@@ -207,152 +208,150 @@
</imageobject>
</mediaobject>
</figure>
<para > <guilabel > AMQP Broker Load</guilabel> </para>
<para > At any given time the load of a message broker node running
either Qpid or RabbitMQ is function of the following
parameters:</para>
<itemizedlist >
<listitem >
<para > Throughput of API calls: the number of API calls (more
precisely rpc.call ops) being served by the OpenStack cloud
dictates the number of direct-based exchanges, related
queues and direct consumers connected to them.</para>
</listitem>
<listitem >
<para > Number of Workers: there is one queue shared amongst
workers with the same personality; however there are as many
exclusive queues as the number of workers; the number of
workers dictates also the number of routing keys within the
topic-based exchange, which is shared amongst all
workers.</para>
</listitem>
</itemizedlist>
<para > The figure below shows the status of a RabbitMQ node after
Nova components’ bootstrap in a test environment. Exchanges and
queues being created by Nova components are:</para>
<itemizedlist >
<listitem >
<para > Exchanges</para>
</listitem>
</itemizedlist>
<orderedlist >
<listitem >
<para > nova (topic exchange)</para>
</listitem>
</orderedlist>
<itemizedlist >
<listitem >
<para > Queues</para>
</listitem>
</itemizedlist>
<orderedlist >
<listitem >
<para > compute.phantom (phantom is hostname)</para>
</listitem>
<listitem >
<para > compute</para>
</listitem>
<listitem >
<para > network.phantom (phantom is hostname)</para>
</listitem>
<listitem >
<para > network</para>
</listitem>
<listitem >
<para > scheduler.phantom (phantom is hostname)</para>
</listitem>
<listitem >
<para > scheduler</para>
</listitem>
</orderedlist>
<para > <guilabel > RabbitMQ Gotchas</guilabel> </para>
<para > Nova uses Kombu to connect to the RabbitMQ environment.
Kombu is a Python library that in turn uses AMQPLib, a library
that implements the standard AMQP 0.8 at the time of writing.
When using Kombu, Invokers and Workers need the following
parameters in order to instantiate a Connection object that
connects to the RabbitMQ server (please note that most of the
following material can be also found in the Kombu documentation;
it has been summarized and revised here for sake of
clarity):</para >
<itemized list >
<listitem >
<para > <emphasis role= "bold" > Hostname:</emphasis> The hostname
to the AMQP server.</para >
</ listitem>
<listitem >
<para > <emphasis role= "bold" > Userid:</emphasis> A valid
username used to authenticate to the server.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Password:</emphasis> The password
<para > <guilabel > AMQP Broker Load</guilabel> </para>
<para > At any given time the load of a message broker node running
either Qpid or RabbitMQ is function of the following
parameters:</para>
<itemizedlist >
<listitem >
<para > Throughput of API calls: the number of API calls (more
precisely rpc.call ops) being served by the OpenStack cloud
dictates the number of direct-based exchanges, related queues
and direct consumers connected to them.</para>
</listitem>
<listitem >
<para > Number of Workers: there is one queue shared amongst
workers with the same personality; however there are as many
exclusive queues as the number of workers; the number of
workers dictates also the number of routing keys within the
topic-based exchange, which is shared amongst all
workers.</para>
</listitem>
</itemizedlist>
<para > The figure below shows the status of a RabbitMQ node after
Nova components’ bootstrap in a test environment. Exchanges and
queues being created by Nova components are:</para>
<itemizedlist >
<listitem >
<para > Exchanges</para>
</listitem>
</itemizedlist>
<orderedlist >
<listitem >
<para > nova (topic exchange)</para>
</listitem>
</orderedlist>
<itemizedlist >
<listitem >
<para > Queues</para>
</listitem>
</itemizedlist>
<orderedlist >
<listitem >
<para > compute.phantom (phantom is hostname)</para>
</listitem>
<listitem >
<para > compute</para>
</listitem>
<listitem >
<para > network.phantom (phantom is hostname)</para>
</listitem>
<listitem >
<para > network</para>
</listitem>
<listitem >
<para > scheduler.phantom (phantom is hostname)</para>
</listitem>
<listitem >
<para > scheduler</para>
</listitem>
</orderedlist>
<para > <guilabel > RabbitMQ Gotchas</guilabel> </para>
<para > Nova uses Kombu to connect to the RabbitMQ environment. Kombu
is a Python library that in turn uses AMQPLib, a library that
implements the standard AMQP 0.8 at the time of writing. When
using Kombu, Invokers and Workers need the following parameters in
order to instantiate a Connection object that connects to the
RabbitMQ server (please note that most of the following material
can be also found in the Kombu documentation; it has been
summarized and revised here for sake of clarity):</para>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Hostname:</emphasis> The hostname to
the AMQP server.</para>
</listitem >
<listitem >
<para > <emphasis role= "bold" > Userid:</emphasis> A valid username
used to authenticate to the server.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Virtual_host :</emphasis> The name
of the virtual host to work with. This virtual host must exist
on the server, and the user must have access to it. Default is
</listitem>
<listitem >
<para > <emphasis role= "bold" > Password :</emphasis> The password
used to authenticate to the server.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Virtual_host:</emphasis> The name of
the virtual host to work with. This virtual host must exist on
the server, and the user must have access to it. Default is
“/”.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Port:</emphasis> The port of the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Port:</emphasis> The port of the
AMQP server. Default is 5672 (amqp).</para>
</listitem>
</itemizedlist>
<para > The following parameters are default:</para>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Insist:</emphasis> insist on
</listitem>
</itemizedlist>
<para > The following parameters are default:</para>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Insist:</emphasis> insist on
connecting to a server. In a configuration with multiple
load-sharing servers, the Insist option tells the server that
the client is insisting on a connection to the specified
server. Default is False.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Connect_timeout:</emphasis> the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Connect_timeout:</emphasis> the
timeout in seconds before the client gives up connecting to
the server. The default is no timeout.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > SSL:</emphasis> use SSL to connect
</listitem>
<listitem >
<para > <emphasis role= "bold" > SSL:</emphasis> use SSL to connect
to the server. The default is False.</para>
</listitem>
</itemizedlist>
<para > More precisely Consumers need the following
parameters:</para >
<itemized list >
<listitem >
<para > <emphasis role= "bold" > Connection:</emphasis> the above
</listitem>
</itemizedlist>
<para > More precisely Consumers need the following parameters:</para>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Connection:</emphasis> the above
mentioned Connection object.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Queue:</emphasis> name of the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Queue:</emphasis> name of the
queue.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Exchange:</emphasis> name of the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Exchange:</emphasis> name of the
exchange the queue binds to.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Routing_key:</emphasis> the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Routing_key:</emphasis> the
interpretation of the routing key depends on the value of the
exchange_type attribute.</para>
</listitem>
</itemizedlist>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Direct exchange:</emphasis> if the
</listitem>
</itemizedlist>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Direct exchange:</emphasis> if the
routing key property of the message and the routing_key
attribute of the queue are identical, then the message is
forwarded to the queue.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Fanout
exchange:</emphasis> messages are forwarded to the queues bound
the exchange, even if the binding does not have a key.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Topic exchange:</emphasis> if the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Fanout exchange:</emphasis> messages
are forwarded to the queues bound the exchange, even if the
binding does not have a key.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Topic exchange:</emphasis> if the
routing key property of the message matches the routing key of
the key according to a primitive pattern matching scheme, then
the message is forwarded to the queue. The message routing key
@@ -362,79 +361,77 @@
matches zero or more words. For example ”.stock.#” matches the
routing keys “usd.stock” and “eur.stock.db” but not
“stock.nasdaq”.</para>
</listitem>
</itemizedlist>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Durable:</emphasis> this flag
</listitem>
</itemizedlist>
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > Durable:</emphasis> this flag
determines the durability of both exchanges and queues;
durable exchanges and queues remain active when a RabbitMQ
server restarts. Non-durable exchanges/queues (transient
exchanges/queues) are purged when a server restarts. It is
worth noting that AMQP specifies that durable queues cannot
bind to transient exchanges. Default is True.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Auto_delete:</emphasis> if set, the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Auto_delete:</emphasis> if set, the
exchange is deleted when all queues have finished using it.
Default is False.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Exclusive:</emphasis> exclusive
</listitem>
<listitem >
<para > <emphasis role= "bold" > Exclusive:</emphasis> exclusive
queues (such as non-shared) may only be consumed from by the
current connection. When exclusive is on, this also implies
auto_delete. Default is False.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Exchange_type:</emphasis> AMQP
</listitem>
<listitem >
<para > <emphasis role= "bold" > Exchange_type:</emphasis> AMQP
defines several default exchange types (routing algorithms)
that covers most of the common messaging use cases.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold"
> Auto_ack:</emphasis> acknowledgement is handled automaticall y
once messages are received. By default auto_ack is set to
False, and the receiver is required to manually handle
acknowledgment.</para >
</ listitem>
<listitem >
<para > <emphasis role= "bold" > No_ack:</emphasis> it disable
</listitem>
<listitem >
<para > <emphasis role= "bold" > Auto_ack:</emphasis> acknowledgement
is handled automatically once messages are received. B y
default auto_ack is set to False, and the receiver is required
to manually handle acknowledgment.</para>
</listitem >
<listitem >
<para > <emphasis role= "bold" > No_ack:</emphasis> it disable
acknowledgement on the server-side. This is different from
auto_ack in that acknowledgement is turned off altogether.
This functionality increases performance but at the cost of
reliability. Messages can get lost if a client dies before it
can deliver them to the application.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Auto_declare:</emphasis> if this is
</listitem>
<listitem >
<para > <emphasis role= "bold" > Auto_declare:</emphasis> if this is
True and the exchange name is set, the exchange will be
automatically declared at instantiation. Auto declare is on by
default. Publishers specify most the parameters of Consumers
(such as they do not specify a queue name), but they can also
specify the following:</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > Delivery_mode:</emphasis> the
default delivery mode used for messages. The value is an
integer. The following delivery modes are supported by
RabbitMQ:</para >
</list item>
</ itemizedlist>
<itemized list >
<listitem >
<para > <emphasis role= "bold" > 1 or “transient”:</emphasis> the
</listitem>
<listitem >
<para > <emphasis role= "bold" > Delivery_mode:</emphasis> the default
delivery mode used for messages. The value is an integer. The
following delivery modes are supported by RabbitMQ:</para>
</listitem >
</itemizedlist >
<itemizedlist >
<listitem >
<para > <emphasis role= "bold" > 1 or “transient”:</emphasis> the
message is transient. Which means it is stored in memory only,
and is lost if the server dies or restarts.</para>
</listitem>
<listitem >
<para > <emphasis role= "bold" > 2 or “persistent”:</emphasis> the
</listitem>
<listitem >
<para > <emphasis role= "bold" > 2 or “persistent”:</emphasis> the
message is persistent. Which means the message is stored both
in-memory, and on disk, and therefore preserved if the server
dies or restarts.</para>
</listitem>
</itemizedlist>
<para > The default value is 2 (persistent). During a send
operation, Publishers can override the delivery mode of messages
so that, for example, transient messages can be sent over a
durable queue.</para>
</listitem>
</itemizedlist>
<para > The default value is 2 (persistent). During a send operation,
Publishers can override the delivery mode of messages so that, for
example, transient messages can be sent over a durable
queue.</para>
</chapter>