Enable notification messaging pools

Adds a 'notification_pool' option in the [listener] configuration
section. This is an alternative to adding a separate topic that will be
recommended for the Newton release following more testing; for now it is
an optional alternative.

To enable:

  [listener]
  notifications_pool = searchlight-listener

You will see a rabbitmq queue named searchlight-listener. Note that your
topic must match the service notifications_topic setting, whether it be
the default of 'notifications' or 'searchlight_indexer'.

Closes-Bug: #1583215
Change-Id: Ie40876e88059d0326efa0b4abaa76b8b64f0b86b

doc/source/authoring-plugins.rst

@@ -110,6 +110,35 @@ make it output notifications by editing ``/etc/neutron/neutron.conf`` and
 adding under the ``[DEFAULT]`` section::
 
     notification_driver = messaging
+
+There are then two ways to configure the service to send notifications that
+Searchlight can receive. The recommended method is to use notification pools,
+touched on in the `messaging documentation`_.
+
+.. _`messaging documentation`: http://docs.openstack.org/developer/oslo.messaging/notification_listener.html
+
+Notification pools
+##################
+
+A notification messaging pool allows additional listeners to receive
+messages on an existing topic. By default, Openstack services send notification
+messages to an oslo.messaging 'topic' named `notifications`. To view these
+notifications while still allowing ``searchlight-listener`` or Ceilometer's
+agent to continue to recieve them, you may use the utility script in
+``test-scripts/listener.py``::
+
+    . ~/devstack/openrc admin admin
+    # If your rabbitmq user/pass are not the same as for devstack, you
+    # can set RABBIT_PASSWORD and/or RABBIT_USER
+    ./test-scripts/listener.py neutron test-notifications
+
+Adding a separate topic
+#######################
+
+In the same config file (``/etc/neutron/neutron.conf``) the following line
+(again, under the ``[DEFAULT]`` section) will cause neutron to output
+notifications to a topic named ``searchlight_indexer``::
+
     notification_topics = searchlight_indexer
 
 .. note::
@@ -129,6 +158,8 @@ tool; there is also a utility script in ``test-scripts/listener.py`` that
 will listen for notifications::
 
     . ~/devstack/openrc admin admin
+    # If your rabbitmq user/pass are not the same as for devstack, you
+    # can set RABBIT_PASSWORD and/or RABBIT_USER
     ./test-scripts/listener.py neutron
 
 .. note::
@@ -141,6 +172,9 @@ will listen for notifications::
         # to
         topic = 'my_test_topic'
 
+Using the results
+#################
+
 Issuing various commands (``neutron net-create``, ``neutron net-update``,
 ``neutron net-delete``) will cause ``listener.py`` to receive notifications.
 Usually the notifications with ``event_type`` ending ``.end`` are the ones of

doc/source/dev-environment.rst

@@ -197,6 +197,14 @@ services. Each integrated service may require additional configuration
 settings. For example, typically, you will need to add the
 ``searchlight_indexer`` notification topic to each service's configuration.
 
+.. note:: In Newton, notification messaging pools will become the default
+          recommended configuration, which does not require changing any
+          service configurations beyond enabling notifications. To enable
+          the use of a messaging pool instead of separate topics, add the
+          ``notifications_pool`` option in the ``listener`` section of
+          ``searchlight.conf``. In this case, ensure that services are using
+          the same notification topic as searchlight.
+
 See :ref:`searchlight-plugins` for plugin installation and general
 configuration information.
 

doc/source/plugins.rst

@@ -265,6 +265,16 @@ to add the ``searchlight_indexer`` notification topic to each service's
 configuration in order for Searchlight to receive incremental updates from
 that service.
 
+.. note:: In Newton, notification messaging pools will become the default
+          recommended configuration, which does not require changing any
+          service configurations beyond enabling notifications.
+
+          To enable the use of notification pools instead of a separate
+          topic, add the ``notifications_pool`` option in the ``listener``
+          section of ``searchlight.conf``. There is no need in this case
+          to add an additional topic. Messages will begin to be delivered
+          to the pool after ``searchlight-listener`` has started.
+
 Please review each plugin's documentation for more information:
 
 .. toctree::

searchlight/listener.py

@@ -36,6 +36,9 @@ listener_opts = [
                min=1,
                help='Number of workers for notification service. A single '
                'notification agent is enabled by default.'),
+    cfg.StrOpt('notifications_pool',
+               help='Use an oslo.messaging pool, which can be an alternative '
+               'to multiple topics. ')
 ]
 
 CONF = cfg.CONF
@@ -122,7 +125,8 @@ class ListenerService(os_service.Service):
             transport,
             targets,
             endpoints,
-            executor='threading')
+            executor='threading',
+            pool=CONF.listener.notifications_pool)
 
         listener.start()
         self.listeners.append(listener)

test-scripts/listener.py

@@ -31,9 +31,9 @@ except ImportError:
 
 
 topic = 'searchlight_indexer'
-password = os.environ.get('OS_PASSWORD')
+password = os.environ.get('RABBIT_PASSWORD', os.environ.get('OS_PASSWORD'))
 host = urlparse.urlparse(os.environ.get('OS_AUTH_URL')).hostname
-username = 'stackrabbit'
+username = os.environ.get('RABBIT_USER', 'stackrabbit')
 
 
 class EP(object):