Support listener pooling in keystone listener

some oslo.messaging transport backends support listener pooling,
when the listener declares a pool it belongs, and each message of
the topic is delivered once to each pool (plus to 'default' pool for
those that have not declared a pool).
In particular, rabbitmq and kafka support it.
This mechanism should be preferrable over setting a separate topic
for barbican to listen notifications on.

This patch adds a new config option `[keystone_notifications]pool_name`
(None by default for backward compatibility) that can be used to set
up barbican-keystone-listener to create a separate pool for its
listeners.

For more details see oslo.messaging docs
https://docs.openstack.org/oslo.messaging/latest/reference/notification_listener.html

Change-Id: Ie011266f5ebe03bc6053bfe68e2bee27e07ea11c

barbican/common/config.py

@@ -183,7 +183,22 @@ ks_queue_opts = [
                         "will be dispatched to one of the servers in a "
                         "round-robin fashion. That's why Barbican service "
                         "should have its own dedicated notification queue so "
-                        "that it receives all of Keystone notifications.")),
+                        "that it receives all of Keystone notifications. "
+                        "Alternatively if the chosen oslo.messaging backend "
+                        "supports listener pooling (for example rabbitmq), "
+                        "setting a non-default 'pool_name' option "
+                        "should be preferred.")),
+    cfg.StrOpt('pool_name',
+               help=u._("Pool name for notifications listener. "
+                        "Setting this to a distinctive value will allow "
+                        "barbican notifications listener to receive its own "
+                        "copy of all messages from the topic without "
+                        "without interfering with other services listening "
+                        "on the same topic. This feature is supported only "
+                        "by some oslo.messaging backends "
+                        "(in particilar by rabbitmq) and for those it is "
+                        "preferrable to use it instead of separate "
+                        "notification topic for barbican.")),
     cfg.BoolOpt('allow_requeue', default=False,
                 help=u._('True enables requeue feature in case of notification'
                          ' processing error. Enable this only when underlying '

barbican/queue/__init__.py

@@ -106,10 +106,13 @@ def get_notification_server(targets, endpoints, serializer=None):
     """
     allow_requeue = getattr(getattr(CONF, KS_NOTIFICATIONS_GRP_NAME),
                             'allow_requeue')
+    pool_name = getattr(getattr(CONF, KS_NOTIFICATIONS_GRP_NAME),
+                        'pool_name')
     TRANSPORT._require_driver_features(requeue=allow_requeue)
     dispatcher = notify_dispatcher.NotificationDispatcher(endpoints,
                                                           serializer)
     # we don't want blocking executor so use eventlet as executor choice
     return listener.NotificationServer(TRANSPORT, targets, dispatcher,
                                        executor='eventlet',
+                                       pool=pool_name,
                                        allow_requeue=allow_requeue)

releasenotes/notes/keystone-listener-pooling-a4fb0dde9e25a21f.yaml

@@ -0,0 +1,14 @@
+---
+features:
+  - |
+    It is now possible for barbican-keystone-listener to listen on the same
+    standard notification topic without interfering with other services by
+    using the notification listener pools feature of oslo.messaging.
+    To use it, set the new ``[keystone_notifications]pool_name`` option
+    to some unique value (but the same for all instances of
+    barbican-keystone-listener service).
+    This feature is available only for those messaging transports of
+    oslo.messaging that support it.
+    At the moment those are rabbitmq and kafka.
+    For more details see
+    `oslo.messagind docs <https://docs.openstack.org/oslo.messaging/latest/reference/notification_listener.html>`_