openstack
/
ceilometer
Code Issues Proposed changes

Sync kombu_reconnect_delay from Oslo 

When reconnecting to a RabbitMQ cluster
with mirrored queues in use, the attempt to release the
connection can hang "indefinitely" somewhere deep down
in Kombu.  Blocking the thread for a bit prior to
release seems to kludge around the problem where it is
otherwise reproduceable.
The value 5.0 fits for low perfomance environments as well.

Cherry-picked from Oslo.messaging:
fcd51a67d18a9e947ae5f57eafa43ac756d1a5a8
Related-bug: #856764

Change-Id: Ifadda4dd9122df9ccb4ecf560ce3db3e38adf2b9
Signed-off-by: Bogdan Dobrelya <bdobrelia@mirantis.com>
This commit is contained in:
Bogdan Dobrelya 2014-05-26 13:28:40 +03:00
parent 94c12fd76c
commit 06eb8bc532
1 changed files with 15 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
ceilometer/openstack/common/rpc/impl_kombu.py
View File

@ -52,6 +52,10 @@ kombu_opts = [
default='',
help=('SSL certification authority file '
'(valid only if SSL enabled)')),
cfg.FloatOpt('kombu_reconnect_delay',
default=1.0,
help='How long to wait before reconnecting in response to an '
'AMQP consumer cancel notification.'),
cfg.StrOpt('rabbit_host',
default='localhost',
help='The RabbitMQ broker address where a single node is used'),
@ -495,6 +499,17 @@ class Connection(object):
LOG.info(_("Reconnecting to AMQP server on "
"%(hostname)s:%(port)d") % params)
try:
# NOTE(bogdando): when reconnecting to a RabbitMQ cluster
# with mirrored queues in use, the attempt to release the
# connection can hang "indefinitely" somewhere deep down
# in Kombu. Blocking the thread for a bit prior to
# release seems to kludge around the problem where it is
# otherwise reproduceable.
if self.conf.kombu_reconnect_delay > 0:
LOG.info(_("Delaying reconnect for %1.1f seconds...") %
self.conf.kombu_reconnect_delay)
time.sleep(self.conf.kombu_reconnect_delay)
self.connection.release()
except self.connection_errors:
pass