Currently Kafka driver for an oslo.messaging uses kafka-python==0.9.5
and mostly broken. This package version supports only low level Kafka
producer and consumer API which are marked as deprecated now [1]. Using
of these interfaces bring a big concern to the message processing,
because current KafkaConsumer has not any consuming coordination. This
fact causes a message duplication for the several consumers of one
topic. This behavior is specific to Ceilometer and services which read
and process notifications from other services.
New version of kafka-python allows to use async thread safe message
producers and coordinated consumers [1].
[1] http://kafka-python.readthedocs.io/en/master/changelog.html#feb-15-2016
The driver is currently experimental, python-kafka<1.0.0 API have major
issue described above that can't make the oslo.messaging driver works,
so we prefer having a working driver with a non-synced dependencies, that the
reverse.
Co-Authored-By: Mehdi Abaakouk <sileht@redhat.com>
Change-Id: I29862ed7bf56b9d8878fa8e9fb1cbd9d643908a4
retry is in test-requirements.txt and requirements.txt
This change removes it from test-requirements.txt
Change-Id: I5236a759bdefe2b47e6d2ea84472f5cbceb45c9a
List of redis sentinel hosts is now supported in order to
use automatic failover when redis master goes down.
Change-Id: I5fad4c9b6c6aea4f8f382f7469899a7d05c068c1
Closes-Bug: #1518292
Adding a driver for Apache Kafka connection, supporting
notification via Kafka. This driver is experimental
until having functional and integration tests
Change-Id: I7a5d8e3259b21d5e29ed3b795d04952e1d13ad08
Implements: blueprint adding-kafka-support
Back in liberty we marked this driver as deprecated. This patch removes
it from the tree. The patch also removes tests, options and other
references in the documentation. Note that one script is being kept
because it's required by the amqp driver.
Depends-On: If4b1773334e424d1f4a4e112bd1f10aca62682a9
Change-Id: I4a9cba314c4a2f24307504fa7b5427424268b114
This will allow us to find potential security issues, such as those fixed by
52e624891f and
c4a7ac0b65 .
Change-Id: I21aa0ca79232784069e55da46920eb43250d8939
Basic functional and unit tests for zmq driver.
Note as the zmq driver is directly dependent on eventlet, this
change also updates the notify logger tests to remove the
direct dependency on threading which was being monkey patched,
causing test failures.
As the zmq driver has a direct dependency on eventlet, tests are
skipped under py3.
Co-Authored-By: Kapil Thangavelu <kapil.thangavelu@canonical.com>
Co-Authored-By: Edward Hope-Morley <edward.hope-morley@canonical.com>
Change-Id: I93b8b2e92d0f2a353d3357a5e61f6d472ec84944
Partial-bug: #1302941
Basic unit tests for in-memory (localhost), ring and Redis based
matchmakers.
Redis tests are opportunitic based on a redis-server being installed
and running.
Test are currently skipped under py3 due to direct dependency on
eventlet.
Co-Authored-By: Kapil Thangavelu <kapil.thangavelu@canonical.com>
Change-Id: I59c4d095033370ab94f645af0d50e2502530f816
Partial-bug: #1302941