This patch switches the kafka python client from kafka-python to
confluent-kafka due to documented threading issues with the
kafka-python consumer and the recommendation to use multiplrocessing.
The confluent-kafka client leverages the high performance librdkafka
C client and is safe for multiple thread use.
* switches to confluent-kafka library
* revises consumer and producer message operations
* utilizes event.tpool method for confluent-kafka blocking calls
* updates unit tests
* adds kafka specific timeouts for functional tests
* adds release note
This patch addresses a number of issues that prevented the functional
tests from running. The functional tests now execute and can complete
succesfully. At times, the test will fail (noticiably in CI) indicating
an underlying issue with consumer interaction with the kafka server.
It would be beneficial to merge this patch as it provides repeatability
and visibility for driver-kafka server integration to facilitate
additional debugging and testing.
* removes use of deprecated get_transport
* override consumer_group for each test
* changed to synchronous send
* update to kafka 1.0.0 server
The kafka functional tests don't use our setup framework but hardcode
the kafka url.
The kafka driver is broken and currently try to be loaded with other
driver tests (and break py3 gate job).
This change remove the hardcoded stuff and uses the normal functional
tests setup to check kafka.
This avoids kafka tests to be ran on other driver tests job.
It also creates the tox target for functional tests and use
pifpaf to setup kafka.
As today, the functional test suite doesn't work all.