kafka doesn't play well with //, each times a new partition/topic is
created, kafka rebalances things across brokers and messages are not
delivered at the expected time.
So this change disable the parallelizism for functional tests for kafka
only.
Change-Id: Ia497334efcd4685850ac1529a7fc38073dae7cd1
- kombu4 wraps recoverable errors as OperationalErrors rather than
raising amqp errors
- also, raise a recoverable error and redeclare if for some reason a
message is double acknowledged... previously, this was hidden.
- ensure socket is not none
- use connect method to ensure connection
Depends-On: I9f980b51901ac31599b9651633956ad2eea6a1ac
Change-Id: I73958c8057353a2eefe1baaa7a41148193d507f7
Now that there exists only a gate job for Python 3.5 and not 3.4,
we should remove those references to the 3.4 that is untested.
Change-Id: I2a00ead626d9eced96487ee82b5d7857126d8355
Currently Kafka driver for an oslo.messaging uses kafka-python==0.9.5
and mostly broken. This package version supports only low level Kafka
producer and consumer API which are marked as deprecated now [1]. Using
of these interfaces bring a big concern to the message processing,
because current KafkaConsumer has not any consuming coordination. This
fact causes a message duplication for the several consumers of one
topic. This behavior is specific to Ceilometer and services which read
and process notifications from other services.
New version of kafka-python allows to use async thread safe message
producers and coordinated consumers [1].
[1] http://kafka-python.readthedocs.io/en/master/changelog.html#feb-15-2016
The driver is currently experimental, python-kafka<1.0.0 API have major
issue described above that can't make the oslo.messaging driver works,
so we prefer having a working driver with a non-synced dependencies, that the
reverse.
Co-Authored-By: Mehdi Abaakouk <sileht@redhat.com>
Change-Id: I29862ed7bf56b9d8878fa8e9fb1cbd9d643908a4
The current zeromq target name are too long and reach the limitation
of 127 chars of the sheband of the virtualenv binary.
A target name must not be more than 22 chars to run in gate with
the tox job template.
This renames them to a shorter one.
It also creates aliases for the legacy jobs that we will remove
when the tox jobs are ready.
Change-Id: I5cbc368e7ea252f8c0c949c8e7ee3a70a560db22
Restore static direct connections which may be useful
for services running RPC on controllers only and talking
to agents via other means (ssh for example).
Change-Id: Icbe45978cb4a8ba5db74e5593ea7be23cba3f44e
The kafka functional tests don't use our setup framework but hardcode
the kafka url.
The kafka driver is broken and currently try to be loaded with other
driver tests (and break py3 gate job).
This change remove the hardcoded stuff and uses the normal functional
tests setup to check kafka.
This avoids kafka tests to be ran on other driver tests job.
It also creates the tox target for functional tests and use
pifpaf to setup kafka.
As today, the functional test suite doesn't work all.
Change-Id: Ie6af35321905a04b38e549bf7268dd40543d9e3a
During the tox refactoring for the constraint stuffs [1], we pass
all target timeout from 60s to 30s, making functional tests to fail.
This change just restore the default, even the py27 job doesn't need
such small timeout.
[1] 78f1137805
Change-Id: I515786a2e8b7cf8d17b5ee970b13a68f565965ea
Adding constraints support to libraries is slightly more complex than
services as the libraries themselves are listed in upper-constraints.txt
which leads to errors that you can't install a specific version and a
constrained version.
This change adds constraints support by also adding a helper script to
edit the constraints to remove oslo.messaging.
Change-Id: I8be883215f27abb58d15b85e8542cbdf32000bac
This commit updates the setup environment for the driver functional
tests to support either a qpidd (broker) or qdrouterd (router)
backend.
This patch:
* configures and runs either qpidd or qdrouterd backend
* removes support of older qpidd versions
* skips batch notification test for qdrouterd backend
Change-Id: Ifcdc187bae4a28dd29c1faa73e239d7e4d77015d
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.
Change-Id: I8871f4e17ab806c00661d0c960a216d0a3fc5449
Now that the bandit linter runs without a bandit.yaml specified,
it should exclude the tests directory similar to how it did before
in the bandit.yaml.
The reason this has not caused a problem yet is because bandit is
still finding a config bandit.yaml in .tox/pep8/etc/bandit/bandit.yaml.
But in a newer version of bandit this will not be the case.
Change-Id: I6cdceaff2aa906e813f1d1e993f60e66358946ee
With latest bandit version, The bandit.yaml is only necessary
for advanced tweaking. We can just use the defaults.
Change-Id: Ib1cb84eb428baa98508e431268fb8bd1bbf2d505
Somehow we sneaked in a change to ignore pep8 on tests/ directory.
Let's cleanup the code and re-enable pep8
Closes-Bug: #1540981
Change-Id: Ic002134a0d0c498991b062466e00068d6ac1f1ab
Gnocchi performs better if measurements are write in batch
When Ceilometer is used with Gnocchi, this is not possible.
This change introduce a new notification listener that allows that.
On the driver side, a default batch implementation is provided.
It's just call the legacy poll method many times.
Driver can override it to provide a better implementation.
For example, kafka handles batch natively and take benefit of this.
Change-Id: I16184da24b8661aff7f4fba6196ecf33165f1a77
Back in liberty we marked this driver as deprecated. This patch removes
it from the tree. The patch also removes tests, options and other
references in the documentation. Note that one script is being kept
because it's required by the amqp driver.
Depends-On: If4b1773334e424d1f4a4e112bd1f10aca62682a9
Change-Id: I4a9cba314c4a2f24307504fa7b5427424268b114
This builds a stripped down tox target for pep8 that doesn't need a
giant venv with all the things. Works fast and lean, and makes julien
fries.
Change-Id: Id5b7671fb7f2b8cbf88745fd12f9238b3c0bb2dd
This will allow us to find potential security issues, such as those fixed by
52e624891f and
c4a7ac0b65 .
Change-Id: I21aa0ca79232784069e55da46920eb43250d8939
In this patch new driver implementation added and registered
in setup.cfg, integrated ith tox functional tests.
Implements: bp rabbit-pika
Depends-On: I7bda78820e657b1e97bf888d4065a917eb317cfb
Change-Id: I40842a03ce73d171644c362e3abfca2990aca58a
Previously, the AMQP 1.0 driver did not pass the proper hostname to
pyngus. This prevented GSSAPI authentication from working properly.
Change-Id: Ibc6678e7cbae6dd5108d1650dbb8ddf837aa3648
Closes-Bug: #1503258
Pyngus 2.0 includes a new API for configuring SASL credentials.
Previous versions of Pyngus did not provide this API - the driver had
to invoke Proton APIs in order to configure user credentials. Moving
to the Pyngus API will preserve compatibility with older versions of
Proton, since the next release of Proton wil be changing its SASL API.
Pyngus 2.0 also adds strict enforcement of callback re-entrancy
constrants. This patch fixes some bad driver reentrancy violations.
Closes-bug: #1473515
Change-Id: Iddccefd3ee3c9092c086fc54e3810f78d5df9338
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names. It also adds a local
hacking check to prevent regressions, while disabling style checks in
the "tests" directory, which will be deleted in a subsequent patch.
The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.
Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.
Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages
Change-Id: Iad2ae13c771c99861ca9493c6ab10edcfbd8684a
Mehdi Abaakouk