15cd99050c added this alias
to workaround an issue with neutron using the internal
path in liberty. That was fixed in neutron in mitaka:
225663888e71d44e9732951f515da925b8799b74
The alias was left for liberty, but liberty is end of life
now so we can remove this.
Change-Id: I145921b98d48e03df38ebcbaebc1a4375911f9ff
Generating the payload of a notificatiton might be expensive due
to database access. This expensive step is unnecessary if the
notification driver is set to noop because it means that the
generated payload will not be sent.
This patch makes it possible for the user of the notifier to check
if the notifier will emit any notification before the payload
is generated.
Cinder already implemented similar behavior in
I77f655d3ef90088ce71304da5d4ea7b543991e90 and nova also plans
to do it in I3e6741d59df49e1e58409314008c2ed609fdedc1
Change-Id: Ib992f5c20fef85224fb00823e1d8d9c6cff19bec
1.As mentioned in [1], we should avoid using six.iteritems to achieve iterators.
We can use dict.items instead, as it will return iterators in PY3 as well.
And dict.items/keys will more readable. 2.In py2, the performance about
list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
Change-Id: Ia235afc3532f62f265f91ca46d2306c72fc2a2a2
'Notification Listener' doesn't work.
I fixed the example which is described in developer guide to work fine.
These are the main points of this correction.
* specify namespace for NotificationFilter.
* set 2 blank lines between class definitions according to E302 of pep8.
* change argument of 'pool' to keyward arguemnt, because 4th argument of
'get_notification_listener' requires the name of message executor.
Change-Id: I75800309f2294fe9f17ea2a2b1638de57ef2922e
Closes-Bug: #1618394
Rework the inline documentation for executors, RPC servers and
clients, notifiers and notification listeners for clarity and flow.
Change-Id: If4f1db853a7fc85340177fd2c9c43a479d72459d
When data[k] is None during match check, we are getting annoying
TypeError exceptions like: "TypeError: expected string or buffer".
This patch adds a check that data[k] is something the regex can do
matching.
Change-Id: I75c8f602a59a90aeb5a6973f2e9e25708194ad78
This was previously undocumented. Most of the information was taken from the
docstring for get_transport
Change-Id: I92652f4de9dd1e59ae5c3f9ce39e5ee808f24c7d
1) try to catch all possible exceptions during message acknowledging
and dispatching to send ack/nack when we can to do it
2) improve logging in case of exceptions during message acknowledging
and dispatching
Depends-On: I2d230d49e5aff6ab4d84ab8c3d2834f85e3405eb
Change-Id: I41a768c5624fa2212257ce20bf9a67d09de0c4ab
fnmatch is not thread safe in python <2.7.10, let's use the safe
one in oslo.utils
Change-Id: If3d480436506ecc5f3c2fcbd51ceaac59f2b7dec
ref: https://hg.python.org/cpython/rev/fe12c34c39eb
1) Add MessageHandler base interface for on_incoming_callback replacement
2) Move message_handler parameter form Listener's __init__() to start()
3) Remove wait method from listener
Change-Id: Id414446817e3d2ff67b815074d042a9ce637ec24
The transport_url may contain credentials to the message queue; For
this reason, this option should be secret for it to not leak into the
logs.
Closes-Bug: #1567233
Change-Id: I49c641a2662976d7220e4222e3c4a4d2586b1336
Current Listener interface has poll() method which return messages
To use it we need have poller thread which is located in MessageHandlerServer
But my investigations of existing driver's code shows that some implemetations have
its own thread inside for processing connection event loop. This event loop received
messages and store in queue object. And then our poller's thread reads this queue
This situation can be improved. we can remove poller's thread, remove queue object
and just call on_message server's callback from connection eventloop thread
This path provide posibility to do this for one of drivers and leave as is other drivers
Change-Id: I3e3d4369d8fdadcecf079d10af58b1e4f5616047
Dispatcher should be responsible for routing message to the
callback method of endpoint object and returning result back to the
server only. But now it is also responsible for sending reply,
ack/reque messages etc.
Also this patch makes small improvements:
1) Notification dispatcher now requeue message if endpoint raises exception
2) unstable behaviour of test_mask_passwords test is fixed
Change-Id: I5f23e23644e90919cb67f81fc306ee85c5e09974
This reverts commit bd81d09c02.
I understand that the change was supposed to fix something, but instead it broke all tests on Python 3!?
It's wrong to replace blindly json.dumps() with jsonutils.dump_as_bytes(). In oslo messaging, the result is usually used as a value in a dictionary, and then the whole dictionary is passed to a second serializer which also serialize to JSON.
Sorry, I don't understand everything, but at least I see that tests passed on py3 before the change, and started to fail with the change.
Maybe json(utils).dumps() is misused in some places, but in this case, you should write a change which only fix these specific places, not replace all calls to dumps().
Change-Id: Icd54ee8e3f5c976dfd50b4b62c7f51288649e112
Looks like there is a disconnect between the __init__
parameter 'topic' in Notifier and what we need when
we look up a driver. We should allow multiple topics
to be specified as a new topics parameter and pass
that along directly.
Change-Id: Id89957411aa219cff92fafec2f448c81cb57b3ca
Since the json.dumps() returns Unicode string On Python 3, to
ensure that the result type is bytes on Python 2 and Python 3
that if the result is used for the message body, let's replace
json.dumps() / oslo_serialization.jsonutils.dumps()
with
oslo_serialization.jsoutils.dump_as_bytes()
Change-Id: I0e0f6b715ffc4a9ad82be52e55696d032b6d0976
The mask_dict_password method is added from oslo_utils 3.4.0, and it's
about five times faster than the mask_password method, so it's better
to use mask_dict_password instead of mask_password.
Change-Id: I6629ed1232f2e6f1c72b9f13b361e2f9e14b50a4
Revert the change I556b112371bec2ec29cea4dc254bb3f9c6d2c07a: the
executor callback API was only used by the aioeventlet executor which
was just removed.
Change-Id: I1223f32594d8c1be28cc43fdd9bf102c86d75325
1. Use translation marker functions, their argument must just be a string
2. Any message with more than one variable should use named
interpolation instead of positional to allow translators
to move the variables around in the string to account for
differences in grammar and writing direction.
3. String interpolation should be delayed to be handled by the logging
code, rather than being done at the point of the logging call.
For more details, please refert to oslo.i18n guideline [1]
Note: this commit doesn't touch test code.
[1] http://docs.openstack.org/developer/oslo.i18n/guidelines.html
Change-Id: I5f013d65b20396bbe0e5a2cceaed2a33fad1af23
The transport/driver features check is done into the get listener
methods.
So when these methods are not used the driver features checks is not
done.
This change moves it into the dispatcher layer to ensure the
requirements are always checked.
This changes a bit the behavior of when the check occurs. Before
it was during the listener object initialisation. Now this
when the listener server start.
Change-Id: I4d81a4e8496f04d62e48317829d5dd8b942d501c
Gnocchi performs better if measurements are write in batch
When Ceilometer is used with Gnocchi, this is not possible.
This change introduce a new notification listener that allows that.
On the driver side, a default batch implementation is provided.
It's just call the legacy poll method many times.
Driver can override it to provide a better implementation.
For example, kafka handles batch natively and take benefit of this.
Change-Id: I16184da24b8661aff7f4fba6196ecf33165f1a77
This change creates a dispatcher abstraction
to document the interface of a dispatcher.
And also allows in the futur to have attributes with default values.
Change-Id: I9a7e5e03f89635a3790b3851f492a1a7aab58feb
In change Ief6f95ea906bfd95b3218a930c9db5d8a764beb9, we
decoupled RPC and Notifications a bit. We should take another
step and separate out the options for notifications into
its own group.
Change-Id: Ib51e2839f9035d0cc0e3f459939d9f9003a8c810
Calling Notifier.audit() won't work with LogDriver as the
logger does not have a audit level anymore, So let's at
least not fail silently and let the operators know that
we are dropping stuff on the floor.
Closes-Bug: #1518170
Change-Id: I74002c72e6763ea8b5df7d97d722619bd4a1950b
There's nothing interesting in the fake context we're passing. Let's
stop depending on oslo.context all together here.
Change-Id: I7784bc3b818e67118e03905857c39eac66765fad
Back in liberty we marked this driver as deprecated. This patch removes
it from the tree. The patch also removes tests, options and other
references in the documentation. Note that one script is being kept
because it's required by the amqp driver.
Depends-On: If4b1773334e424d1f4a4e112bd1f10aca62682a9
Change-Id: I4a9cba314c4a2f24307504fa7b5427424268b114
Ifb96c2ae9868426cac2700bf4917c27c02c90b15 moved the _impl_messaging
module to oslo_messaging.notify.messaging which breaks neutron.
Neutron is fixed on master for mitaka but neutron on stable/liberty
is broken, and changing neutron on stable/liberty to use the new
path would require a global-requirements minimum version bump for
oslo.messaging to 2.6.0, which we want to avoid for people that have
already shipped liberty.
So provide an alias to the moved module so neutron in stable/liberty
continues to work. We deprecate the module so consumers know they
need to upgrade and move off this. We may need to cap oslo.messaging
in global-requirements on stable/liberty at some point when we remove
the deprecated alias module.
Change-Id: I29453e0fbf30b0a571c2b1afc7cc81d1a11535f0
Closes-Bug: #1513630
Add a new configuration option for setting up
an alternate notification_transport_url that
can be used for notifications. This allows
operators to separate the transport mechanisms
used for RPC and Notifications.
DocImpact
Closes-Bug: #1504622
Change-Id: Ief6f95ea906bfd95b3218a930c9db5d8a764beb9
We currently use yaml.load to read a user-written config file. This can
lead to malicious code execution, so we should use yaml.safe_load
instead.
Found using bandit.
Change-Id: I27792f0435bc3cb9b9d31846d07a8d47a1e7679d
Our class hierarchy hides classes and modules that so its hard
for folks to write a custom Notification driver. We should
make these public and document them
Closes-Bug: #1426046
Change-Id: Ifb96c2ae9868426cac2700bf4917c27c02c90b15
When logging a message, any secrets and passwords should be masked. This
uses oslo_utils.strutils to mask any passwords that are to be logged.
Change-Id: I263d44c0f2e900c5f6e210cbd7ec56e48d0d5bb2
Closes-Bug: #1487038
Jenkins