1015 Commits

Author SHA1 Message Date
Oleksii Zamiatin
141f59bd9b Notifier implementation
Notifier implementation for zmq driver (ROUTER/DEALER variant).
Publishers/consumers refactoring in order to make them pluggable.

Change-Id: I2dd42cc805aa72b929a4dfa17498cd8b9c0ed7af
2015-08-05 13:35:24 +03:00
Victor Sergeyev
64831f29ee ZMQ: Minor matchmaker improvement
Added more information to logs and prefix to key in matchmaker_redis

Change-Id: I5d718c4a84dedc6654e13f9ca740a2deba5a7e43
2015-08-04 15:31:10 +03:00
Jenkins
2aa35bb6d9 Merge "ZMQ: Lazify driver code" into feature/zmq 2015-08-01 08:25:40 +00:00
Doug Royal
dec09ae5ff Add unit tests for zmq_async
Change option from boolean zmq_native to string zmq_concurrency. This
eliminates ambiguity by requiring the user to explicitly name
the mechanism they want to use for concurrency.

Change-Id: I341a3eee73a0449716d3ee0df690bbe6af39bdf0
2015-07-31 18:04:37 -05:00
Victor Sergeyev
ed2d60ff01 ZMQ: Lazify driver code
Some OpenStack services (e.g. Glance) makes a forks, so there is a sense
to initialize socket and thread related stuff `on demand`, not in
__init__().

Change-Id: Ie2012b31df86049cc841a0aaed16e6b879e0bcec
2015-07-30 17:18:08 +03:00
Doug Hellmann
c90525bfea Remove oslo namespace package
Blueprint remove-namespace-packages

Cherry-picked from: 03265410e0294e176d18dd42b57268a8056eb8fc

Change-Id: Ibaba19ef10b4902c4f4f9fbdf7078e66b75f2035
2015-07-29 11:17:57 +03:00
Victor Sergeyev
99b24b3888 Register matchmaker_redis_opts in RedisMatchMaker
Change-Id: Ib74051be2f8b79a07e0c8a38769f4d47cf792bc7
2015-07-29 11:16:25 +03:00
Victor Sergeyev
5920e7bef6 ZMQ: Removed unused code and tests
The code of ZMQ driver is under the active development now, so after
modifications some code can became unused. This patch removes this code.

Change-Id: I4cfa75560eabf82618f31584b4645fd2630ac9cb
2015-07-28 13:07:16 +03:00
Victor Sergeyev
de629d8104 ZMQ: Run more functional tests
Change-Id: Ia7b001bf5aba1120544dcc15c5200c50ebe731f6
2015-07-27 18:18:05 +03:00
Oleksii Zamiatin
315e56ae2b Get rid of proxy process in zmq
As far as we use redis as a name service we don't need a proxy,
becase we can pass binded port over name service too.

Change-Id: I59bbe2b34dcedfeef113ef06d6a988e1c413405e
2015-07-27 12:29:08 +03:00
Oleksii Zamiatin
e2c3e36d75 Close sockets properly
All socket-connections should properly
die after their parents being stopped.

Change-Id: I6a83ed2d5ef194e8b068c1d8bd6813f48636c5fb
2015-07-21 12:52:31 +03:00
Oleksii Zamiatin
75660cedac Target direct usage
Reduced ZmqTopic as unneeded duplication of
oslo_messaging Target.

Change-Id: Ie9c8e59cb8a2b08c26dbc42f7af4e808362a7524
2015-07-20 13:17:24 +03:00
Doug Royal
9c06fa84a0 Move zmq tests into a subdirectory
There are no code changes in this commit. The zmq tests are being
organized to match the development of the new zmq driver.

Change-Id: Id79a2ade3874c56d8d5c3eca4689d49ba68d4418
2015-07-19 21:46:37 -05:00
Victor Sergeyev
9e4831c022 ZMQ: Initial matchmaker implementation
This patch replaces the old outdated matchmakers and replace it into the
new ones.

Call/Cast test_specific_server() functional tests passes now.

Change-Id: I8635396110d30d26812f39b242fbbabd1a0feaaa
2015-07-16 09:23:46 +00:00
Oleksii Zamiatin
ebcadf3d5e Fix threading zmq poller and proxy
- Fixed universal proxy to not get stuck with multiple backends
- Fixed threading pollers/executors (proxy side)
    - Driver option to switch green/no-green impl.
    - Swtiched to no-green in real-world proxy (green left for unit tests)
- Minor names fixes in serializer

Change-Id: Id6508101521d8914228c639ed58ecd29db0ef456
2015-07-15 16:46:04 +03:00
Doug Royal
12aff74f53 Add unit tests for zmq_serializer
Change-Id: I428f2f80ca10bc888c809e4d6e7862e2ee5d442c
2015-07-10 15:54:26 -05:00
Victor Sergeyev
48f2a87a27 Fix work with timeout in CallRequest.receive_reply()
Refactored CallRequest.receive_reply() method to raise MessagingTimeout
exception, when timeout is reached.  Removed unused _to_milliseconds() method

Functional test CallTestCase.test_timeout() passes now

Change-Id: Idc3224646c3626a56606d019ff7ff155d3e3201a
2015-07-10 15:50:47 +03:00
Victor Sergeyev
bcdc0e88ec ZMQ: Allow to raise remote exception
This patch adds possibility to re-raise on client's side exception, that
was raised on server side - serialize it on server side, restore and
re-raise on client.
Allowed to pass `allowed_remote_exmods` parameter from impl_zmq to
CallRequest class

Functional test CallTestCase.test_exception() passes now, so added it to
tox.ini. Modified zmq_receiver to be able run functional tests.

Change-Id: Ic055f3574962f3e80a0528d5d99320386303634e
2015-07-09 19:57:55 +00:00
Oleksii Zamiatin
7df65f2937 Local Fanout implementation
Fanout unit-test passes now
No matchmaker used yet (multi-host fanout wouldn't work)

Change-Id: I9362adab4f7c7eba8120b51efe1b8c2056df3bbe
2015-07-09 22:50:44 +03:00
Doug Hellmann
76f44879e1 Drop use of 'oslo' namespace package
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.

The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.

Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.

Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages
Cherry-picked from 3b6ca5b6de9e7964e1a3c41b5b0146e116b248ba

Change-Id: I05a408b76d4f31b803769a27759e91df770511bb
2015-07-09 17:25:00 +03:00
Flavio Percoco
4073851bf8 Fix qpid's functional gate
The loggin verbosity changed in 0.32 which our gate is pulling from the
updates repository. This patch updates the script to make it work again.

Closes-bug: #1468917
Change-Id: I88a1bc794246beb156d4301bd503fa51a7cd2cce
(cherry picked from commit 079c04f0d06539f3507c772ccc68807a6d01a73a)
2015-07-06 12:19:14 +00:00
Oleksii Zamiatin
73cd49129f Initial commit for new zmq driver implementation
- Minimal RPC (CALL + direct CAST) implementation
- Has up and running oslo_messaging/tests/drivers/test_impl_zmq
- Pep8 fixed.
- Works over REQ/REP pipeline according to [1]
- Has a beginning of eventlet/threading behavior differentiation

Fanout and Notifier are not yet supported
Devstack not yet fixed
Functional tests not yet fixed

..[1] - https://review.openstack.org/#/c/171131/

Change-Id: I44cd48070bf7c7f46152fdf0e54664a7dee97de9
2015-06-30 18:16:02 +03:00
Doug Royal
76ec03c8f9 fix typo
Change-Id: I8a935ffc795b7233e2e83ae0680786d34dfd6ec0
2015-06-25 17:44:28 -05:00
Doug Hellmann
8587774d36 Update .gitreview for feature/zmq
Change-Id: I773307e72a61da6aac6adda9592abcb3c65e8e67
2015-06-16 09:55:00 -04:00
Jenkins
a3a3c3faa8 Merge "Use inferred=True by default" 2015-06-16 09:49:22 +00:00
Jenkins
f3d70a923d Merge "Enable amqp's protocol unit tests everywhere" 2015-06-16 09:42:28 +00:00
Flavio Percoco
c8845b4bef Use inferred=True by default
Rabbitmq's amqp1.0 plugin doesn't have support for vbin8, which is the
default encoding used by qpid-proton. In order to workaround this issue
we should use inferred=True.

Closes-bug: #1465409
Change-Id: Id265917244d7e152c5e13a10df367a3e59de8e50
2015-06-15 23:16:57 +02:00
Flavio Percoco
f06b19628f Enable amqp's protocol unit tests everywhere
Now that python-qpid-proton has taken an approach similar to zmq's, it's
possible to install it in boxes where the C libraries are not present in
the system. This patch takes advantage of that to enable amqp protocol's
tests everywhere.

Change-Id: Ibce4c791aafadf45fa876d6a4f5373fde94769ff
2015-06-15 20:52:24 +02:00
Jenkins
4062c1caa6 Merge "Switch badges from 'pypip.in' to 'shields.io'" 2015-06-15 17:22:13 +00:00
Jenkins
ffcb3a34b4 Merge "Switch to warnings module instead of versionutils" 2015-06-14 01:45:12 +00:00
Jenkins
3800dd61c5 Merge "Don't use devstack to setup our functional env" 2015-06-13 00:47:18 +00:00
Jenkins
38558db420 Merge "Add drivers to the documentation" 2015-06-12 21:10:53 +00:00
Jenkins
985b1597b4 Merge "Provide better detection of failures during message send" 2015-06-12 11:09:11 +00:00
Joshua Harlow
a3b56af37f Switch badges from 'pypip.in' to 'shields.io'
The badge hosting at 'pypip.in' has been broken for a while
now, it was supposed to be fixed in the following PR(s):

- https://github.com/badges/pypipins/issues/39
- https://github.com/badges/pypipins/issues/38
- https://github.com/badges/pypipins/issues/37

That does not appear to have actually happened and/or is not
getting addressed, so switch to a more reliable badge hosting
site that provides the same badge information.

Change-Id: Ia1fd048e8f1810c4aaabc24bd01c7f0c18b79fd6
2015-06-11 20:38:19 -07:00
Mehdi Abaakouk
80ece65a54 Don't use devstack to setup our functional env
This adds a setup script for each tox functional target to start a
rabbitmq-server, qpidd or redis daemon dedicated for the functional
testing.

This script is responsible to spawn a preconfigured daemon needed for
the functional tests.

This also changes the gate script to just install the required packages
instead of setup a devstack.

This also fixes the zmq config options loading in tests

Closes-bug: #1442612

Change-Id: I27eb2c1d3d0ca67aa361c83e41372138e03d9bdd
2015-06-11 12:01:01 +02:00
Davanum Srinivas
2bf8d2f917 Switch to warnings module instead of versionutils
versionutils.py is no longer in oslo-incubator. So we can
either use versionutils from oslo_log or debtcollector instead.
However, oslo.messaging does not use oslo.log currently and we
should not be adding yet another library as a dependency here,
so we should just use the base python warnings library for
our limited use of deprecated() method.

Change-Id: Ib8a487051c894fa4828da65d4890b7a4f57f1d12
2015-06-10 22:23:16 -04:00
OpenStack Proposal Bot
2f14ca162f Updated from global requirements
Change-Id: I95c7c2b87735dae3e98ad401d05cf382022c1199
2015-06-11 00:47:50 +00:00
Jenkins
62e7280e0f Merge "rabbit: Add logging on blocked connection" 2015-06-10 17:11:50 +00:00
Victor Sergeyev
2e5ba4538e Get mox from mox3, not from six.moves
oslotest 1.7.0 breakes oslo.messaging unittests, because it doesn't add
mox to six.moves anymore - see change Ic59c73abb9b09cb594bf7df4173d7f99f81d526c

This patch fixes imports on test run.

Change-Id: Ia40b7733bd2e74b1a1703f2e1a5245f01debbd36
2015-06-10 14:44:59 +03:00
Mehdi Abaakouk
1f8ccd3ac5 rabbit: Add logging on blocked connection
When the broker will block the connection for a server-side issue
like disk full, it notifies the client.

This change adds the callback methods when this occurs to inform
the deployer about the reason of this blocking.

Change-Id: I5164b9e1b720f022b45a5718258df036ba8808ed
Closes-bug: #1454449
2015-06-09 10:35:24 +02:00
Kenneth Giusti
c127594de6 Provide better detection of failures during message send
This change causes the message sender to block until the messaging
infrastructure (e.g. the broker) assumes ownership of the message (or
fails to accept it).  If the message is accepted, then the sender will
either pend for a response (in the case of RPC), or simply return (in
the case of notification).  If the message is rejected by the
messaging infrastructure a MessagingException will be raised at the
sender.

Change-Id: I3f4a1ed1c17e18f6d629f16e6b5c99de45b083d6
Closes-Bug: #1377228
2015-06-08 20:42:10 -04:00
Jenkins
85c069e154 Merge "Set places to 0 in self.assertAlmostEqual()" 2015-06-08 16:41:39 +00:00
Jenkins
02ab25ecc6 Merge "Reduce magic conf attribute usage" 2015-06-08 16:40:22 +00:00
Jenkins
63bbdfc7c9 Merge "rabbit: test for new reply behavior" 2015-06-05 11:20:37 +00:00
Jenkins
2d133bee84 Merge "Enable fanout_target scenarios in test_impl_rabbit" 2015-06-05 11:20:11 +00:00
Victor Sergeyev
3044abbaf4 Reduce magic conf attribute usage
Connection class contains `magic` attribute conf, which contains (or at
least should contain) all config options. Set these config options as
class attributes to make the code more clear

Change-Id: Ib67f148e2ecf3def37e57d6e3359bced932fbc90
2015-06-05 11:12:58 +03:00
Jenkins
f51a4ef6c6 Merge "Imported Translations from Transifex" 2015-06-04 18:11:35 +00:00
Jenkins
d685e6f80a Merge "replace rpc_response_timeout use in rabbit driver" 2015-06-04 17:50:16 +00:00
OpenStack Proposal Bot
9f146bc299 Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I55f2a8f6f81f12735f0cf5c0ad20401283bb4e08
2015-06-04 06:08:05 +00:00
Ben Nemec
94bb8adfd0 Remove leftover oslo.config reference
There was still a reference to the oslo.config namespace package
in _drivers/base.py.  This converts it to the new package name.

Change-Id: I9c3878094bcf8015c30d87f693f51e0d48b31a33
2015-06-04 00:06:40 +00:00