10 Commits

Author SHA1 Message Date
Oleksii Zamiatin
2b50ea5a30 [zmq] Additional configurations for f-tests
Extend functional testing to cover all zmq deployment configurations.

Change-Id: I03a1bb27a4cca429405a19e9de3ba593bbf1db63
2016-07-04 12:44:44 +03:00
Gevorg Davoian
dfd5c791c9 [zmq] Fix message sending when using proxy and not using PUB/SUB
This patch fixes the problem of sending messages in the following
configuration: use_router_proxy=True and use_pub_sub=False. Right
now we have incorrect both cast and fanout messages: cast acts as
fanout (instead of round-robin) and fanout duplicate each message
twice.

Change-Id: I2b039aa462ddde300fd50b3754e4fe66abf6e635
2016-06-18 08:48:10 +03:00
ozamiatin
8ee19159d2 [zmq] Refactoring of zmq client
Split ZmqClient in order to create different clients
for different configurations (direct/proxy).

Change-Id: Ib374f62f53f2c82278ce5bc555ea440e8eba6617
2016-05-31 14:47:49 +00:00
ozamiatin
4efd2d98cc [zmq] Reduce object serialization on router proxy
Switched from envelope object to binary routing key.

Change-Id: I1a5501771d33ecc2211b9d72e88f15871f838da3
Closes-Bug: #1580448
2016-05-11 12:37:09 +03:00
ozamiatin
b5955b6ca9 [zmq] Redesign router proxy
In this change router was redesigned in a way most
appropriate for routing concept of zmq.ROUTER socket.

DEALER(cli)-ROUTER(proxy)-DEALER(srv) instead of
DEALER-ROUTER-DEALER-ROUTER (3 layers instead of 4)

The main reason is to use zmq.DEALER identity in message
routing. For this reason DealerConsumer was introduced
server-side. RouterConsumer is left for peer-to-peer
DEALER-ROUTER deployment option.

Also handled assertions in receive-methods in order
to not stop server when received message with wrong format.

Change-Id: If25edf500fa8d220d4233bb13d67121824e841c6
Closes-Bug: #1558601
Related-Bug: #1555007
2016-05-04 15:12:10 +03:00
Oleksii Zamiatin
cc1cb30321 [zmq] Reduce number of connections
In this change 'use_router_proxy' option was added
to switch between direct connections and proxy.

Proxy was reimplemented and splitted onto two
types of proxies:

    * PUBLISHER proxy for fanout pattern
    * ROUTER proxy for direct messaging

Each type of proxy is configured over command line
argument --type.

Deployment guide was updated accordingly to the change.

Change-Id: If36e9c26e2a8ebe622cfa7e9f2a07b1a69aabe34
Closes-Bug: #1555007
2016-03-31 15:40:36 +03:00
Oleksii Zamiatin
3067dbd198 Non-blocking outgoing queue was implemented
The main issue with eventlet.green.zmq is that libzmq as a C-library
is completely monkey-patch unfriendly. So any blocking call inside
the native library makes calling process stuck. We can't avoid this
actually in an absolutely normal situation when a client appears
earlier than listener we have all client process get stuck until listener
raised. If the listener for example is also blocked awaiting for some
other service to appear we have a chain of locks which may occasionally
result in a dead-lock. The other situation with Notifier is quite similar.

For that reason zmq-broker was restored, but now it serves as an outgoing
queue on a client side. Servers remained the same dynamically port-binded.
Now all clients can still use green-zmq, but presence of the broker-queue
on a host guarantees that green threads will never blocked in a client
because all messages will wait their listeners inside the broker queue.
The broker process's modules are not monkey-patched, they make use of native
threading and native zmq.

Possibility to run without broker also remains. The option zmq_use_broker
introduced for that reason.

Closes-Bug: #1497315

Change-Id: I786b100fd6ee1cf4b99139db0ca044d358d36345
2015-09-28 14:14:53 +03:00
Oleksii Zamiatin
315e56ae2b Get rid of proxy process in zmq
As far as we use redis as a name service we don't need a proxy,
becase we can pass binded port over name service too.

Change-Id: I59bbe2b34dcedfeef113ef06d6a988e1c413405e
2015-07-27 12:29:08 +03:00
Oleksii Zamiatin
ebcadf3d5e Fix threading zmq poller and proxy
- Fixed universal proxy to not get stuck with multiple backends
- Fixed threading pollers/executors (proxy side)
    - Driver option to switch green/no-green impl.
    - Swtiched to no-green in real-world proxy (green left for unit tests)
- Minor names fixes in serializer

Change-Id: Id6508101521d8914228c639ed58ecd29db0ef456
2015-07-15 16:46:04 +03:00
Mehdi Abaakouk
80ece65a54 Don't use devstack to setup our functional env
This adds a setup script for each tox functional target to start a
rabbitmq-server, qpidd or redis daemon dedicated for the functional
testing.

This script is responsible to spawn a preconfigured daemon needed for
the functional tests.

This also changes the gate script to just install the required packages
instead of setup a devstack.

This also fixes the zmq config options loading in tests

Closes-bug: #1442612

Change-Id: I27eb2c1d3d0ca67aa361c83e41372138e03d9bdd
2015-06-11 12:01:01 +02:00