Support for binary data in the websocket transport
Add spec for support for binary communications over websocket transport which will allow to reduce network traffic between Zaqar and it's clients and also to increase performance. Change-Id: I35b0912e2c5c0e31bca22706209ddee0cba50a1f
This commit is contained in:
Eva Balycheva
parent
2e8c4f2c31
commit
cd5c2adc9c
|
|
@ -5,5 +5,5 @@
|
|||
.. toctree::
|
||||
:glob:
|
||||
:maxdepth: 2
|
||||
.. FIXME(mdnadeem): Uncomment after adding first real spec.
|
||||
.. *
|
||||
|
||||
*
|
||||
|
|
@ -0,0 +1,216 @@
|
|||
..
|
||||
This template should be in ReSTructured text. The filename in the git
|
||||
repository should match the launchpad URL, for example a URL of
|
||||
https://blueprints.launchpad.net/zaqar/+spec/awesome-thing should be named
|
||||
awesome-thing.rst.
|
||||
|
||||
Please do not delete any of the sections in this
|
||||
template. If you have nothing to say for a whole section, just write: None
|
||||
|
||||
For help with syntax, see http://sphinx-doc.org/rest.html
|
||||
To test out your formatting, see http://www.tele3.cz/jbar/rest/rest.html
|
||||
|
||||
===================================================
|
||||
Support for binary data in the websocket transport
|
||||
===================================================
|
||||
|
||||
https://blueprints.launchpad.net/zaqar/+spec/websocket-binary-support
|
||||
|
||||
Implement support for binary communications over websocket transport which will
|
||||
allow to reduce network traffic between Zaqar and it's clients and also to
|
||||
increase performance.
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
Currently websocket transport uses only text messages packed in JSON format to
|
||||
exchange data with clients.
|
||||
|
||||
An attempt to send binary data to Zaqar's websocket transport will lead to
|
||||
receiving 'bad request' (HTTP error 400) response from Zaqar.
|
||||
|
||||
But websocket protocol actually allows communication via binary messages.
|
||||
Data packed(serialized) in binary format can be smaller and also may take less
|
||||
CPU time to pack/unpack it.
|
||||
|
||||
By using binary messages for communications it's possible to reduce network
|
||||
traffic between Zaqar and it's clients and also to increase Zaqar's
|
||||
performance. It can also increase client performance if the client is not
|
||||
written in JavaScript.
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
We can serialize/deserialize data via MsgPack technology which was already
|
||||
proposed and approved in persistent-transport_ specification.
|
||||
|
||||
.. _persistent-transport: https://github.com/openstack/zaqar-specs/blob/master/specs/kilo/approved/persistent-transport.rst
|
||||
|
||||
Deserialization of incoming messages to Zaqar
|
||||
---------------------------------------------
|
||||
|
||||
As you can see in `protocol.py#L55`_, Zaqar already can distinguish binary and
|
||||
text messages.
|
||||
|
||||
.. _protocol.py#L55: https://github.com/openstack/zaqar/blob/stable/liberty/zaqar/transport/websocket/protocol.py#L55
|
||||
|
||||
So we will make Zaqar try to deserialize each received binary message via
|
||||
'msgpack' library into a python object, just like Zaqar currently deserializes
|
||||
JSON messages via 'json' library. After successful deserialization the
|
||||
message can be processed as usual.
|
||||
|
||||
Serialization of response and subscription messages from Zaqar
|
||||
--------------------------------------------------------------
|
||||
|
||||
The response from Zaqar will be serialized in the same format as the incoming
|
||||
request.
|
||||
|
||||
The subscription messages from Zaqar will be serialized in binary or text
|
||||
format based on the format of subscription create request.
|
||||
|
||||
Let's take a look at the case where the client suddenly changes it's messaging
|
||||
format while connection is open and subscription is still alive. There might
|
||||
be a problem when the client will not be able to deserialize the messages
|
||||
coming from the old subscription, because it has switched the format by
|
||||
changing some boolean variable (switch). But this situation is nearly
|
||||
impossible and, if not, the problem is solvable:
|
||||
|
||||
#. It's hard to imagine a situation when it might be useful for the client to
|
||||
change it's serialization format. Such client will probably never exist.
|
||||
#. Even if such client will exist in the future, the client developer will
|
||||
take care of it by adding the code checking the format of each incoming
|
||||
message and deserializing the message accordingly inside "on message"
|
||||
method. Zaqar does exactly this, for example. All websocket implementations
|
||||
provide a convenient way to determine the format of each particular message.
|
||||
#. Even if the client developer will not take care of this potential problem
|
||||
(he is inexperienced, for example) and will make the client just expect
|
||||
all incoming messages in the new format after switching while the old
|
||||
subscription is still alive, the client will crash on the **first incoming
|
||||
message from this subscription** and the problem will not go unnoticed.
|
||||
|
||||
|
||||
Websocket html client example upgrade
|
||||
-------------------------------------
|
||||
|
||||
The websocket html `client example`_ will be changed to make it also being able
|
||||
to communicate with Zaqar via binary messages.
|
||||
|
||||
.. _client example: https://github.com/openstack/zaqar/blob/stable/liberty/examples/websocket.html
|
||||
|
||||
Drawbacks
|
||||
---------
|
||||
|
||||
The performance of web clients on JavaScript
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Seems like MsgPack serialization/deserialization works slower than JSON in
|
||||
JavaScript. Well, no surprise since JSON abbreviation expands to "JavaScript
|
||||
Object Notation".
|
||||
It's a drawback for Zaqar web clients as it decreases their performance.
|
||||
|
||||
How the change addresses this drawback:
|
||||
|
||||
#. The server is one and clients are many. The server processes much more
|
||||
messages than each particular client. So there are many cases when it's
|
||||
reasonable to use binary serialization to increase server performance by
|
||||
sacrificing performance of JavaScript clients.
|
||||
|
||||
#. The additional delays on JavaScript clients are slightly compensated by
|
||||
faster responses from the server.
|
||||
|
||||
#. JavaScript clients can always use good old text format for communications
|
||||
with the server, while other clients can use binary format with the same
|
||||
server simultaneously.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
These alternatives completely exclude the nearly impossible case where the
|
||||
client may suddenly switch it's messaging format, while the connection is open
|
||||
and the old subscription is still alive, and while proper deserialization of
|
||||
incoming messages to the client is managed by some client's boolean variable
|
||||
(switch).
|
||||
|
||||
Alternatives of serialization of response and subscription messages from Zaqar
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Optional parameter approach
|
||||
"""""""""""""""""""""""""""
|
||||
|
||||
The response from Zaqar will be serialized in the same format as the incoming
|
||||
request.
|
||||
|
||||
The subscription messages from Zaqar will be serialized in binary
|
||||
or text format based on the new optional parameter passed in the message
|
||||
body of subscription create request. We can add this parameter to websocket
|
||||
API, name it ``accept``, allow it to have only two possible values: ``json``
|
||||
and ``msgpack`` and make ``json`` value as default if ``accept`` parameter was
|
||||
not passed.
|
||||
By making ``accept`` parameter optional, we are achieving backward
|
||||
compatibility.
|
||||
|
||||
Subscription request message example before implementation::
|
||||
|
||||
{
|
||||
'action': 'subscription_create',
|
||||
'headers': {'Client-ID': 31209ff3-ba03-4cec-b4ca-655f4899f8aa, 'X-Project-ID': superproject},
|
||||
'body': {'queue_name': 'superqueue', 'ttl': 3600}
|
||||
}
|
||||
|
||||
Subscription request message after implementation can be the same as before or
|
||||
it can also pass ``accept`` parameter explicitly::
|
||||
|
||||
{
|
||||
'action': 'subscription_create',
|
||||
'headers': {'Client-ID': 31209ff3-ba03-4cec-b4ca-655f4899f8aa, 'X-Project-ID': superproject},
|
||||
'body': {'queue_name': 'superqueue', 'ttl': 3600, 'accept': 'json'}
|
||||
}
|
||||
|
||||
First message approach
|
||||
""""""""""""""""""""""
|
||||
|
||||
The response and subscription messages from Zaqar will be serialized in binary
|
||||
or text format based on in which format the particular client has sent it's
|
||||
first successfully parsed message to Zaqar since connecting.
|
||||
|
||||
Once the format of communications between Zaqar and the client was
|
||||
established by that way, the client will not be able to change it, unless the
|
||||
client will reconnect to Zaqar and send the first message in other format.
|
||||
|
||||
Zaqar configuration approach
|
||||
""""""""""""""""""""""""""""
|
||||
|
||||
The response and subscription messages from Zaqar will be serialized in binary
|
||||
or text format based on the boolean variable in Zaqar server configuration.
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
ubershy
|
||||
|
||||
Milestones
|
||||
----------
|
||||
|
||||
Target Milestone for completion:
|
||||
Mitaka-2
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
None
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
.. note::
|
||||
|
||||
This work is licensed under a Creative Commons Attribution 3.0
|
||||
Unported License.
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
Loading…
Reference in New Issue