Add the NATS transport driver to oslo.messaging
nats-transport is the spec for a new transport driver for oslo.messaging to use the NATS messaging system. Change-Id: Ic6346947bbf650fb53581842c64fdec99090c242 Blueprint: nats-transport
This commit is contained in:
parent
3b050c4df1
commit
924127f088
|
@ -0,0 +1,213 @@
|
||||||
|
=====================================:wq
|
||||||
|
NATS Driver for Messaging Transport
|
||||||
|
=====================================
|
||||||
|
|
||||||
|
This specification proposes a new transport driver for oslo.messaging
|
||||||
|
that uses NATS. NATS is a high-performance messaging system [1]_ [2]_
|
||||||
|
written in Go. NATS is designed as a central nervous system for the
|
||||||
|
cloud control plane [3]_. While NATS Sreaming has message persistency
|
||||||
|
like RabbitMQ, NATS itself doesn't but it's not a problem for
|
||||||
|
oslo.messaging because every message has its own lifetime. NATS server
|
||||||
|
supports clustering for high availability.
|
||||||
|
|
||||||
|
|
||||||
|
Problem description
|
||||||
|
===================
|
||||||
|
|
||||||
|
RabbitMQ is widely used in OpenStack deployments but it has problems:
|
||||||
|
|
||||||
|
* Big footprint: RabbitMQ is only one using Erlang for a typical
|
||||||
|
OpenStack deployment and it requires 40MB+ disk space and ~30
|
||||||
|
packages in Ubuntu. RDO stein contains 43 packages of Erlang.
|
||||||
|
|
||||||
|
* HA performance: "Avoid multiple nodes (HA)" is one of the best
|
||||||
|
practices from the point of view of performance [4]_.
|
||||||
|
|
||||||
|
* HA stability: Japanese OpenStackers discussed RabbitMQ operation
|
||||||
|
problems at OpenStack Ops Workshop #6 at Tokyo [5]_ and an attendee
|
||||||
|
said that clustered RabbitMQ servers in his OpenStack deployment
|
||||||
|
suddenly died without failover and he decided to stop using RabbitMQ
|
||||||
|
clustering.
|
||||||
|
|
||||||
|
|
||||||
|
Proposed change
|
||||||
|
===============
|
||||||
|
|
||||||
|
Implements a new driver of oslo.messaging to use NATS as the RPC and
|
||||||
|
notification message transport. It uses pynats2 [6]_, threading based
|
||||||
|
python 3.6+ client for NATS.
|
||||||
|
|
||||||
|
Impact on Existing APIs
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
Nothing.
|
||||||
|
|
||||||
|
Security impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
Nothing. The NATS driver should have 2 security capabilities like
|
||||||
|
other drivers: authentication and encryption. pynats2 python client
|
||||||
|
library has both, so we can use them in the driver.
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
Nothing.
|
||||||
|
|
||||||
|
|
||||||
|
Configuration Impact
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
To use the NATS driver, spefify the value for transport_url in section
|
||||||
|
[Default]:
|
||||||
|
|
||||||
|
transport_url = nats://[user:pass@]host[:port][,[userN:passN@]hostN:portN]/[virtual_host]
|
||||||
|
|
||||||
|
For TLS connection,
|
||||||
|
|
||||||
|
transport_url = nats+tls://[user:pass@]host[:port][,[userN:passN@]hostN:portN]/[virtual_host]
|
||||||
|
|
||||||
|
The default value of port is 4222. You can specify multiple hosts with
|
||||||
|
each user:pass. The NATS server doesn't support virtual host but
|
||||||
|
pynats2 module has pseudo virtual host capability and the driver uses
|
||||||
|
it.
|
||||||
|
|
||||||
|
The driver has options in section [oslo_messaging_nats]:
|
||||||
|
|
||||||
|
socket_timeout:
|
||||||
|
Seconds to waiting recieved data from NATS server
|
||||||
|
Type: float
|
||||||
|
Default value: 2.0
|
||||||
|
|
||||||
|
ping_interval:
|
||||||
|
Interval in seconds to send PING to NATS server
|
||||||
|
Type: float
|
||||||
|
Default value: 10.0
|
||||||
|
|
||||||
|
tls_verify:
|
||||||
|
Verify the NATS server certificate
|
||||||
|
Type: bool
|
||||||
|
Default value: False
|
||||||
|
|
||||||
|
tls_cacert:
|
||||||
|
path to the CA certificate file for the NATS server verification
|
||||||
|
Type: string
|
||||||
|
|
||||||
|
tls_client_cert:
|
||||||
|
path to the client certificate file for the NATS connection
|
||||||
|
Type: string
|
||||||
|
|
||||||
|
tls_client_key:
|
||||||
|
path to the client key file for the NATS connection
|
||||||
|
Type: string
|
||||||
|
|
||||||
|
Developer Impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
Nothing.
|
||||||
|
|
||||||
|
Testing Impact
|
||||||
|
--------------
|
||||||
|
|
||||||
|
The NATS server will be used as the messaging backend for RPC and
|
||||||
|
notifications in testing. No alternative backend such as RabbitMQ or
|
||||||
|
Apache Kafka should be used.
|
||||||
|
|
||||||
|
The NATS driver tests should be implemented and the driver must pass
|
||||||
|
all tests.
|
||||||
|
|
||||||
|
The driver must pass the following gate checks with deployed in a
|
||||||
|
hybrid messaging configuration.
|
||||||
|
|
||||||
|
* oslo.messaging-tox-py36-func-nats
|
||||||
|
|
||||||
|
pynats2 python client library only supports Python 3.6+. It doesn't
|
||||||
|
use asyncio, so we can make it available with Python 2.x. However,
|
||||||
|
Python 2.x will be unsupported on Jan 1st, 2020, and the NATS driver
|
||||||
|
won't be required backward compatibility because it's new one, so
|
||||||
|
gates for Python 2.x won't be used or implemented for it.
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
akirayoshiyama@gmail.com (Akira72 or yosshy on IRC)
|
||||||
|
|
||||||
|
Other contributors:
|
||||||
|
No one for now. Contributors are welcome.
|
||||||
|
|
||||||
|
Milestones
|
||||||
|
----------
|
||||||
|
|
||||||
|
Target Milestone for completion: V release
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
* Deploy environment for latest software release and dependencies
|
||||||
|
* Implement the NATS driver
|
||||||
|
* Write documentation
|
||||||
|
* Functional test integration
|
||||||
|
* Implement devstack plugin
|
||||||
|
* Upstream CI integration
|
||||||
|
* Send announcement to openstack-discuss following release
|
||||||
|
|
||||||
|
Incubation
|
||||||
|
==========
|
||||||
|
|
||||||
|
None.
|
||||||
|
|
||||||
|
Adoption
|
||||||
|
--------
|
||||||
|
|
||||||
|
The NATS driver is expected to be adopted in many deployments which
|
||||||
|
operators doesn't like to use RabbitMQ.
|
||||||
|
|
||||||
|
Library
|
||||||
|
-------
|
||||||
|
|
||||||
|
oslo.messaging
|
||||||
|
|
||||||
|
Anticipated API Stabilization
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
The NATS driver documentation will be added to the libary. This
|
||||||
|
documentation will follow the style of documentation provided by the
|
||||||
|
other drivers and should include the following topics:
|
||||||
|
|
||||||
|
* theory of operation (overview) of the NATS messaging backend
|
||||||
|
* pre-requisites
|
||||||
|
* driver options overview
|
||||||
|
* the NATS server operations
|
||||||
|
* devstack support
|
||||||
|
* platforms and software
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
The driver will require pynats2, threading based NATS python client.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. [1] https://nats.io/
|
||||||
|
.. [2] https://bravenewgeek.com/dissecting-message-queues/
|
||||||
|
.. [3] https://nats.io/blog/introducing-nats-streaming/#why-nats-streaming
|
||||||
|
.. [4] https://www.cloudamqp.com/blog/2018-01-08-part2-rabbitmq-best-practice-for-high-performance.html#avoid-multiple-nodes-ha
|
||||||
|
.. [5] https://etherpad.openstack.org/p/JP-Ops-workshop-6-rabbit
|
||||||
|
.. [6] https://github.com/yosshy/pynats2
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0
|
||||||
|
Unported License.
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
Loading…
Reference in New Issue