Add the NATS transport driver to oslo.messaging
nats-transport is the spec for a new transport driver for oslo.messaging to use the NATS messaging system. Change-Id: Ic6346947bbf650fb53581842c64fdec99090c242 Blueprint: https://blueprints.launchpad.net/oslo.messaging/+spec/nats-transport
This commit is contained in:
Akira Yoshiyama
parent
3b050c4df1
commit
805e8f9ffe
|
|
@ -0,0 +1,213 @@
|
|||
=====================================
|
||||
NATS Driver for Messaging Transport
|
||||
=====================================
|
||||
|
||||
This specification proposes a new transport driver for oslo.messaging
|
||||
that uses NATS. NATS is a high-performance messaging system [1]_ [2]_
|
||||
written in Go. NATS is designed as a central nervous system for the
|
||||
cloud control plane [3]_. While NATS Sreaming has message persistency
|
||||
like RabbitMQ, NATS itself doesn't but it's not a problem for
|
||||
oslo.messaging because every message has its own lifetime. NATS server
|
||||
supports clustering for high availability.
|
||||
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
RabbitMQ is widely used in OpenStack deployments but it has problems:
|
||||
|
||||
* Big footprint: RabbitMQ is only one using Erlang for a typical
|
||||
OpenStack deployment and it requires 40MB+ disk space and ~30
|
||||
packages in Ubuntu. RDO stein contains 43 packages of Erlang.
|
||||
|
||||
* HA performance: "Avoid multiple nodes (HA)" is one of the best
|
||||
practices from the point of view of performance [4]_.
|
||||
|
||||
* HA stability: Japanese OpenStackers discussed RabbitMQ operation
|
||||
problems at OpenStack Ops Workshop #6 at Tokyo [5]_ and an attendee
|
||||
said that clustered RabbitMQ servers in his OpenStack deployment
|
||||
suddenly died without failover and he decided to stop using RabbitMQ
|
||||
clustering.
|
||||
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
Implements a new driver of oslo.messaging to use NATS as the RPC and
|
||||
notification message transport. It uses pynats2 [6]_, threading based
|
||||
python 3.6+ client for NATS.
|
||||
|
||||
Impact on Existing APIs
|
||||
-----------------------
|
||||
|
||||
Nothing.
|
||||
|
||||
Security impact
|
||||
---------------
|
||||
|
||||
Nothing. The NATS driver should have 2 security capabilities like
|
||||
other drivers: authentication and encryption. pynats2 python client
|
||||
library has both, so we can use them in the driver.
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
|
||||
Nothing.
|
||||
|
||||
|
||||
Configuration Impact
|
||||
--------------------
|
||||
|
||||
To use the NATS driver, spefify the value for transport_url in section
|
||||
[Default]:
|
||||
|
||||
transport_url = nats://[user:pass@]host[:port][,[userN:passN@]hostN:portN]/[virtual_host]
|
||||
|
||||
For TLS connection,
|
||||
|
||||
transport_url = nats+tls://[user:pass@]host[:port][,[userN:passN@]hostN:portN]/[virtual_host]
|
||||
|
||||
The default value of port is 4222. You can specify multiple hosts with
|
||||
each user:pass. The NATS server doesn't support virtual host but
|
||||
pynats2 module has pseudo virtual host capability and the driver uses
|
||||
it.
|
||||
|
||||
The driver has options in section [oslo_messaging_nats]:
|
||||
|
||||
socket_timeout:
|
||||
Seconds to waiting recieved data from NATS server
|
||||
Type: float
|
||||
Default value: 2.0
|
||||
|
||||
ping_interval:
|
||||
Interval in seconds to send PING to NATS server
|
||||
Type: float
|
||||
Default value: 10.0
|
||||
|
||||
tls_verify:
|
||||
Verify the NATS server certificate
|
||||
Type: bool
|
||||
Default value: False
|
||||
|
||||
tls_cacert:
|
||||
path to the CA certificate file for the NATS server verification
|
||||
Type: string
|
||||
|
||||
tls_client_cert:
|
||||
path to the client certificate file for the NATS connection
|
||||
Type: string
|
||||
|
||||
tls_client_key:
|
||||
path to the client key file for the NATS connection
|
||||
Type: string
|
||||
|
||||
Developer Impact
|
||||
----------------
|
||||
|
||||
Nothing.
|
||||
|
||||
Testing Impact
|
||||
--------------
|
||||
|
||||
The NATS server will be used as the messaging backend for RPC and
|
||||
notifications in testing. No alternative backend such as RabbitMQ or
|
||||
Apache Kafka should be used.
|
||||
|
||||
The NATS driver tests should be implemented and the driver must pass
|
||||
all tests.
|
||||
|
||||
The driver must pass the following gate checks with deployed in a
|
||||
hybrid messaging configuration.
|
||||
|
||||
* oslo.messaging-tox-py36-func-nats
|
||||
|
||||
pynats2 python client library only supports Python 3.6+. It doesn't
|
||||
use asyncio, so we can make it available with Python 2.x. However,
|
||||
Python 2.x will be unsupported on Jan 1st, 2020, and the NATS driver
|
||||
won't be required backward compatibility because it's new one, so
|
||||
gates for Python 2.x won't be used or implemented for it.
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
akirayoshiyama@gmail.com (Akira72 or yosshy on IRC)
|
||||
|
||||
Other contributors:
|
||||
No one for now. Contributors are welcome.
|
||||
|
||||
Milestones
|
||||
----------
|
||||
|
||||
Target Milestone for completion: V release
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
* Deploy environment for latest software release and dependencies
|
||||
* Implement the NATS driver
|
||||
* Write documentation
|
||||
* Functional test integration
|
||||
* Implement devstack plugin
|
||||
* Upstream CI integration
|
||||
* Send announcement to openstack-discuss following release
|
||||
|
||||
Incubation
|
||||
==========
|
||||
|
||||
None
|
||||
|
||||
Adoption
|
||||
--------
|
||||
|
||||
The NATS driver is expected to be adopted in many deployments which
|
||||
operators doesn't like to use RabbitMQ.
|
||||
|
||||
Library
|
||||
-------
|
||||
|
||||
oslo.messaging
|
||||
|
||||
Anticipated API Stabilization
|
||||
-----------------------------
|
||||
|
||||
None
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
The NATS driver documentation will be added to the libary. This
|
||||
documentation will follow the style of documentation provided by the
|
||||
other drivers and should include the following topics:
|
||||
|
||||
* theory of operation (overview) of the NATS messaging backend
|
||||
* pre-requisites
|
||||
* driver options overview
|
||||
* the NATS server operations
|
||||
* devstack support
|
||||
* platforms and software
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
The driver will require pynats2, threading based NATS python client.
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
.. [1] https://nats.io/
|
||||
.. [2] https://bravenewgeek.com/dissecting-message-queues/
|
||||
.. [3] https://nats.io/blog/introducing-nats-streaming/#why-nats-streaming
|
||||
.. [4] https://www.cloudamqp.com/blog/2018-01-08-part2-rabbitmq-best-practice-for-high-performance.html#avoid-multiple-nodes-ha
|
||||
.. [5] https://etherpad.openstack.org/p/JP-Ops-workshop-6-rabbit
|
||||
.. [6] https://github.com/yosshy/pynats2
|
||||
|
||||
.. note::
|
||||
|
||||
This work is licensed under a Creative Commons Attribution 3.0
|
||||
Unported License.
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
Loading…
Reference in New Issue