Firehose: A unified message bus for Infra services
This commit adds a spec for implementing a unified Infra message bus, the "firehose." Change-Id: I6295aec6025600a9ef02de5249a3a5c834dcba8c Story: #2000643
This commit is contained in:
Matthew Treinish
Jeremy Stanley
parent
d98fc41e73
commit
05b0c53133
|
|
@ -42,6 +42,7 @@ permits.
|
|||
specs/deploy-ci-dashboard
|
||||
specs/deploy-stackviz
|
||||
specs/doc-publishing
|
||||
specs/firehose
|
||||
specs/jenkins-job-builder_2.0.0-api-changes
|
||||
specs/nodepool-launch-workers
|
||||
specs/nodepool-workers
|
||||
|
|
|
|||
|
|
@ -0,0 +1,165 @@
|
|||
::
|
||||
This work is licensed under a Creative Commons Attribution 3.0
|
||||
Unported License.
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
==================================================
|
||||
Firehose: A unified message bus for Infra services
|
||||
==================================================
|
||||
|
||||
https://storyboard.openstack.org/#!/story/2000643
|
||||
|
||||
Using MQTT start running a unified message bus that community infrastructure
|
||||
services can emit events on. This will enable efficient external consumption of
|
||||
infrastructure events for experimentation and rapid innovation of tooling
|
||||
within the community.
|
||||
|
||||
Problem Description
|
||||
===================
|
||||
|
||||
There are a lot of services running in openstack-infra doing lots of different
|
||||
things all the time. Following events generated by these services can be
|
||||
tricky, and we don't have a consistent solution to making arbitrary state
|
||||
change details externally visible (so we fall back to relying on those with
|
||||
access to the machines letting us know what operations are being performed).
|
||||
What event data does exist is inconsistently presented in a variety of places
|
||||
over different protocols and requires significant effort to correlate
|
||||
cross-service.
|
||||
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
||||
The goal of this spec is to create a unified message bus that Infra services
|
||||
can publish event streams to. This will enable external public consumption of
|
||||
Infra events in a consistent manner.
|
||||
|
||||
To accomplish this we will deploy a new server and run the mosquitto_ MQTT_
|
||||
broker on it. MQTT protocol was designed in the late '90s to be lightweight
|
||||
pub-sub protocol that would work with connections to remote locations that
|
||||
needed a "small code footprint" and where there was limited network bandwith.
|
||||
More recently this has made it quite popular in the IoT space. However, it is
|
||||
also a good fit for the distributed cloud model that a lot of Infra services
|
||||
use. We will also deploy mosquitto using websockets_ this should also provide
|
||||
a benefit to international community members where things like the Gerrit event
|
||||
stream are problematic to access over Gerrit's SSH port.
|
||||
|
||||
.. _mosquitto: http://mosquitto.org/
|
||||
.. _MQTT: http://mqtt.org/
|
||||
.. _websockets: http://jpmens.net/2014/07/03/the-mosquitto-mqtt-broker-gets-websockets-support/
|
||||
|
||||
That will provide the base infrastructure for publishing and subscribing to
|
||||
events but nothing will be using MQTT after that. To start the event reporting
|
||||
we'll leverage germqtt_ (and make it an Infra project). This will be the
|
||||
initial usage for the unified message bus and make the Gerrit event stream
|
||||
available over MQTT. While the scope of this spec ends with getting Gerrit into
|
||||
MQTT, once we have an example we can more easily add more sources reporting
|
||||
directly into MQTT.
|
||||
|
||||
.. _germqtt: https://github.com/mtreinish/germqtt
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
Use fedmsg
|
||||
^^^^^^^^^^
|
||||
The fedora infrastructure team had a similar need/desire to have a unified
|
||||
message bus and they started the fedmsg_ project to address this need.
|
||||
The primary issue with fedmsg is that it depends on AMQP or ZMQ both of which
|
||||
we have had previous issues with. The choice to go with MQTT was partially a
|
||||
reaction to that.
|
||||
|
||||
.. _fedmsg: http://www.fedmsg.com/en/latest/
|
||||
|
||||
|
||||
Use a different mqtt broker
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
There are several open source MQTT brokers besides Mosquitto out there
|
||||
including ones like RabbitMQ_, VerneMQ_, and mosca_.
|
||||
|
||||
.. _RabbitMQ: http://www.rabbitmq.com/blog/2012/09/12/mqtt-adapter/
|
||||
.. _VerneMQ: https://github.com/erlio/vernemq
|
||||
.. _mosca: https://github.com/mcollina/mosca
|
||||
|
||||
Mosquitto was chosen because it's a lightweight C program which will need less
|
||||
resources to run it. It also should be simpler to deploy since we don't have to
|
||||
worry about deploying a JavaVM, nodejs, or erlang (which seems to be quite
|
||||
popular for writing message brokers).
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
|
||||
* mtreinish
|
||||
* fungi
|
||||
|
||||
|
||||
Gerrit Topic
|
||||
------------
|
||||
|
||||
Use Gerrit topic "firehose" for all patches related to this spec.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git-review -t firehose
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
#. Write puppet-mosquitto module
|
||||
#. Deploy firehose01 server with puppet-mosquitto module and a
|
||||
firehose.openstack.org CNAME to it
|
||||
#. Add germqtt to openstack-infra
|
||||
#. Write puppet-germqtt
|
||||
#. Deploy puppet-germqtt on firehose01
|
||||
|
||||
Repositories
|
||||
------------
|
||||
|
||||
2 New repos will be created in the initial implemention, puppet-mosquitto and
|
||||
puppet-germqtt.
|
||||
|
||||
Servers
|
||||
-------
|
||||
|
||||
1 new server will need to be created for running mosquitto, firehose01. Since
|
||||
mosquitto and MQTT is fairly lightweight we probably can get away with a
|
||||
smaller server (2GB flavor should be plenty).
|
||||
|
||||
DNS Entries
|
||||
-----------
|
||||
|
||||
Forward/reverse DNS entries will be added for firehose01.openstack.org, as well
|
||||
as a CNAME of firehose.openstack.org.
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
Initial service documentation will be added to the
|
||||
openstack-infra/system-config repo as usual. An availability announcement will
|
||||
be sent to the openstack-dev mailing list.
|
||||
|
||||
Security
|
||||
--------
|
||||
|
||||
Sensitive data should not be published over the message bus. It is public and
|
||||
anonymously accessible.
|
||||
|
||||
Testing
|
||||
-------
|
||||
|
||||
The new Puppet modules will run our usual Puppet test and style jobs, and the
|
||||
germqtt repo will have typical Python unit testing and style checks as well.
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
- This will be the first use of germqtt in infra (which will be added as an
|
||||
infra project)
|
||||
- 2 new puppet modules will be created, 1 to deploy mosquitto and the other to
|
||||
run germqtt
|
||||
Loading…
Reference in New Issue