Merge "Remove firehose.openstack.org"
This commit is contained in:
commit
cb5898ae0a
|
@ -1,366 +0,0 @@
|
||||||
:title: Firehose
|
|
||||||
|
|
||||||
.. _firehose:
|
|
||||||
|
|
||||||
Firehose
|
|
||||||
########
|
|
||||||
|
|
||||||
The unified message bus for Infra services.
|
|
||||||
|
|
||||||
At a Glance
|
|
||||||
===========
|
|
||||||
|
|
||||||
:Hosts:
|
|
||||||
* firehose*.openstack.org
|
|
||||||
:Puppet:
|
|
||||||
* https://opendev.org/opendev/puppet-mosquitto
|
|
||||||
* https://opendev.org/opendev/puppet-germqtt
|
|
||||||
* https://opendev.org/opendev/puppet-lpmqtt
|
|
||||||
* :git_file:`modules/openstack_project/manifests/firehose.pp`
|
|
||||||
:Projects:
|
|
||||||
* https://mosquitto.org/
|
|
||||||
* http://opendev.org/opendev/germqtt/
|
|
||||||
* http://opendev.org/opendev/lpmqtt/
|
|
||||||
|
|
||||||
Overview
|
|
||||||
========
|
|
||||||
|
|
||||||
The firehose is an infra run MQTT broker that is a place for any infra run
|
|
||||||
service to publish events to. The concept behind it is that if anything needs
|
|
||||||
to consume an event from an infra run service we should have a single place
|
|
||||||
to go for consuming them.
|
|
||||||
|
|
||||||
firehose.openstack.org hosts an instance of Mosquitto to be the MQTT broker
|
|
||||||
and also locally runs an instance of germqtt to publish the gerrit event
|
|
||||||
stream over MQTT and lpmqtt to publish a launchpad event stream over MQTT.
|
|
||||||
|
|
||||||
Connection Info
|
|
||||||
---------------
|
|
||||||
|
|
||||||
firehose.openstack.org has 2 open ports for MQTT traffic:
|
|
||||||
|
|
||||||
* **1883** - The default MQTT port
|
|
||||||
* **80** - Uses websockets for the MQTT communication
|
|
||||||
* **8883** - The default SSL/TLS MQTT port
|
|
||||||
* **443** - The SSL/TLS websockets port
|
|
||||||
|
|
||||||
|
|
||||||
Topics
|
|
||||||
------
|
|
||||||
|
|
||||||
Topics at a top level are set based on the name of the service publishing the
|
|
||||||
messages. The higher levels are specified by the publisher. For example::
|
|
||||||
|
|
||||||
gerrit/openstack-infra/germqtt/comment-added
|
|
||||||
|
|
||||||
is a typical message topic on firehose. The top level 'gerrit' specifies the
|
|
||||||
service the message is from, and the rest of the message comes from germqtt
|
|
||||||
(the daemon used for publishing the gerrit events)
|
|
||||||
|
|
||||||
MQTT topics are hierarchical and you can filter your subscription on part of the
|
|
||||||
hierarchy. `[1]`_
|
|
||||||
|
|
||||||
.. _[1]: https://mosquitto.org/man/mqtt-7.html
|
|
||||||
|
|
||||||
Services Publishing to firehose
|
|
||||||
-------------------------------
|
|
||||||
|
|
||||||
As of right now the following services publish messages to the firehose:
|
|
||||||
|
|
||||||
+-----------------+------------------+----------------------------+
|
|
||||||
| Service | Base Topic | Source of Messages |
|
|
||||||
+=================+==================+============================+
|
|
||||||
| ansible | ansible | `ansible_mqtt_plugin`_ |
|
|
||||||
+-----------------+------------------+----------------------------+
|
|
||||||
| gerrit | gerrit | `germqtt`_ |
|
|
||||||
+-----------------+------------------+----------------------------+
|
|
||||||
| launchpad | launchpad | `lpmqtt`_ |
|
|
||||||
+-----------------+------------------+----------------------------+
|
|
||||||
| subunit worker | gearman-subunit | `subunit-gearman-worker`_ |
|
|
||||||
+-----------------+------------------+----------------------------+
|
|
||||||
| logstash worker | gearman-logstash | `logstash-gearman-worker`_ |
|
|
||||||
+-----------------+------------------+----------------------------+
|
|
||||||
|
|
||||||
.. _germqtt: http://opendev.org/opendev/germqtt/
|
|
||||||
.. _lpmqtt: http://opendev.org/opendev/lpmqtt/
|
|
||||||
.. _subunit-gearman-worker: https://opendev.org/opendev/puppet-subunit2sql/src/branch/master/files/subunit-gearman-worker.py
|
|
||||||
.. _ansible_mqtt_plugin: https://opendev.org/opendev/system-config/src/branch/master/modules/openstack_project/files/puppetmaster/mqtt.py
|
|
||||||
.. _logstash-gearman-worker: https://opendev.org/opendev/puppet-log_processor/src/branch/master/files/log-gearman-worker.py
|
|
||||||
|
|
||||||
For a full schema description see :ref:`firehose_schema`
|
|
||||||
|
|
||||||
Client Usage
|
|
||||||
============
|
|
||||||
There is no outside access to publishing messages to the firehose available,
|
|
||||||
however anyone is able to subscribe to any topic services publish to. To
|
|
||||||
interact with the firehose you need to use the MQTT protocol. The specific
|
|
||||||
contents of the payload are dictated by the service publishing the
|
|
||||||
messages. So this section only covers how to subscribe and receive the messages
|
|
||||||
not how to consume the content received.
|
|
||||||
|
|
||||||
Available Clients
|
|
||||||
-----------------
|
|
||||||
The MQTT community wiki maintains a page that lists available client bindings
|
|
||||||
for many languages here: https://github.com/mqtt/mqtt.github.io/wiki/libraries
|
|
||||||
For python using the `paho-mqtt`_ library is recommended
|
|
||||||
|
|
||||||
.. _paho-mqtt: https://pypi.org/project/paho-mqtt/
|
|
||||||
|
|
||||||
CLI Example
|
|
||||||
-----------
|
|
||||||
The mosquitto project also provides both a CLI publisher and subscriber client
|
|
||||||
that can be used to easily subscribe to any topic and receive the messages. On
|
|
||||||
debian based distributions these are included in the mosquitto-clients package.
|
|
||||||
For example, to subscribe to every topic on the firehose you would run::
|
|
||||||
|
|
||||||
mosquitto_sub -h firehose.openstack.org --topic '#'
|
|
||||||
|
|
||||||
You can adjust the value of the topic parameter to make what you're subscribing
|
|
||||||
to more specific.
|
|
||||||
|
|
||||||
MQTT Protocol Example
|
|
||||||
---------------------
|
|
||||||
Interacting with firehose on the unencrypted MQTT port is normally pretty easy in
|
|
||||||
most language bindings. Here are some examples that will have the same behavior
|
|
||||||
as the CLI example above and will subscribe to all topics on the firehose and
|
|
||||||
print it to STDOUT.
|
|
||||||
|
|
||||||
|
|
||||||
Python
|
|
||||||
''''''
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
import paho.mqtt.client as mqtt
|
|
||||||
|
|
||||||
|
|
||||||
def on_connect(client, userdata, flags, rc):
|
|
||||||
print("Connected with result code " + str(rc))
|
|
||||||
client.subscribe('#')
|
|
||||||
|
|
||||||
def on_message(client, userdata, msg):
|
|
||||||
print(msg.topic+" "+str(msg.payload))
|
|
||||||
|
|
||||||
# Create a websockets client
|
|
||||||
client = mqtt.Client()
|
|
||||||
client.on_connect = on_connect
|
|
||||||
client.on_message = on_message
|
|
||||||
|
|
||||||
# Connect to the firehose
|
|
||||||
client.connect('firehose.openstack.org')
|
|
||||||
# Listen forever
|
|
||||||
client.loop_forever()
|
|
||||||
|
|
||||||
Haskell
|
|
||||||
'''''''
|
|
||||||
This requires the `mqtt-hs`_ library to be installed.
|
|
||||||
|
|
||||||
.. _mqtt-hs: https://hackage.haskell.org/package/mqtt-hs
|
|
||||||
|
|
||||||
.. code-block:: haskell
|
|
||||||
|
|
||||||
{-# Language DataKinds, OverloadedStrings #-}
|
|
||||||
|
|
||||||
module Subscribe where
|
|
||||||
|
|
||||||
import Control.Concurrent
|
|
||||||
import Control.Concurrent.STM
|
|
||||||
import Control.Monad (unless, forever)
|
|
||||||
import System.Exit (exitFailure)
|
|
||||||
import System.IO (hPutStrLn, stderr)
|
|
||||||
|
|
||||||
import qualified Network.MQTT as MQTT
|
|
||||||
|
|
||||||
topic :: MQTT.Topic
|
|
||||||
topic = "#"
|
|
||||||
|
|
||||||
handleMsg :: MQTT.Message MQTT.PUBLISH -> IO ()
|
|
||||||
handleMsg msg = do
|
|
||||||
let t = MQTT.topic $ MQTT.body msg
|
|
||||||
p = MQTT.payload $ MQTT.body msg
|
|
||||||
print t
|
|
||||||
print p
|
|
||||||
|
|
||||||
main :: IO ()
|
|
||||||
main = do
|
|
||||||
cmds <- MQTT.mkCommands
|
|
||||||
pubChan <- newTChanIO
|
|
||||||
let conf = (MQTT.defaultConfig cmds pubChan)
|
|
||||||
{ MQTT.cHost = "firehose.openstack.org" }
|
|
||||||
_ <- forkIO $ do
|
|
||||||
qosGranted <- MQTT.subscribe conf [(topic, MQTT.Handshake)]
|
|
||||||
forever $ atomically (readTChan pubChan) >>= handleMsg
|
|
||||||
terminated <- MQTT.run conf
|
|
||||||
print terminated
|
|
||||||
|
|
||||||
Go
|
|
||||||
''
|
|
||||||
.. code-block:: go
|
|
||||||
|
|
||||||
package main
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
MQTT "github.com/eclipse/paho.mqtt.golang"
|
|
||||||
"os"
|
|
||||||
"strconv"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
func onMessageReceived(client MQTT.Client, msg MQTT.Message) {
|
|
||||||
fmt.Printf("TOPIC: %s\n", msg.Topic())
|
|
||||||
fmt.Printf("MSG: %s\n", msg.Payload())
|
|
||||||
}
|
|
||||||
func main() {
|
|
||||||
hostname, _ := os.Hostname()
|
|
||||||
opts := &MQTT.ClientOptions{
|
|
||||||
ClientID: hostname+strconv.Itoa(time.Now().Second()),
|
|
||||||
}
|
|
||||||
opts.AddBroker("tcp://firehose.openstack.org:1883")
|
|
||||||
opts.OnConnect = func(c MQTT.Client) {
|
|
||||||
if token := c.Subscribe("#", 0, onMessageReceived); token.Wait() && token.Error() != nil {
|
|
||||||
fmt.Println(token.Error())
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
client := MQTT.NewClient(opts)
|
|
||||||
if token := client.Connect(); token.Wait() && token.Error() != nil {
|
|
||||||
panic(token.Error())
|
|
||||||
}
|
|
||||||
for {
|
|
||||||
time.Sleep(1 * time.Second)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
Websocket Example
|
|
||||||
-----------------
|
|
||||||
In addition to using the raw MQTT protocol firehose.o.o provides a websocket
|
|
||||||
interface on port 80 that MQTT traffic can go through. This is especially useful
|
|
||||||
for web applications that intend to consume any events from MQTT. To see an
|
|
||||||
example of this in action you can try: http://mitsuruog.github.io/what-mqtt/
|
|
||||||
(the source is available here: https://github.com/mitsuruog/what-mqtt) and use
|
|
||||||
that to subscribe to any topics on firehose.openstack.org.
|
|
||||||
|
|
||||||
Another advantage of using websockets over port 80 is that it's much more
|
|
||||||
firewall friendly, especially in environments that are more locked down. If you
|
|
||||||
would like to consume events from the firehose and are concerned about a
|
|
||||||
firewall blocking your access, the websocket interface is a good choice.
|
|
||||||
|
|
||||||
You can also use the paho-mqtt python library to subscribe to mqtt over
|
|
||||||
websockets fairly easily. For example this script will subscribe to all topics
|
|
||||||
on the firehose and print it to STDOUT
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
:emphasize-lines: 12,17
|
|
||||||
|
|
||||||
import paho.mqtt.client as mqtt
|
|
||||||
|
|
||||||
|
|
||||||
def on_connect(client, userdata, flags, rc):
|
|
||||||
print("Connected with result code " + str(rc))
|
|
||||||
client.subscribe('#')
|
|
||||||
|
|
||||||
def on_message(client, userdata, msg):
|
|
||||||
print(msg.topic+" "+str(msg.payload))
|
|
||||||
|
|
||||||
# Create a websockets client
|
|
||||||
client = mqtt.Client(transport="websockets")
|
|
||||||
client.on_connect = on_connect
|
|
||||||
client.on_message = on_message
|
|
||||||
|
|
||||||
# Connect to the firehose
|
|
||||||
client.connect('firehose.openstack.org', port=80)
|
|
||||||
# Listen forever
|
|
||||||
client.loop_forever()
|
|
||||||
|
|
||||||
Using SSL/TLS
|
|
||||||
-------------
|
|
||||||
If you would like to connect to the firehose using ssl to encrypt the events you
|
|
||||||
receive from MQTT you just need to connect with ssl enabled via either of the
|
|
||||||
encrypted ports. If you'd like to verify the server ssl certificate when
|
|
||||||
connecting you'll need to provide a CA bundle to use as most MQTT clients do
|
|
||||||
not know how to use the system trusted CA bundle like most http clients.
|
|
||||||
|
|
||||||
To connect to the firehose and subscribe to all topics you can use the
|
|
||||||
mosquitto CLI client::
|
|
||||||
|
|
||||||
mosquitto_sub --topic '#' -h firehose.openstack.org --cafile /etc/ca-certificates/extracted/tls-ca-bundle.pem -p 8883
|
|
||||||
|
|
||||||
You can use python:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
:emphasize-lines: 15,20
|
|
||||||
|
|
||||||
import paho.mqtt.client as mqtt
|
|
||||||
|
|
||||||
|
|
||||||
def on_connect(client, userdata, flags, rc):
|
|
||||||
print("Connected with result code " + str(rc))
|
|
||||||
client.subscribe('#')
|
|
||||||
|
|
||||||
|
|
||||||
def on_message(client, userdata, msg):
|
|
||||||
print(msg.topic+" "+str(msg.payload))
|
|
||||||
|
|
||||||
|
|
||||||
# Create an SSL encrypted websockets client
|
|
||||||
client = mqtt.Client()
|
|
||||||
client.tls_set(ca_certs='/etc/ca-certificates/extracted/tls-ca-bundle.pem')
|
|
||||||
client.on_connect = on_connect
|
|
||||||
client.on_message = on_message
|
|
||||||
|
|
||||||
# Connect to the firehose
|
|
||||||
client.connect('firehose.openstack.org', port=8883)
|
|
||||||
client.loop_forever()
|
|
||||||
|
|
||||||
|
|
||||||
Or with ruby:
|
|
||||||
|
|
||||||
.. code-block:: ruby
|
|
||||||
:emphasize-lines: 6,7,8
|
|
||||||
|
|
||||||
require 'rubygems'
|
|
||||||
require 'mqtt'
|
|
||||||
|
|
||||||
client = MQTT::Client.new
|
|
||||||
client.host = 'firehose.openstack.org'
|
|
||||||
client.ssl = true
|
|
||||||
client.cert_file = '/etc/ca-certificates/extracted/tls-ca-bundle.pem'
|
|
||||||
client.port = 8883
|
|
||||||
client.connect()
|
|
||||||
client.subscribe('#')
|
|
||||||
|
|
||||||
client.get do |topic,message|
|
|
||||||
puts message
|
|
||||||
end
|
|
||||||
|
|
||||||
Example Use Cases
|
|
||||||
=================
|
|
||||||
|
|
||||||
Event Notifications
|
|
||||||
-------------------
|
|
||||||
|
|
||||||
A common use case for the event bus is to get a notification when an event
|
|
||||||
occurs. There is an open source tool, `mqttwarn`_ that makes setting this up
|
|
||||||
off the firehose (or any other mqtt broker) very straightforward.
|
|
||||||
|
|
||||||
.. _mqttwarn: https://github.com/jpmens/mqttwarn
|
|
||||||
|
|
||||||
You can use mqttwarn to setup custom notifications to a large number of tools
|
|
||||||
and services. (both local and remote). You can read the full docs on how to
|
|
||||||
configure and use mqttwarn at https://github.com/jpmens/mqttwarn/wiki and
|
|
||||||
https://github.com/jpmens/mqttwarn/blob/master/README.md
|
|
||||||
|
|
||||||
|
|
||||||
IMAP and MX
|
|
||||||
===========
|
|
||||||
|
|
||||||
We're using Cyrus as an IMAP server in order to consume launchpad bug
|
|
||||||
events via email. The configuration of the admin password account and
|
|
||||||
creation of the lpmqtt user for Cyrus were completed using the
|
|
||||||
following::
|
|
||||||
|
|
||||||
$ sudo saslpasswd2 cyrus
|
|
||||||
$ cyradm --user=cyrus --server=localhost
|
|
||||||
Password:
|
|
||||||
localhost> create user.lpmqtt
|
|
||||||
|
|
||||||
An MX record has also been set up to point to the firehose server.
|
|
|
@ -1,434 +0,0 @@
|
||||||
:title: Firehose Schema
|
|
||||||
|
|
||||||
.. _firehose_schema:
|
|
||||||
|
|
||||||
Firehose Schema
|
|
||||||
###############
|
|
||||||
|
|
||||||
This attempts to document the topic and payload schema for all the services
|
|
||||||
reporting to the firehose. However since much of what is reported to firehose
|
|
||||||
is dynamically generated it is possible this document misses a case.
|
|
||||||
|
|
||||||
Gerrit
|
|
||||||
======
|
|
||||||
|
|
||||||
Messages on firehose for gerrit are generated using the `germqtt`_ project. For
|
|
||||||
the most part these are basically identical to what gerrit returns on it's
|
|
||||||
native event stream except over MQTT.
|
|
||||||
|
|
||||||
.. _germqtt: http://opendev.org/opendev/germqtt/
|
|
||||||
|
|
||||||
Topics
|
|
||||||
------
|
|
||||||
|
|
||||||
The topics for gerrit are generated dynamically. However, they follow a fairly
|
|
||||||
straightforward pattern. The basic formula for this is::
|
|
||||||
|
|
||||||
gerrit/<git namespace>/<repo name>/<gerrit event>
|
|
||||||
|
|
||||||
So for example a typical topic would be::
|
|
||||||
|
|
||||||
gerrit/openstack/nova/comment-added
|
|
||||||
|
|
||||||
The ``git namespace`` and ``repo name`` are pretty self explanatory and are just
|
|
||||||
from the git repository the change in gerrit is for. The event is defined in the gerrit event stream. You can see the full reference for topics in the Gerrit
|
|
||||||
docs for `Gerrit events`_. However, for simplicity the possible values are:
|
|
||||||
|
|
||||||
* change-abandoned
|
|
||||||
* change-merged
|
|
||||||
* change-restored
|
|
||||||
* comment-added
|
|
||||||
* draft-published
|
|
||||||
* hashtags-changed
|
|
||||||
* merge-failed
|
|
||||||
* patchset-created
|
|
||||||
* ref-updated
|
|
||||||
* reviewer-added
|
|
||||||
* topic-changed
|
|
||||||
|
|
||||||
Payload
|
|
||||||
-------
|
|
||||||
The payload for gerrit messages are basically the same JSON that gets returned
|
|
||||||
by gerrit's event stream command. Instead of repeating the entire gerrit schema
|
|
||||||
doc here just refer to gerrit's docs on the `JSON payload`_ which documents the
|
|
||||||
contents of each JSON object and refer to the doc on `Gerrit events`_ for which
|
|
||||||
JSON objects are included with which event type.
|
|
||||||
|
|
||||||
.. _JSON payload: https://review.opendev.org/Documentation/json.html
|
|
||||||
.. _Gerrit events: https://review.opendev.org/Documentation/cmd-stream-events.html#events
|
|
||||||
|
|
||||||
Launchpad
|
|
||||||
=========
|
|
||||||
The messages sent to firehose for launchpad are generated using `lpmqtt`_
|
|
||||||
|
|
||||||
.. _lpmqtt: http://opendev.org/opendev/lpmqtt/
|
|
||||||
|
|
||||||
Topics
|
|
||||||
------
|
|
||||||
|
|
||||||
The topics for lpmqtt follow a pretty simple formula::
|
|
||||||
|
|
||||||
launchpad/<project>/<event type>/<bug number>
|
|
||||||
|
|
||||||
the ``project`` is the launchpad project name, ``event type`` will always be
|
|
||||||
"bug" (or not present). The intent of this was to be "bug" or "blueprint", but
|
|
||||||
due to limitations in launchpad getting notifications from blueprints is not
|
|
||||||
possible. The flexibility was left in the schema just in case this ever changes.
|
|
||||||
The ``bug number`` is obviously the bug number from launchpad.
|
|
||||||
|
|
||||||
It's also worth noting that only the base topic is a guaranteed field. Depending
|
|
||||||
on the notification email from launchpad some of the other fields may not be
|
|
||||||
present. In those cases the topic will be populated left to right until a
|
|
||||||
missing field is encountered.
|
|
||||||
|
|
||||||
Payload
|
|
||||||
-------
|
|
||||||
|
|
||||||
The payload of messages is dynamically generated and dependent on the
|
|
||||||
notification received from launchpad, and launchpad isn't always consistent in
|
|
||||||
what fields are present in those notifications.
|
|
||||||
|
|
||||||
However, for bug event types there is a standard format. The fields which
|
|
||||||
are always present for bugs (which should normally be the only message for
|
|
||||||
firehose) are:
|
|
||||||
|
|
||||||
* commenters
|
|
||||||
* bug-reporter
|
|
||||||
* bug-modifier
|
|
||||||
* bug-number
|
|
||||||
* event-type
|
|
||||||
|
|
||||||
The remaining fields are dynamic and depend on launchpad. An example message
|
|
||||||
payload (with the body trimmed) for a bug is::
|
|
||||||
|
|
||||||
{
|
|
||||||
"status": "Triaged",
|
|
||||||
"project": "octavia",
|
|
||||||
"assignee": "email@fakedomain.com",
|
|
||||||
"bug-reporter": "Full name (username)",
|
|
||||||
"event-type": "bug",
|
|
||||||
"bug-number": "1680938",
|
|
||||||
"commenters": ["username"]
|
|
||||||
"tags": ["rfe"],
|
|
||||||
"importance": "Medium",
|
|
||||||
"bug-modifier": "Full Name (username)",
|
|
||||||
"body": "notification body, often is just bug comment or summary",
|
|
||||||
}
|
|
||||||
|
|
||||||
Subunit Workers
|
|
||||||
===============
|
|
||||||
|
|
||||||
The messages for the subunit workers are generated directly in the
|
|
||||||
`subunit gearman worker scripts`_.
|
|
||||||
|
|
||||||
.. _subunit gearman worker scripts: https://opendev.org/opendev/puppet-subunit2sql/src/branch/master/files/subunit-gearman-worker.py
|
|
||||||
|
|
||||||
Topics
|
|
||||||
------
|
|
||||||
|
|
||||||
The topics for the subunit workers follow a simple pattern::
|
|
||||||
|
|
||||||
gearman-subunit/<worker hostname>/<git namespace/<repo name>/<change number>
|
|
||||||
|
|
||||||
Where ``worker hostname`` is the host which processed the subunit file, as
|
|
||||||
of right now there are 2, subunit-worker01 and subunit-worker02, but there may
|
|
||||||
be more (or fewer) in the future. The ``git namespace`` and ``repo name`` are
|
|
||||||
pretty self explanatory, and are just for the git repo under test that the
|
|
||||||
subunit was emitted from. ``change number`` is the gerrit change number for the
|
|
||||||
job that launched the tests the subunit is for.
|
|
||||||
|
|
||||||
Payload
|
|
||||||
-------
|
|
||||||
The payload for the messages from the subunit workers is pretty straightforward
|
|
||||||
json that contains 3 fields: ``status``, ``build_uuid``, and ``source_url``.
|
|
||||||
|
|
||||||
An example is::
|
|
||||||
|
|
||||||
{
|
|
||||||
'status': 'success',
|
|
||||||
'build_uuid': '45f7c1ddbfd74c6aba94662623bd61b8'
|
|
||||||
'source_url': 'A url',
|
|
||||||
}
|
|
||||||
|
|
||||||
Ansible
|
|
||||||
=======
|
|
||||||
|
|
||||||
We have mqtt events emitted from ansible being run on :ref:`bridge`.
|
|
||||||
These events are generated using a `MQTT Ansible Callback Plugin`_.
|
|
||||||
|
|
||||||
.. _MQTT Ansible Callback Plugin: https://opendev.org/opendev/system-config/src/branch/master/modules/openstack_project/files/puppetmaster/mqtt.py
|
|
||||||
|
|
||||||
Topics
|
|
||||||
------
|
|
||||||
|
|
||||||
The topics for ansible are a bit more involved than some of the other services
|
|
||||||
publishing to firehose. It depends on the type of event that ansible just
|
|
||||||
finished. There are 3 categories of events which have slightly different topic
|
|
||||||
formulas (and payloads).
|
|
||||||
|
|
||||||
Playbook Events
|
|
||||||
'''''''''''''''
|
|
||||||
Whenever a playbook action occurs the callback plugin will emit an event for
|
|
||||||
it. The topics for playbook events fall into this pattern::
|
|
||||||
|
|
||||||
ansible/playbook/<playbook uuid>/action/<playbook action>/<status>
|
|
||||||
|
|
||||||
``playbook uuid`` is pretty self explanatory here, it's the uuid ansible uses
|
|
||||||
to uniquely identify the playbook being run. ``playbook action`` is the action
|
|
||||||
that the event is for, this is either going to be ``start`` or ``finish``.
|
|
||||||
``status`` is only set on ``finish`` and will be one of the following:
|
|
||||||
|
|
||||||
* ``OK``
|
|
||||||
* ``FAILED``
|
|
||||||
|
|
||||||
to indicate whether the playbook successfully executed or not.
|
|
||||||
|
|
||||||
Playbook Stats Events
|
|
||||||
'''''''''''''''''''''
|
|
||||||
|
|
||||||
At the end of a playbook these events are emitted for each host that tasks were
|
|
||||||
run on. The topics for these events fall into the following pattern::
|
|
||||||
|
|
||||||
ansible/playbook/<playbook uuid>/stats/<hostname>
|
|
||||||
|
|
||||||
In this case ``playbook uuid`` is the same as above and the internal ansible
|
|
||||||
unique playbook identifier. ``hostname`` here is the host that ansible was
|
|
||||||
running tasks on as part of the playbook.
|
|
||||||
|
|
||||||
Task Events
|
|
||||||
'''''''''''
|
|
||||||
|
|
||||||
At the end of each individual task the callback plugin will emit an event. Those
|
|
||||||
events' topics fall into the following pattern::
|
|
||||||
|
|
||||||
ansible/playbook/<playbook uuid>/task/<hostname>/<status>
|
|
||||||
|
|
||||||
``playbook uuid`` is the same as in the previous 2 event types. ``hostname`` is
|
|
||||||
the hostname the task was executed on. ``status`` is the result of the task
|
|
||||||
and will be one of the following:
|
|
||||||
|
|
||||||
* ``OK``
|
|
||||||
* ``FAILED``
|
|
||||||
* ``UNREACHABLE``
|
|
||||||
|
|
||||||
Payload
|
|
||||||
-------
|
|
||||||
|
|
||||||
Just as with the topics the message payloads depend on the event type. Each
|
|
||||||
event uses a JSON payload with slightly different fields.
|
|
||||||
|
|
||||||
Playbook Events
|
|
||||||
'''''''''''''''
|
|
||||||
|
|
||||||
For playbook events the payload falls into this schema on playbook starts::
|
|
||||||
|
|
||||||
{
|
|
||||||
"status": "OK",
|
|
||||||
"host": <hostname>
|
|
||||||
"session": <session id>,
|
|
||||||
"playbook_name": <playbook name>,
|
|
||||||
"playbook_id": <playbook uuid>,
|
|
||||||
"ansible_type": "start",
|
|
||||||
}
|
|
||||||
|
|
||||||
When a playbook finishes the payload is slightly smaller and the schema is::
|
|
||||||
|
|
||||||
{
|
|
||||||
"playbook_id": <playbook uuid>,
|
|
||||||
"playbook_name": <playbook name>,
|
|
||||||
"status": <status>,
|
|
||||||
}
|
|
||||||
|
|
||||||
In both cases ``playbook uuid`` is the same field from the topic.
|
|
||||||
``playbook name`` is the human readable name for the playbook. If one is
|
|
||||||
set in the playbook this will be that. ``status`` will be whether the
|
|
||||||
playbook was successfully executed or not. It will always be ``OK`` on starts
|
|
||||||
(otherwise the event isn't emitted) but on failures, just like in the topic,
|
|
||||||
this will be one of the following:
|
|
||||||
|
|
||||||
* ``OK``
|
|
||||||
* ``FAILED``
|
|
||||||
|
|
||||||
``session id`` is a UUID generated by the callback plugin to uniquely identify
|
|
||||||
the execution of the playbook. ``hostname`` is the hostname where the ansible
|
|
||||||
playbook was launched. (which is not necessarily where tasks are being run)
|
|
||||||
|
|
||||||
|
|
||||||
An example of this from the system is for a start event::
|
|
||||||
|
|
||||||
{
|
|
||||||
"status": "OK",
|
|
||||||
"playbook_name": "localhost:!disabled",
|
|
||||||
"ansible_type": "start",
|
|
||||||
"host": "puppetmaster.openstack.org",
|
|
||||||
"session": "14d6e568-2c75-11e7-bd24-bc764e048db9",
|
|
||||||
"playbook_id": "5a95e9da-8d33-4dbb-a8b3-a77affc065d0"
|
|
||||||
}
|
|
||||||
|
|
||||||
and for a finish::
|
|
||||||
|
|
||||||
{
|
|
||||||
"status": "FAILED",
|
|
||||||
"playbook_name": "compute*.ic.openstack.org:!disabled",
|
|
||||||
"playbook_id": "b259ac6d-6cb5-4403-bb8d-0ff2131c3d7a"
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
Playbook Stats Events
|
|
||||||
'''''''''''''''''''''
|
|
||||||
|
|
||||||
The schema for stats events is::
|
|
||||||
|
|
||||||
{
|
|
||||||
"host": <hostname>,
|
|
||||||
"ansible_host": <execute hostname>,
|
|
||||||
"playbook_id": <playbook uuid>,
|
|
||||||
"playbook_name": <playbook name>,
|
|
||||||
"stats": {
|
|
||||||
"unreachable": int,
|
|
||||||
"skipped": int,
|
|
||||||
"ok": int,
|
|
||||||
"changed": int,
|
|
||||||
"failures": int,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
``playbook uuid`` is the same field from the topic. ``playbook name`` is the
|
|
||||||
human readable name for the playbook. If one is set in the playbook this will be
|
|
||||||
that. ``execute hostname`` is the hostname where the tasks were being executed,
|
|
||||||
while ``hostname`` is the hostname where ansible launched the playbook. The
|
|
||||||
``stats`` subdict contains the task status counts where the key is the tasks
|
|
||||||
statuses.
|
|
||||||
|
|
||||||
An example from the running system is::
|
|
||||||
|
|
||||||
{
|
|
||||||
"playbook_name": "compute*.ic.openstack.org:!disabled",
|
|
||||||
"host": "puppetmaster.openstack.org",
|
|
||||||
"stats": {
|
|
||||||
"unreachable": 0,
|
|
||||||
"skipped": 5,
|
|
||||||
"ok": 13,
|
|
||||||
"changed": 1,
|
|
||||||
"failures": 0
|
|
||||||
},
|
|
||||||
"playbook_id": "b259ac6d-6cb5-4403-bb8d-0ff2131c3d7a",
|
|
||||||
"ansible_host": "controller00.vanilla.ic.openstack.org"
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
Task Events
|
|
||||||
'''''''''''
|
|
||||||
|
|
||||||
The schema for tasks events is::
|
|
||||||
|
|
||||||
{
|
|
||||||
"status": <status>,
|
|
||||||
"host": <hostname>,
|
|
||||||
"ansible_host": <execute hostname>,
|
|
||||||
"session": <session id>,
|
|
||||||
"ansible_type": "task",
|
|
||||||
"playbook_name": <playbook name>,
|
|
||||||
"playbook_id": <playbook uuid>,
|
|
||||||
"ansible_task": <task name>,
|
|
||||||
"ansible_result": <ansible result>
|
|
||||||
}
|
|
||||||
|
|
||||||
``playbook uuid`` is the same field from the topic. ``playbook name`` is the
|
|
||||||
human readable name for the playbook. If one is set in the playbook this will be
|
|
||||||
that. ``execute hostname`` is the hostname where the tasks were being executed,
|
|
||||||
while ``hostname`` is the hostname where ansible launched the playbook. ``task
|
|
||||||
name``, like the name implies, is the human readable name of the task executed.
|
|
||||||
If one was specified in the playbook that will be the value. ``status`` is the
|
|
||||||
result of the task and will be one of the following:
|
|
||||||
|
|
||||||
* ``OK``
|
|
||||||
* ``FAILED``
|
|
||||||
* ``UNREACHABLE``
|
|
||||||
|
|
||||||
``session id`` is a UUID generated by the callback plugin to uniquely identify
|
|
||||||
the execution of the playbook.
|
|
||||||
|
|
||||||
``ansible result`` is a free form subdict that comes directly from ansible to
|
|
||||||
completely describe the task that just finished. The structure here is fully
|
|
||||||
dependent on ansible internals and the way that the task was invoked in the
|
|
||||||
playbook. Note, that sometimes this can be quite large in size depending on the
|
|
||||||
task and whether facts were enabled or not.
|
|
||||||
|
|
||||||
An example of a task event from the running system is::
|
|
||||||
|
|
||||||
{
|
|
||||||
"status": "OK",
|
|
||||||
"host": "puppetmaster.openstack.org",
|
|
||||||
"session": "092aa3fa-2c73-11e7-bd24-bc764e048db9",
|
|
||||||
"playbook_name": "compute*.ic.openstack.org:!disabled",
|
|
||||||
"ansible_result": {
|
|
||||||
"_ansible_parsed": true,
|
|
||||||
"_ansible_no_log": false,
|
|
||||||
"stdout": "",
|
|
||||||
"changed": false,
|
|
||||||
"stderr": "",
|
|
||||||
"rc": 0,
|
|
||||||
"invocation": {
|
|
||||||
"module_name": "puppet",
|
|
||||||
"module_args": {
|
|
||||||
"logdest": "syslog",
|
|
||||||
"execute": null,
|
|
||||||
"facter_basename": "ansible",
|
|
||||||
"tags": null,
|
|
||||||
"puppetmaster": null,
|
|
||||||
"show_diff": false,
|
|
||||||
"certname": null,
|
|
||||||
"manifest": "/opt/system-config/manifests/site.pp",
|
|
||||||
"environment": "production",
|
|
||||||
"debug": false,
|
|
||||||
"noop": false,
|
|
||||||
"timeout": "30m",
|
|
||||||
"facts": null
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"stdout_lines": []
|
|
||||||
},
|
|
||||||
"ansible_type": "task",
|
|
||||||
"ansible_task": "TASK: puppet : run puppet",
|
|
||||||
"playbook_id": "b259ac6d-6cb5-4403-bb8d-0ff2131c3d7a",
|
|
||||||
"ansible_host": "compute014.chocolate.ic.openstack.org"
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
Logstash Workers
|
|
||||||
================
|
|
||||||
|
|
||||||
The messages for the subunit workers are generated directly in the
|
|
||||||
`logstash gearman worker scripts`_.
|
|
||||||
|
|
||||||
.. _logstash gearman worker scripts: https://opendev.org/opendev/puppet-log_processor/src/branch/master/files/log-gearman-worker.py
|
|
||||||
|
|
||||||
Topics
|
|
||||||
------
|
|
||||||
|
|
||||||
The topics for the subunit workers follow a simple pattern::
|
|
||||||
|
|
||||||
gearman-logstash/<worker hostname>/<git namespace>/<repo name>/<change number>/<action>
|
|
||||||
|
|
||||||
Where ``worker hostname`` is the host which processed the log file. The
|
|
||||||
``git namespace`` and ``repo name`` are pretty self explanatory, and are just
|
|
||||||
for the git repo under test that the log was generated. ``change number`` is
|
|
||||||
the gerrit change number for the job that launched the tests the subunit is for.
|
|
||||||
In the case of periodic or post queue jobs, this will either say ``periodic`` or
|
|
||||||
``post`` because there isn't an associated change number. The action field is
|
|
||||||
the phase of the log processing that just completed. Right now the only
|
|
||||||
possible value is ``retrieve_logs`` but there may be others in the future.
|
|
||||||
|
|
||||||
Payload
|
|
||||||
-------
|
|
||||||
The payload for the messages from the logstash workers is pretty straightforward
|
|
||||||
json that contains 3 fields: ``status``, ``build_uuid``, and ``source_url``.
|
|
||||||
|
|
||||||
An example is::
|
|
||||||
|
|
||||||
{
|
|
||||||
'status': 'success',
|
|
||||||
'build_uuid': '45f7c1ddbfd74c6aba94662623bd61b8'
|
|
||||||
'source_url': 'A url',
|
|
||||||
}
|
|
|
@ -37,8 +37,6 @@ Major Systems
|
||||||
refstack
|
refstack
|
||||||
codesearch
|
codesearch
|
||||||
signing
|
signing
|
||||||
firehose
|
|
||||||
firehose_schema
|
|
||||||
github
|
github
|
||||||
activity
|
activity
|
||||||
asterisk
|
asterisk
|
||||||
|
|
|
@ -222,7 +222,6 @@ cacti_hosts:
|
||||||
- elasticsearch07.openstack.org
|
- elasticsearch07.openstack.org
|
||||||
- ethercalc02.openstack.org
|
- ethercalc02.openstack.org
|
||||||
- etherpad01.opendev.org
|
- etherpad01.opendev.org
|
||||||
- firehose01.openstack.org
|
|
||||||
- gitea-lb01.opendev.org
|
- gitea-lb01.opendev.org
|
||||||
- gitea01.opendev.org
|
- gitea01.opendev.org
|
||||||
- gitea02.opendev.org
|
- gitea02.opendev.org
|
||||||
|
|
|
@ -154,13 +154,6 @@ all:
|
||||||
region_name: DFW
|
region_name: DFW
|
||||||
public_v4: 104.130.124.120
|
public_v4: 104.130.124.120
|
||||||
public_v6: 2001:4800:7818:104:be76:4eff:fe02:b0ff
|
public_v6: 2001:4800:7818:104:be76:4eff:fe02:b0ff
|
||||||
firehose01.openstack.org:
|
|
||||||
ansible_host: 104.130.155.115
|
|
||||||
location:
|
|
||||||
cloud: openstackci-rax
|
|
||||||
region_name: DFW
|
|
||||||
public_v4: 104.130.155.115
|
|
||||||
public_v6: 2001:4800:7818:103:be76:4eff:fe04:40aa
|
|
||||||
gitea-lb01.opendev.org:
|
gitea-lb01.opendev.org:
|
||||||
ansible_host: 38.108.68.124
|
ansible_host: 38.108.68.124
|
||||||
location:
|
location:
|
||||||
|
|
|
@ -1,25 +0,0 @@
|
||||||
exim_local_domains: "@:firehose.openstack.org"
|
|
||||||
# TODO(jeblair): have the cyrus router check to see if there is a
|
|
||||||
# cyrus account.
|
|
||||||
exim_routers:
|
|
||||||
- dnslookup: '{{ exim_dnslookup_router }}'
|
|
||||||
- system_aliases: '{{ exim_system_aliases_router }}'
|
|
||||||
- cyrus: |
|
|
||||||
driver = accept
|
|
||||||
domains = +local_domains
|
|
||||||
local_part_suffix = +*
|
|
||||||
local_part_suffix_optional
|
|
||||||
transport = cyrus
|
|
||||||
- localuser: '{{ exim_localuser_router }}'
|
|
||||||
exim_transports:
|
|
||||||
- cyrus: |
|
|
||||||
driver = lmtp
|
|
||||||
socket = /var/run/cyrus/socket/lmtp
|
|
||||||
user = cyrus
|
|
||||||
batch_max = 35
|
|
||||||
iptables_extra_public_tcp_ports:
|
|
||||||
- 25
|
|
||||||
- 80
|
|
||||||
- 443
|
|
||||||
- 1883
|
|
||||||
- 8883
|
|
|
@ -7,7 +7,6 @@ iptables_extra_allowed_hosts:
|
||||||
protocol: udp
|
protocol: udp
|
||||||
|
|
||||||
iptables_extra_allowed_groups:
|
iptables_extra_allowed_groups:
|
||||||
- {'protocol': 'udp', 'port': '8125', 'group': 'firehose'}
|
|
||||||
- {'protocol': 'udp', 'port': '8125', 'group': 'mirror-update'}
|
- {'protocol': 'udp', 'port': '8125', 'group': 'mirror-update'}
|
||||||
- {'protocol': 'udp', 'port': '8125', 'group': 'logstash'}
|
- {'protocol': 'udp', 'port': '8125', 'group': 'logstash'}
|
||||||
- {'protocol': 'udp', 'port': '8125', 'group': 'nodepool'}
|
- {'protocol': 'udp', 'port': '8125', 'group': 'nodepool'}
|
||||||
|
|
|
@ -7,7 +7,6 @@ iptables_extra_allowed_hosts:
|
||||||
protocol: udp
|
protocol: udp
|
||||||
|
|
||||||
iptables_extra_allowed_groups:
|
iptables_extra_allowed_groups:
|
||||||
- {'protocol': 'udp', 'port': '8125', 'group': 'firehose'}
|
|
||||||
- {'protocol': 'udp', 'port': '8125', 'group': 'mirror-update'}
|
- {'protocol': 'udp', 'port': '8125', 'group': 'mirror-update'}
|
||||||
- {'protocol': 'udp', 'port': '8125', 'group': 'logstash'}
|
- {'protocol': 'udp', 'port': '8125', 'group': 'logstash'}
|
||||||
- {'protocol': 'udp', 'port': '8125', 'group': 'nodepool'}
|
- {'protocol': 'udp', 'port': '8125', 'group': 'nodepool'}
|
||||||
|
|
|
@ -55,7 +55,6 @@ groups:
|
||||||
elasticsearch: elasticsearch[0-9]*.open*.org
|
elasticsearch: elasticsearch[0-9]*.open*.org
|
||||||
ethercalc: ethercalc*.open*.org
|
ethercalc: ethercalc*.open*.org
|
||||||
etherpad: etherpad[0-9]*.open*.org
|
etherpad: etherpad[0-9]*.open*.org
|
||||||
firehose: firehose[0-9]*.open*.org
|
|
||||||
gerrit:
|
gerrit:
|
||||||
- review[0-9]*.open*.org
|
- review[0-9]*.open*.org
|
||||||
gitea:
|
gitea:
|
||||||
|
@ -138,7 +137,6 @@ groups:
|
||||||
- eavesdrop[0-9]*.open*.org
|
- eavesdrop[0-9]*.open*.org
|
||||||
- elasticsearch[0-9]*.open*.org
|
- elasticsearch[0-9]*.open*.org
|
||||||
- ethercalc[0-9]*.open*.org
|
- ethercalc[0-9]*.open*.org
|
||||||
- firehose[0-9]*.open*.org
|
|
||||||
- health[0-9]*.openstack.org
|
- health[0-9]*.openstack.org
|
||||||
- lists*.katacontainers.io
|
- lists*.katacontainers.io
|
||||||
- lists*.open*.org
|
- lists*.open*.org
|
||||||
|
@ -166,7 +164,6 @@ groups:
|
||||||
- eavesdrop[0-9]*.open*.org
|
- eavesdrop[0-9]*.open*.org
|
||||||
- elasticsearch[0-9]*.open*.org
|
- elasticsearch[0-9]*.open*.org
|
||||||
- ethercalc[0-9]*.open*.org
|
- ethercalc[0-9]*.open*.org
|
||||||
- firehose[0-9]*.open*.org
|
|
||||||
- health[0-9]*.openstack.org
|
- health[0-9]*.openstack.org
|
||||||
- lists*.katacontainers.io
|
- lists*.katacontainers.io
|
||||||
- lists*.open*.org
|
- lists*.open*.org
|
||||||
|
|
|
@ -175,24 +175,6 @@ node /^elasticsearch\d+\.open.*\.org$/ {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
# Node-OS: xenial
|
|
||||||
node /^firehose\d+\.open.*\.org$/ {
|
|
||||||
class { 'openstack_project::server': }
|
|
||||||
class { 'openstack_project::firehose':
|
|
||||||
gerrit_ssh_host_key => hiera('gerrit_ssh_rsa_pubkey_contents'),
|
|
||||||
gerrit_public_key => hiera('germqtt_gerrit_ssh_public_key'),
|
|
||||||
gerrit_private_key => hiera('germqtt_gerrit_ssh_private_key'),
|
|
||||||
mqtt_password => hiera('mqtt_service_user_password'),
|
|
||||||
ca_file => hiera('mosquitto_tls_ca_file'),
|
|
||||||
cert_file => hiera('mosquitto_tls_server_cert_file'),
|
|
||||||
key_file => hiera('mosquitto_tls_server_key_file'),
|
|
||||||
imap_hostname => hiera('lpmqtt_imap_server'),
|
|
||||||
imap_username => hiera('lpmqtt_imap_username'),
|
|
||||||
imap_password => hiera('lpmqtt_imap_password'),
|
|
||||||
statsd_host => 'graphite.opendev.org',
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
# A machine to run Storyboard
|
# A machine to run Storyboard
|
||||||
# Node-OS: xenial
|
# Node-OS: xenial
|
||||||
node /^storyboard\d+\.opendev\.org$/ {
|
node /^storyboard\d+\.opendev\.org$/ {
|
||||||
|
|
|
@ -9,7 +9,6 @@ docs.starlingx.io 443
|
||||||
ethercalc.openstack.org 443
|
ethercalc.openstack.org 443
|
||||||
etherpad.openstack.org 443
|
etherpad.openstack.org 443
|
||||||
files.openstack.org 443
|
files.openstack.org 443
|
||||||
firehose.openstack.org 8883
|
|
||||||
git.airshipit.org 443
|
git.airshipit.org 443
|
||||||
git.openstack.org 443
|
git.openstack.org 443
|
||||||
git.starlingx.io 443
|
git.starlingx.io 443
|
||||||
|
|
|
@ -1,87 +0,0 @@
|
||||||
# Copyright 2016 Hewlett-Packard Development Company, L.P.
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
||||||
# not use this file except in compliance with the License. You may obtain
|
|
||||||
# a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
||||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
||||||
# License for the specific language governing permissions and limitations
|
|
||||||
# under the License.
|
|
||||||
#
|
|
||||||
# firehose glue class.
|
|
||||||
#
|
|
||||||
class openstack_project::firehose (
|
|
||||||
$gerrit_username = 'germqtt',
|
|
||||||
$gerrit_public_key,
|
|
||||||
$gerrit_private_key,
|
|
||||||
$gerrit_ssh_host_key,
|
|
||||||
$imap_username,
|
|
||||||
$imap_hostname,
|
|
||||||
$imap_password,
|
|
||||||
$mqtt_hostname = 'firehose.openstack.org',
|
|
||||||
$mqtt_password,
|
|
||||||
$mqtt_username = 'infra',
|
|
||||||
$statsd_host,
|
|
||||||
$ca_file,
|
|
||||||
$cert_file,
|
|
||||||
$key_file,
|
|
||||||
) {
|
|
||||||
include mosquitto
|
|
||||||
class {'mosquitto::server':
|
|
||||||
infra_service_username => $mqtt_username,
|
|
||||||
infra_service_password => $mqtt_password,
|
|
||||||
enable_tls => true,
|
|
||||||
enable_tls_websocket => true,
|
|
||||||
ca_file => $ca_file,
|
|
||||||
cert_file => $cert_file,
|
|
||||||
key_file => $key_file,
|
|
||||||
websocket_tls_port => 443,
|
|
||||||
}
|
|
||||||
|
|
||||||
include germqtt
|
|
||||||
class {'germqtt::server':
|
|
||||||
gerrit_username => $gerrit_username,
|
|
||||||
gerrit_public_key => $gerrit_public_key,
|
|
||||||
gerrit_private_key => $gerrit_private_key,
|
|
||||||
gerrit_ssh_host_key => $gerrit_ssh_host_key,
|
|
||||||
mqtt_username => $mqtt_username,
|
|
||||||
mqtt_password => $mqtt_password,
|
|
||||||
}
|
|
||||||
|
|
||||||
package {'cyrus-imapd':
|
|
||||||
ensure => latest,
|
|
||||||
}
|
|
||||||
|
|
||||||
package {'sasl2-bin':
|
|
||||||
ensure => latest,
|
|
||||||
}
|
|
||||||
|
|
||||||
package {'cyrus-admin':
|
|
||||||
ensure => latest,
|
|
||||||
}
|
|
||||||
|
|
||||||
service {'cyrus-imapd':
|
|
||||||
ensure => running,
|
|
||||||
}
|
|
||||||
|
|
||||||
include lpmqtt
|
|
||||||
class {'lpmqtt::server':
|
|
||||||
mqtt_username => $mqtt_username,
|
|
||||||
mqtt_password => $mqtt_password,
|
|
||||||
imap_hostname => $imap_hostname,
|
|
||||||
imap_username => $imap_username,
|
|
||||||
imap_password => $imap_password,
|
|
||||||
imap_use_ssl => false,
|
|
||||||
imap_delete_old => true,
|
|
||||||
}
|
|
||||||
|
|
||||||
include mqtt_statsd
|
|
||||||
class {'mqtt_statsd::server':
|
|
||||||
mqtt_hostname => $mqtt_hostname,
|
|
||||||
statsd_hostname => $statsd_host,
|
|
||||||
}
|
|
||||||
}
|
|
|
@ -2,7 +2,6 @@ letsencrypt_certcheck_additional_domains:
|
||||||
- ask.openstack.org 443
|
- ask.openstack.org 443
|
||||||
- ethercalc.openstack.org 443
|
- ethercalc.openstack.org 443
|
||||||
- etherpad.openstack.org 443
|
- etherpad.openstack.org 443
|
||||||
- firehose.openstack.org 8883
|
|
||||||
- git.openstack.org 443
|
- git.openstack.org 443
|
||||||
- openstackid-dev.openstack.org 443
|
- openstackid-dev.openstack.org 443
|
||||||
- openstackid.org 443
|
- openstackid.org 443
|
||||||
|
|
|
@ -13,11 +13,6 @@ results:
|
||||||
- afs-client
|
- afs-client
|
||||||
- kerberos-client
|
- kerberos-client
|
||||||
|
|
||||||
firehose01.openstack.org:
|
|
||||||
- firehose
|
|
||||||
- puppet
|
|
||||||
- puppet4
|
|
||||||
|
|
||||||
graphite02.opendev.org:
|
graphite02.opendev.org:
|
||||||
- graphite
|
- graphite
|
||||||
- letsencrypt
|
- letsencrypt
|
||||||
|
|
Loading…
Reference in New Issue