From ef024eb098ff4fa9956d822591fe3f6f7a63fca3 Mon Sep 17 00:00:00 2001
From: Matthew Treinish <mtreinish@kortar.org>
Date: Sat, 14 Mar 2015 00:37:11 -0400
Subject: [PATCH] Add docs for subunit2sql
This commit adds docs to ci.openstack.org for subunit2sql in the
logstash page.
Change-Id: Id1dc7c373af78fd20e0448086bd91080565b957a
---
doc/source/logstash.rst | 63 ++++++++++++++++++++++++++++++++++-------
1 file changed, 53 insertions(+), 10 deletions(-)
diff --git a/doc/source/logstash.rst b/doc/source/logstash.rst
index 6d2a6a7c4b..1032ed32f8 100644
--- a/doc/source/logstash.rst
+++ b/doc/source/logstash.rst
@@ -79,16 +79,16 @@ bottleneck very quickly. This looks something like:
jenkins
|
|
- gearman-client
- / | \
- / | \
- gearman gearman gearman
- worker1 worker2 worker3
- | | |
- logstash logstash logstash
- indexer1 indexer2 indexer3
- \ | /
- \ | /
+ gearman-client ---------------
+ / | \ |
+ / | \ |
+ gearman gearman gearman subunit gearman
+ worker1 worker2 worker3 worker01
+ | | | |
+ logstash logstash logstash |
+ indexer1 indexer2 indexer3 |
+ \ | / subunit2sql
+ \ | / DB
elasticsearch
cluster
|
@@ -119,6 +119,20 @@ can be found at
* https://git.openstack.org/cgit/openstack-infra/puppet-log_processor/tree/files/log-gearman-client.py
* https://git.openstack.org/cgit/openstack-infra/puppet-log_processor/tree/files/log-gearman-worker.py
+Subunit Gearman Worker
+----------------------
+
+Using the same mechanism as the Log pushers there is an additional class of
+gearman worker that takes the subunit output from test runs and stores them in
+a subunit2SQL database. Right now this is only done with the subunit output
+from gate queue tempest runs.
+
+If you are interested in technical details the source of this script can be
+found at:
+
+* https://git.openstack.org/cgit/openstack-infra/puppet-subunit2sql/tree/files/subunit-gearman-worker.py
+
+
Logstash
--------
@@ -204,6 +218,35 @@ be reached at http://logstash.openstack.org. See
:ref:`query-logstash` for more info on using Kibana to perform
queries.
+subunit2SQL
+-----------
+subunit2SQL is a python project for taking subunit v2 streams and storing them
+in a SQL database. More information on the subunit protocol can be found here:
+https://github.com/testing-cabal/subunit/blob/master/README
+
+subunit2sql provides a database schema, several utilities for interacting with
+the database, and a python library to build tooling on top of the database.
+More information about using subunit2sql can be found at:
+http://docs.openstack.org/developer/subunit2sql/
+
+Our instance of the subunit2SQL database is running on a MySQL database and has
+been configured to be remotely accessible to allow for public querying. The
+public query access is provided with the following credentials::
+
+ username: query
+ password: query
+ hostname: logstash.openstack.org
+ database name: subunit2sql
+ database port: 3306
+
+simpleproxy
+-----------
+Simpleproxy is a simple tcp proxy which allows forwarding tcp connections from
+one host to another. We use it to forward mysql traffic from a publicly
+accessible host to the trove instance running the subunit2sql MySQL DB. This
+allows for public access to the data on the database through the host
+logstash.openstack.org.
+
.. _query-logstash:
Querying Logstash