Add documentation to each collector.

.gitignore

@@ -52,3 +52,6 @@ ChangeLog
 *~
 .*.swp
 .*sw?
+
+doc/source/modules.rst
+doc/source/os_performance_tools.*

README.rst

@@ -2,11 +2,8 @@
 os-performance-tools
 ===============================
 
-Python scripts for use in OpenStack's QA process
+Python scripts for use in OpenStack's Performance Measurement
 
-Please feel here a long description which must be at least 3 lines wrapped on
-80 cols, so that distribution package maintainers can use it in their packages.
-Note that this is a hard requirement.
 
 * Free software: Apache license
 * Documentation: http://docs.openstack.org/developer/os-performance-tools
@@ -16,4 +13,6 @@ Note that this is a hard requirement.
 Features
 --------
 
-* TODO
+* Collect counters from various backends and save as JSON and/or Subunit attachments
+* Record delta with previous JSON counters
+* Push counters into statsd

doc/source/usage.rst

@@ -2,6 +2,22 @@
 Usage
 ========
 
-To use os-performance-tools in a project::
+os-collect-counters
+-------------------
 
-    import os_performance_tools
+This is a command line program which will try to collect data from
+various backends as rapidly as possible and then save a representation
+of these as JSON. It can optionall wrap this JSON in subunit, which is
+useful for appending to a subunit stream from previous tests.
+
+os_performance_tools.counters2statsd
+------------------------------------
+
+This module is useful in a subunit processing stream for saving the
+counters into statsd. It should be added as a target before processing
+the stream, and then upon reaching the end, it will send the counters into
+statsd. Currently this is limited to an attachment named 'counters.json'
+but other schemes may be implemented in the future.
+
+Please see counters_ for information on the backends and counters that
+are supported.

os_performance_tools/collectors/mysql.py

@@ -12,6 +12,12 @@
 #
 # This file was forked from dstat's mysql5_innodb plugin but retains none of
 # that original code other than a list of well known MySQL variable names.
+''' MySQL is accessed via the configuration options found at ~/.my.cnf. This is
+parsed not by libmysqlclient, which may or may not be present, but by
+configparser. As such, some options that are usually usable from that file may
+be ignored by this module. Everything from the "client" section will be passed
+through to pymysql's connect method.
+'''
 
 import logging
 import os
@@ -40,6 +46,13 @@ COLLECT_COUNTERS = (
     'Queries',
     'Slow_queries',
 )
+'''These counters' meaning are all documented in the `MySQL manual`_. They are
+intended to show a picture of how much has been asked of MySQL, and how busy
+MySQL was while executing commands. Each one will be recorded unaltered by name
+in the resulting counters mapping.
+
+.. _`MySQL manual`: http://dev.mysql.com/doc/refman/5.6/en/server-status-variables.html
+'''
 
 
 def _get_config():

os_performance_tools/collectors/queues.py

@@ -9,6 +9,20 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+'''RabbitMQ is the primary message queue backend for OpenStack. It
+is accessed via the status plugin, which must be enabled for this
+collector to work. The environment variables OS_QA_RABBITMQ_API,
+OS_QA_RABBITMQ_USER, OS_QA_RABBITMQ_PASS, are used to configure how the
+status API is contacted by the collector.
+
+The counters reported are entirely 'published' methods, meaning this is
+just a measure of how many messages were pushed into RabbitMQ.
+
+Because of the way oslo.messaging uses RabbitMQ, we won't know what exact
+application reply queues are attached to. So all of those messages end up
+in the 'reply' counter. Fanouts also have a random string added to them,
+so we strip that off.
+'''
 
 import base64
 import json

tox.ini

@@ -21,7 +21,8 @@ commands = {posargs}
 commands = python setup.py test --coverage --testr-args='{posargs}'
 
 [testenv:docs]
-commands = python setup.py build_sphinx
+commands = sphinx-apidoc -o doc/source os_performance_tools
+    python setup.py build_sphinx
 
 [testenv:debug]
 commands = oslo_debug_helper {posargs}