Merge branch 'master' of github.com:shakhat/performa

README.rst

@@ -1,19 +1,35 @@
-===============================
+========
-performa
+Performa
-===============================
+========
 
-Performance testing toolkit
+What Performa is?
+-----------------
 
-Please feel here a long description which must be at least 3 lines wrapped on
+Performa is distributed scenario runner, results processor and report generator.
-80 cols, so that distribution package maintainers can use it in their packages.
+It is the strong mixture of powerful technologies:
-Note that this is a hard requirement.
+ * Ansible_ for easy running modularized code in distributed manner
+ * MongoDB_ for storing and transforming data
+ * Pygal_ for creating light-weight charts
+ * Jinja2_ for templating the world
 
-* Free software: Apache license
-* Documentation: http://docs.openstack.org/developer/performa
-* Source: http://git.openstack.org/cgit/openstack/performa
-* Bugs: http://bugs.launchpad.net/performa
 
-Features
+Example
---------
+-------
 
-* TODO
+::
+
+    performa --mongo-url 127.0.0.1 --mongo-db performa --scenario db/sysbench \
+        --hosts "{target: [192.168.20.20]}" --remote-user developer --debug \
+        --book doc/source/test_results/db/sysbench
+
+This example runs Performa tool with scenario 'sb/sysbench'. It uses MongoDB
+located at localhost and database named 'performa'. The scenario is executed
+against remote host 192.168.20.20 which can be accessed with user 'developer'.
+The report is stored into 'doc/source/test_results/db/sysbench' folder.
+
+.. references:
+
+.. _Ansible: http://docs.ansible.com/
+.. _MongoDB: https://docs.mongodb.org/manual/
+.. _Pygal: http://www.pygal.org/
+.. _Jinja2: http://jinja.pocoo.org/

config-generator.conf

@@ -0,0 +1,5 @@
+[DEFAULT]
+output_file = etc/performa.conf
+wrap_width = 79
+namespace = performa.engine.config
+namespace = oslo_log

doc/source/index.rst

@@ -1,25 +1,31 @@
-.. performa documentation master file, created by
+Performa
-   sphinx-quickstart on Tue Jul  9 22:26:36 2013.
+========
-   You can adapt this file completely to your liking, but it should at least
+
-   contain the root `toctree` directive.
+What Performa is?
+-----------------
+
+Performa is distributed scenario runner, results processor and report generator.
+It is the strong mixture of powerful technologies:
+ * Ansible_ for easy running modularized code in distributed manner
+ * MongoDB_ for storing and transforming data
+ * Pygal_ for creating light-weight charts
+ * Jinja2_ for templating the world
 
-Welcome to performa's documentation!
-====================================
 
 Contents:
 
 .. toctree::
    :maxdepth: 2
 
-   readme
    installation
    usage
+   tools
    contributing
 
-Indices and tables
-==================
 
-* :ref:`genindex`
+.. references:
-* :ref:`modindex`
-* :ref:`search`
 
+.. _Ansible: http://docs.ansible.com/
+.. _MongoDB: https://docs.mongodb.org/manual/
+.. _Pygal: http://www.pygal.org/
+.. _Jinja2: http://jinja.pocoo.org/

doc/source/installation.rst

@@ -10,3 +10,4 @@ Or, if you have virtualenvwrapper installed::
 
     $ mkvirtualenv performa
     $ pip install performa
+

doc/source/tools.rst

@@ -0,0 +1,18 @@
+===================
+CLI Tools Reference
+===================
+
+performa
+--------
+
+Runs specified scenario, stores data and generates report.
+
+.. literalinclude:: tools/performa.txt
+
+
+performa-report
+---------------
+
+Generates report based on previously collected data.
+
+.. literalinclude:: tools/shaker-report.txt

doc/source/tools/performa-report.txt

@@ -0,0 +1,80 @@
+usage: performa-report [-h] [--book BOOK] [--config-dir DIR]
+                       [--config-file PATH] [--debug] [--hosts HOSTS]
+                       [--log-config-append PATH]
+                       [--log-date-format DATE_FORMAT] [--log-dir LOG_DIR]
+                       [--log-file PATH] [--mongo-db MONGO_DB]
+                       [--mongo-url MONGO_URL] [--nodebug] [--nouse-syslog]
+                       [--noverbose] [--nowatch-log-file]
+                       [--remote-user REMOTE_USER] [--scenario SCENARIO]
+                       [--syslog-log-facility SYSLOG_LOG_FACILITY] [--tag TAG]
+                       [--use-syslog] [--verbose] [--version]
+                       [--watch-log-file]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --book BOOK           Generate report in ReST format and store it into the
+                        specified folder, defaults to env[PERFORMA_BOOK].
+  --config-dir DIR      Path to a config directory to pull *.conf files from.
+                        This file set is sorted, so as to provide a
+                        predictable parse order if individual options are
+                        over-ridden. The set is parsed after the file(s)
+                        specified via previous --config-file, arguments hence
+                        over-ridden options in the directory take precedence.
+  --config-file PATH    Path to a config file to use. Multiple config files
+                        can be specified, with values in later files taking
+                        precedence. Defaults to None.
+  --debug, -d           If set to true, the logging level will be set to DEBUG
+                        instead of the default INFO level.
+  --hosts HOSTS         Hosts inventory definition in YAML format, Can be
+                        specified via env[PERFORMA_HOSTS].
+  --log-config-append PATH, --log_config PATH
+                        The name of a logging configuration file. This file is
+                        appended to any existing logging configuration files.
+                        For details about logging configuration files, see the
+                        Python logging module documentation. Note that when
+                        logging configuration files are used then all logging
+                        configuration is set in the configuration file and
+                        other logging configuration options are ignored (for
+                        example, logging_context_format_string).
+  --log-date-format DATE_FORMAT
+                        Defines the format string for %(asctime)s in log
+                        records. Default: None . This option is ignored if
+                        log_config_append is set.
+  --log-dir LOG_DIR, --logdir LOG_DIR
+                        (Optional) The base directory used for relative
+                        log_file paths. This option is ignored if
+                        log_config_append is set.
+  --log-file PATH, --logfile PATH
+                        (Optional) Name of log file to send logging output to.
+                        If no default is set, logging will go to stderr as
+                        defined by use_stderr. This option is ignored if
+                        log_config_append is set.
+  --mongo-db MONGO_DB   Mongo DB, defaults to env[PERFORMA_MONGO_DB].
+  --mongo-url MONGO_URL
+                        Mongo URL, defaults to env[PERFORMA_MONGO_URL].
+  --nodebug             The inverse of --debug
+  --nouse-syslog        The inverse of --use-syslog
+  --noverbose           The inverse of --verbose
+  --nowatch-log-file    The inverse of --watch-log-file
+  --remote-user REMOTE_USER
+                        User for connecting to remote systems, defaults to
+                        env[PERFORMA_REMOTE_USER].
+  --scenario SCENARIO   Scenario to play. Can be a file name or one of
+                        aliases: "db/sysbench", "mq/omsimulator". Defaults to
+                        env[PERFORMA_SCENARIO].
+  --syslog-log-facility SYSLOG_LOG_FACILITY
+                        Syslog facility to receive log lines. This option is
+                        ignored if log_config_append is set.
+  --tag TAG             Tag the execution, defaults to env[PERFORMA_TAG].
+  --use-syslog          Use syslog for logging. Existing syslog format is
+                        DEPRECATED and will be changed later to honor RFC5424.
+                        This option is ignored if log_config_append is set.
+  --verbose, -v         If set to false, the logging level will be set to
+                        WARNING instead of the default INFO level.
+  --version             show program's version number and exit
+  --watch-log-file      Uses logging handler designed to watch file system.
+                        When log file is moved or removed this handler will
+                        open a new log file with specified path
+                        instantaneously. It makes sense only if log_file
+                        option is specified and Linux platform is used. This
+                        option is ignored if log_config_append is set.

doc/source/tools/performa.txt

@@ -0,0 +1,78 @@
+usage: performa [-h] [--book BOOK] [--config-dir DIR] [--config-file PATH]
+                [--debug] [--hosts HOSTS] [--log-config-append PATH]
+                [--log-date-format DATE_FORMAT] [--log-dir LOG_DIR]
+                [--log-file PATH] [--mongo-db MONGO_DB]
+                [--mongo-url MONGO_URL] [--nodebug] [--nouse-syslog]
+                [--noverbose] [--nowatch-log-file] [--remote-user REMOTE_USER]
+                [--scenario SCENARIO]
+                [--syslog-log-facility SYSLOG_LOG_FACILITY] [--tag TAG]
+                [--use-syslog] [--verbose] [--version] [--watch-log-file]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --book BOOK           Generate report in ReST format and store it into the
+                        specified folder, defaults to env[PERFORMA_BOOK].
+  --config-dir DIR      Path to a config directory to pull *.conf files from.
+                        This file set is sorted, so as to provide a
+                        predictable parse order if individual options are
+                        over-ridden. The set is parsed after the file(s)
+                        specified via previous --config-file, arguments hence
+                        over-ridden options in the directory take precedence.
+  --config-file PATH    Path to a config file to use. Multiple config files
+                        can be specified, with values in later files taking
+                        precedence. Defaults to None.
+  --debug, -d           If set to true, the logging level will be set to DEBUG
+                        instead of the default INFO level.
+  --hosts HOSTS         Hosts inventory definition in YAML format, Can be
+                        specified via env[PERFORMA_HOSTS].
+  --log-config-append PATH, --log_config PATH
+                        The name of a logging configuration file. This file is
+                        appended to any existing logging configuration files.
+                        For details about logging configuration files, see the
+                        Python logging module documentation. Note that when
+                        logging configuration files are used then all logging
+                        configuration is set in the configuration file and
+                        other logging configuration options are ignored (for
+                        example, logging_context_format_string).
+  --log-date-format DATE_FORMAT
+                        Defines the format string for %(asctime)s in log
+                        records. Default: None . This option is ignored if
+                        log_config_append is set.
+  --log-dir LOG_DIR, --logdir LOG_DIR
+                        (Optional) The base directory used for relative
+                        log_file paths. This option is ignored if
+                        log_config_append is set.
+  --log-file PATH, --logfile PATH
+                        (Optional) Name of log file to send logging output to.
+                        If no default is set, logging will go to stderr as
+                        defined by use_stderr. This option is ignored if
+                        log_config_append is set.
+  --mongo-db MONGO_DB   Mongo DB, defaults to env[PERFORMA_MONGO_DB].
+  --mongo-url MONGO_URL
+                        Mongo URL, defaults to env[PERFORMA_MONGO_URL].
+  --nodebug             The inverse of --debug
+  --nouse-syslog        The inverse of --use-syslog
+  --noverbose           The inverse of --verbose
+  --nowatch-log-file    The inverse of --watch-log-file
+  --remote-user REMOTE_USER
+                        User for connecting to remote systems, defaults to
+                        env[PERFORMA_REMOTE_USER].
+  --scenario SCENARIO   Scenario to play. Can be a file name or one of
+                        aliases: "db/sysbench", "mq/omsimulator". Defaults to
+                        env[PERFORMA_SCENARIO].
+  --syslog-log-facility SYSLOG_LOG_FACILITY
+                        Syslog facility to receive log lines. This option is
+                        ignored if log_config_append is set.
+  --tag TAG             Tag the execution, defaults to env[PERFORMA_TAG].
+  --use-syslog          Use syslog for logging. Existing syslog format is
+                        DEPRECATED and will be changed later to honor RFC5424.
+                        This option is ignored if log_config_append is set.
+  --verbose, -v         If set to false, the logging level will be set to
+                        WARNING instead of the default INFO level.
+  --version             show program's version number and exit
+  --watch-log-file      Uses logging handler designed to watch file system.
+                        When log file is moved or removed this handler will
+                        open a new log file with specified path
+                        instantaneously. It makes sense only if log_file
+                        option is specified and Linux platform is used. This
+                        option is ignored if log_config_append is set.

doc/source/usage.rst

@@ -1,7 +1,14 @@
-========
+=====
 Usage
-========
+=====
 
-To use performa in a project::
+**Example**::
 
-    import performa
+    performa --mongo-url 127.0.0.1 --mongo-db performa --scenario db/sysbench \
+        --hosts "{target: [192.168.20.20]}" --remote-user developer --debug \
+        --book doc/source/test_results/db/sysbench
+
+This example runs Performa tool with scenario 'sb/sysbench'. It uses MongoDB
+located at localhost and database named 'performa'. The scenario is executed
+against remote host 192.168.20.20 which can be accessed with user 'developer'.
+The report is stored into 'doc/source/test_results/db/sysbench' folder.

etc/performa.conf

@@ -0,0 +1,125 @@
+[DEFAULT]
+
+#
+# From oslo_log
+#
+
+# If set to true, the logging level will be set to DEBUG instead of the default
+# INFO level. (boolean value)
+#debug = false
+
+# If set to false, the logging level will be set to WARNING instead of the
+# default INFO level. (boolean value)
+# This option is deprecated for removal.
+# Its value may be silently ignored in the future.
+#verbose = true
+
+# The name of a logging configuration file. This file is appended to any
+# existing logging configuration files. For details about logging configuration
+# files, see the Python logging module documentation. Note that when logging
+# configuration files are used then all logging configuration is set in the
+# configuration file and other logging configuration options are ignored (for
+# example, logging_context_format_string). (string value)
+# Deprecated group/name - [DEFAULT]/log_config
+#log_config_append = <None>
+
+# Defines the format string for %%(asctime)s in log records. Default:
+# %(default)s . This option is ignored if log_config_append is set. (string
+# value)
+#log_date_format = %Y-%m-%d %H:%M:%S
+
+# (Optional) Name of log file to send logging output to. If no default is set,
+# logging will go to stderr as defined by use_stderr. This option is ignored if
+# log_config_append is set. (string value)
+# Deprecated group/name - [DEFAULT]/logfile
+#log_file = <None>
+
+# (Optional) The base directory used for relative log_file  paths. This option
+# is ignored if log_config_append is set. (string value)
+# Deprecated group/name - [DEFAULT]/logdir
+#log_dir = <None>
+
+# Uses logging handler designed to watch file system. When log file is moved or
+# removed this handler will open a new log file with specified path
+# instantaneously. It makes sense only if log_file option is specified and
+# Linux platform is used. This option is ignored if log_config_append is set.
+# (boolean value)
+#watch_log_file = false
+
+# Use syslog for logging. Existing syslog format is DEPRECATED and will be
+# changed later to honor RFC5424. This option is ignored if log_config_append
+# is set. (boolean value)
+#use_syslog = false
+
+# Syslog facility to receive log lines. This option is ignored if
+# log_config_append is set. (string value)
+#syslog_log_facility = LOG_USER
+
+# Log output to standard error. This option is ignored if log_config_append is
+# set. (boolean value)
+#use_stderr = true
+
+# Format string to use for log messages with context. (string value)
+#logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s
+
+# Format string to use for log messages when context is undefined. (string
+# value)
+#logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s
+
+# Additional data to append to log message when logging level for the message
+# is DEBUG. (string value)
+#logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d
+
+# Prefix each line of exception output with this format. (string value)
+#logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s
+
+# Defines the format string for %(user_identity)s that is used in
+# logging_context_format_string. (string value)
+#logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s
+
+# List of package logging levels in logger=LEVEL pairs. This option is ignored
+# if log_config_append is set. (list value)
+#default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=INFO,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,requests.packages.urllib3.util.retry=WARN,urllib3.util.retry=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN,taskflow=WARN,keystoneauth=WARN,oslo.cache=INFO,dogpile.core.dogpile=INFO
+
+# Enables or disables publication of error events. (boolean value)
+#publish_errors = false
+
+# The format for an instance that is passed with the log message. (string
+# value)
+#instance_format = "[instance: %(uuid)s] "
+
+# The format for an instance UUID that is passed with the log message. (string
+# value)
+#instance_uuid_format = "[instance: %(uuid)s] "
+
+# Enables or disables fatal status of deprecations. (boolean value)
+#fatal_deprecations = false
+
+#
+# From performa.engine.config
+#
+
+# Scenario to play. Can be a file name or one of aliases: "db/sysbench",
+# "mq/omsimulator". Defaults to env[PERFORMA_SCENARIO]. (string value)
+#scenario = <None>
+
+# Mongo URL, defaults to env[PERFORMA_MONGO_URL]. (string value)
+#mongo_url = <None>
+
+# Mongo DB, defaults to env[PERFORMA_MONGO_DB]. (string value)
+#mongo_db = <None>
+
+# User for connecting to remote systems, defaults to env[PERFORMA_REMOTE_USER].
+# (string value)
+#remote_user = <None>
+
+# Hosts inventory definition in YAML format, Can be specified via
+# env[PERFORMA_HOSTS]. (string value)
+#hosts = <None>
+
+# Generate report in ReST format and store it into the specified folder,
+# defaults to env[PERFORMA_BOOK].  (string value)
+#book = <None>
+
+# Tag the execution, defaults to env[PERFORMA_TAG]. (string value)
+#tag = <None>

tools/cli_auto_doc.py

@@ -0,0 +1,62 @@
+# Copyright (c) 2015 Mirantis Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import sys
+
+try:
+    import ConfigParser as configparser
+except ImportError:
+    import configparser
+
+
+def split_multiline(value):
+    value = [element for element in
+             (line.strip() for line in value.split('\n'))
+             if element]
+    return value
+
+
+def get_entry_points(config):
+    if 'entry_points' not in config:
+        return {}
+    return dict((option, split_multiline(value))
+                for option, value in config['entry_points'].items())
+
+
+def make(cfg, dest):
+    parser = configparser.RawConfigParser()
+    parser.read(cfg)
+    config = {}
+    for section in parser.sections():
+        config[section] = dict(parser.items(section))
+    entry_points = get_entry_points(config)
+
+    console_scripts = entry_points.get('console_scripts')
+    if console_scripts:
+        for item in console_scripts:
+            tool = item.split('=')[0].strip()
+            print('Running %s' % tool)
+            os.system('%(tool)s --help > %(dest)s/%(tool)s.txt' %
+                      dict(tool=tool, dest=dest))
+
+
+if len(sys.argv) < 2:
+    print('Usage: cli_auto_doc <dest folder>')
+    sys.exit(1)
+
+
+print('Generating docs from help to console tools')
+make(cfg='setup.cfg', dest=sys.argv[1])

tools/svg2png.sh

@@ -0,0 +1,3 @@
+#!/bin/bash -xe
+
+find doc/source/ -name *svg | sed "s/\.svg$//" | xargs -I% cairosvg "%.svg" -o "%.png"

tox.ini

@@ -19,6 +19,20 @@ commands = {posargs}
 [testenv:pep8]
 commands = flake8
 
+[testenv:genconfig]
+commands =
+    oslo-config-generator --config-file=config-generator.conf
+    python tools/cli_auto_doc.py doc/source/tools
+
+[testenv:svg2png]
+deps =
+    cairosvg
+    lxml
+    tinycss
+    cssselect
+whitelist_externals = bash
+commands = bash tools/svg2png.sh
+
 [testenv:docs]
 commands = python setup.py build_sphinx