x/performa
Code Issues Proposed changes

Add documentation bits

Browse Source
This commit is contained in:
Ilya Shakhat 2016-03-01 13:17:11 +03:00
parent 3d1f4a6a00
commit 754d1a4f8f
12 changed files with 445 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
README.rst
View File

@ -1,19 +1,35 @@
===============================
performa
===============================
========
Performa
========
Performance testing toolkit
What Performa is?
-----------------
Please feel here a long description which must be at least 3 lines wrapped on
80 cols, so that distribution package maintainers can use it in their packages.
Note that this is a hard requirement.
Performa is distributed scenario runner, results processor and report generator.
It is the strong mixture of powerful technologies:
* Ansible_ for easy running modularized code in distributed manner
* MongoDB_ for storing and transforming data
* Pygal_ for creating light-weight charts
* Jinja2_ for templating the world
* Free software: Apache license
* Documentation: http://docs.openstack.org/developer/performa
* Source: http://git.openstack.org/cgit/openstack/performa
* Bugs: http://bugs.launchpad.net/performa
Features
--------
Example
-------
* TODO
::
performa --mongo-url 127.0.0.1 --mongo-db performa --scenario db/sysbench \
--hosts "{target: [192.168.20.20]}" --remote-user developer --debug \
--book doc/source/test_results/db/sysbench
This example runs Performa tool with scenario 'sb/sysbench'. It uses MongoDB
located at localhost and database named 'performa'. The scenario is executed
against remote host 192.168.20.20 which can be accessed with user 'developer'.
The report is stored into 'doc/source/test_results/db/sysbench' folder.
.. references:
.. _Ansible: http://docs.ansible.com/
.. _MongoDB: https://docs.mongodb.org/manual/
.. _Pygal: http://www.pygal.org/
.. _Jinja2: http://jinja.pocoo.org/

5
config-generator.conf Normal file
View File

@ -0,0 +1,5 @@
[DEFAULT]
output_file = etc/performa.conf
wrap_width = 79
namespace = performa.engine.config
namespace = oslo_log

30
doc/source/index.rst
View File

@ -1,25 +1,31 @@
.. performa documentation master file, created by
sphinx-quickstart on Tue Jul 9 22:26:36 2013.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Performa
========
What Performa is?
-----------------
Performa is distributed scenario runner, results processor and report generator.
It is the strong mixture of powerful technologies:
* Ansible_ for easy running modularized code in distributed manner
* MongoDB_ for storing and transforming data
* Pygal_ for creating light-weight charts
* Jinja2_ for templating the world
Welcome to performa's documentation!
====================================
Contents:
.. toctree::
:maxdepth: 2
readme
installation
usage
tools
contributing
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
.. references:
.. _Ansible: http://docs.ansible.com/
.. _MongoDB: https://docs.mongodb.org/manual/
.. _Pygal: http://www.pygal.org/
.. _Jinja2: http://jinja.pocoo.org/

1
doc/source/installation.rst
View File

@ -10,3 +10,4 @@ Or, if you have virtualenvwrapper installed::
$ mkvirtualenv performa
$ pip install performa

18
doc/source/tools.rst Normal file
View File

@ -0,0 +1,18 @@
===================
CLI Tools Reference
===================
performa
--------
Runs specified scenario, stores data and generates report.
.. literalinclude:: tools/performa.txt
performa-report
---------------
Generates report based on previously collected data.
.. literalinclude:: tools/shaker-report.txt

80
doc/source/tools/performa-report.txt Normal file
View File

@ -0,0 +1,80 @@
usage: performa-report [-h] [--book BOOK] [--config-dir DIR]
[--config-file PATH] [--debug] [--hosts HOSTS]
[--log-config-append PATH]
[--log-date-format DATE_FORMAT] [--log-dir LOG_DIR]
[--log-file PATH] [--mongo-db MONGO_DB]
[--mongo-url MONGO_URL] [--nodebug] [--nouse-syslog]
[--noverbose] [--nowatch-log-file]
[--remote-user REMOTE_USER] [--scenario SCENARIO]
[--syslog-log-facility SYSLOG_LOG_FACILITY] [--tag TAG]
[--use-syslog] [--verbose] [--version]
[--watch-log-file]
optional arguments:
-h, --help show this help message and exit
--book BOOK Generate report in ReST format and store it into the
specified folder, defaults to env[PERFORMA_BOOK].
--config-dir DIR Path to a config directory to pull *.conf files from.
This file set is sorted, so as to provide a
predictable parse order if individual options are
over-ridden. The set is parsed after the file(s)
specified via previous --config-file, arguments hence
over-ridden options in the directory take precedence.
--config-file PATH Path to a config file to use. Multiple config files
can be specified, with values in later files taking
precedence. Defaults to None.
--debug, -d If set to true, the logging level will be set to DEBUG
instead of the default INFO level.
--hosts HOSTS Hosts inventory definition in YAML format, Can be
specified via env[PERFORMA_HOSTS].
--log-config-append PATH, --log_config PATH
The name of a logging configuration file. This file is
appended to any existing logging configuration files.
For details about logging configuration files, see the
Python logging module documentation. Note that when
logging configuration files are used then all logging
configuration is set in the configuration file and
other logging configuration options are ignored (for
example, logging_context_format_string).
--log-date-format DATE_FORMAT
Defines the format string for %(asctime)s in log
records. Default: None . This option is ignored if
log_config_append is set.
--log-dir LOG_DIR, --logdir LOG_DIR
(Optional) The base directory used for relative
log_file paths. This option is ignored if
log_config_append is set.
--log-file PATH, --logfile PATH
(Optional) Name of log file to send logging output to.
If no default is set, logging will go to stderr as
defined by use_stderr. This option is ignored if
log_config_append is set.
--mongo-db MONGO_DB Mongo DB, defaults to env[PERFORMA_MONGO_DB].
--mongo-url MONGO_URL
Mongo URL, defaults to env[PERFORMA_MONGO_URL].
--nodebug The inverse of --debug
--nouse-syslog The inverse of --use-syslog
--noverbose The inverse of --verbose
--nowatch-log-file The inverse of --watch-log-file
--remote-user REMOTE_USER
User for connecting to remote systems, defaults to
env[PERFORMA_REMOTE_USER].
--scenario SCENARIO Scenario to play. Can be a file name or one of
aliases: "db/sysbench", "mq/omsimulator". Defaults to
env[PERFORMA_SCENARIO].
--syslog-log-facility SYSLOG_LOG_FACILITY
Syslog facility to receive log lines. This option is
ignored if log_config_append is set.
--tag TAG Tag the execution, defaults to env[PERFORMA_TAG].
--use-syslog Use syslog for logging. Existing syslog format is
DEPRECATED and will be changed later to honor RFC5424.
This option is ignored if log_config_append is set.
--verbose, -v If set to false, the logging level will be set to
WARNING instead of the default INFO level.
--version show program's version number and exit
--watch-log-file Uses logging handler designed to watch file system.
When log file is moved or removed this handler will
open a new log file with specified path
instantaneously. It makes sense only if log_file
option is specified and Linux platform is used. This
option is ignored if log_config_append is set.

78
doc/source/tools/performa.txt Normal file
View File

@ -0,0 +1,78 @@
usage: performa [-h] [--book BOOK] [--config-dir DIR] [--config-file PATH]
[--debug] [--hosts HOSTS] [--log-config-append PATH]
[--log-date-format DATE_FORMAT] [--log-dir LOG_DIR]
[--log-file PATH] [--mongo-db MONGO_DB]
[--mongo-url MONGO_URL] [--nodebug] [--nouse-syslog]
[--noverbose] [--nowatch-log-file] [--remote-user REMOTE_USER]
[--scenario SCENARIO]
[--syslog-log-facility SYSLOG_LOG_FACILITY] [--tag TAG]
[--use-syslog] [--verbose] [--version] [--watch-log-file]
optional arguments:
-h, --help show this help message and exit
--book BOOK Generate report in ReST format and store it into the
specified folder, defaults to env[PERFORMA_BOOK].
--config-dir DIR Path to a config directory to pull *.conf files from.
This file set is sorted, so as to provide a
predictable parse order if individual options are
over-ridden. The set is parsed after the file(s)
specified via previous --config-file, arguments hence
over-ridden options in the directory take precedence.
--config-file PATH Path to a config file to use. Multiple config files
can be specified, with values in later files taking
precedence. Defaults to None.
--debug, -d If set to true, the logging level will be set to DEBUG
instead of the default INFO level.
--hosts HOSTS Hosts inventory definition in YAML format, Can be
specified via env[PERFORMA_HOSTS].
--log-config-append PATH, --log_config PATH
The name of a logging configuration file. This file is
appended to any existing logging configuration files.
For details about logging configuration files, see the
Python logging module documentation. Note that when
logging configuration files are used then all logging
configuration is set in the configuration file and
other logging configuration options are ignored (for
example, logging_context_format_string).
--log-date-format DATE_FORMAT
Defines the format string for %(asctime)s in log
records. Default: None . This option is ignored if
log_config_append is set.
--log-dir LOG_DIR, --logdir LOG_DIR
(Optional) The base directory used for relative
log_file paths. This option is ignored if
log_config_append is set.
--log-file PATH, --logfile PATH
(Optional) Name of log file to send logging output to.
If no default is set, logging will go to stderr as
defined by use_stderr. This option is ignored if
log_config_append is set.
--mongo-db MONGO_DB Mongo DB, defaults to env[PERFORMA_MONGO_DB].
--mongo-url MONGO_URL
Mongo URL, defaults to env[PERFORMA_MONGO_URL].
--nodebug The inverse of --debug
--nouse-syslog The inverse of --use-syslog
--noverbose The inverse of --verbose
--nowatch-log-file The inverse of --watch-log-file
--remote-user REMOTE_USER
User for connecting to remote systems, defaults to
env[PERFORMA_REMOTE_USER].
--scenario SCENARIO Scenario to play. Can be a file name or one of
aliases: "db/sysbench", "mq/omsimulator". Defaults to
env[PERFORMA_SCENARIO].
--syslog-log-facility SYSLOG_LOG_FACILITY
Syslog facility to receive log lines. This option is
ignored if log_config_append is set.
--tag TAG Tag the execution, defaults to env[PERFORMA_TAG].
--use-syslog Use syslog for logging. Existing syslog format is
DEPRECATED and will be changed later to honor RFC5424.
This option is ignored if log_config_append is set.
--verbose, -v If set to false, the logging level will be set to
WARNING instead of the default INFO level.
--version show program's version number and exit
--watch-log-file Uses logging handler designed to watch file system.
When log file is moved or removed this handler will
open a new log file with specified path
instantaneously. It makes sense only if log_file
option is specified and Linux platform is used. This
option is ignored if log_config_append is set.

15
doc/source/usage.rst
View File

@ -1,7 +1,14 @@
========
=====
Usage
========
=====
To use performa in a project::
**Example**::
import performa
performa --mongo-url 127.0.0.1 --mongo-db performa --scenario db/sysbench \
--hosts "{target: [192.168.20.20]}" --remote-user developer --debug \
--book doc/source/test_results/db/sysbench
This example runs Performa tool with scenario 'sb/sysbench'. It uses MongoDB
located at localhost and database named 'performa'. The scenario is executed
against remote host 192.168.20.20 which can be accessed with user 'developer'.
The report is stored into 'doc/source/test_results/db/sysbench' folder.

125
etc/performa.conf Normal file
View File

@ -0,0 +1,125 @@
[DEFAULT]
#
# From oslo_log
#
# If set to true, the logging level will be set to DEBUG instead of the default
# INFO level. (boolean value)
#debug = false
# If set to false, the logging level will be set to WARNING instead of the
# default INFO level. (boolean value)
# This option is deprecated for removal.
# Its value may be silently ignored in the future.
#verbose = true
# The name of a logging configuration file. This file is appended to any
# existing logging configuration files. For details about logging configuration
# files, see the Python logging module documentation. Note that when logging
# configuration files are used then all logging configuration is set in the
# configuration file and other logging configuration options are ignored (for
# example, logging_context_format_string). (string value)
# Deprecated group/name - [DEFAULT]/log_config
#log_config_append = <None>
# Defines the format string for %%(asctime)s in log records. Default:
# %(default)s . This option is ignored if log_config_append is set. (string
# value)
#log_date_format = %Y-%m-%d %H:%M:%S
# (Optional) Name of log file to send logging output to. If no default is set,
# logging will go to stderr as defined by use_stderr. This option is ignored if
# log_config_append is set. (string value)
# Deprecated group/name - [DEFAULT]/logfile
#log_file = <None>
# (Optional) The base directory used for relative log_file paths. This option
# is ignored if log_config_append is set. (string value)
# Deprecated group/name - [DEFAULT]/logdir
#log_dir = <None>
# Uses logging handler designed to watch file system. When log file is moved or
# removed this handler will open a new log file with specified path
# instantaneously. It makes sense only if log_file option is specified and
# Linux platform is used. This option is ignored if log_config_append is set.
# (boolean value)
#watch_log_file = false
# Use syslog for logging. Existing syslog format is DEPRECATED and will be
# changed later to honor RFC5424. This option is ignored if log_config_append
# is set. (boolean value)
#use_syslog = false
# Syslog facility to receive log lines. This option is ignored if
# log_config_append is set. (string value)
#syslog_log_facility = LOG_USER
# Log output to standard error. This option is ignored if log_config_append is
# set. (boolean value)
#use_stderr = true
# Format string to use for log messages with context. (string value)
#logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s
# Format string to use for log messages when context is undefined. (string
# value)
#logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s
# Additional data to append to log message when logging level for the message
# is DEBUG. (string value)
#logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d
# Prefix each line of exception output with this format. (string value)
#logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s
# Defines the format string for %(user_identity)s that is used in
# logging_context_format_string. (string value)
#logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s
# List of package logging levels in logger=LEVEL pairs. This option is ignored
# if log_config_append is set. (list value)
#default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=INFO,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,requests.packages.urllib3.util.retry=WARN,urllib3.util.retry=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN,taskflow=WARN,keystoneauth=WARN,oslo.cache=INFO,dogpile.core.dogpile=INFO
# Enables or disables publication of error events. (boolean value)
#publish_errors = false
# The format for an instance that is passed with the log message. (string
# value)
#instance_format = "[instance: %(uuid)s] "
# The format for an instance UUID that is passed with the log message. (string
# value)
#instance_uuid_format = "[instance: %(uuid)s] "
# Enables or disables fatal status of deprecations. (boolean value)
#fatal_deprecations = false
#
# From performa.engine.config
#
# Scenario to play. Can be a file name or one of aliases: "db/sysbench",
# "mq/omsimulator". Defaults to env[PERFORMA_SCENARIO]. (string value)
#scenario = <None>
# Mongo URL, defaults to env[PERFORMA_MONGO_URL]. (string value)
#mongo_url = <None>
# Mongo DB, defaults to env[PERFORMA_MONGO_DB]. (string value)
#mongo_db = <None>
# User for connecting to remote systems, defaults to env[PERFORMA_REMOTE_USER].
# (string value)
#remote_user = <None>
# Hosts inventory definition in YAML format, Can be specified via
# env[PERFORMA_HOSTS]. (string value)
#hosts = <None>
# Generate report in ReST format and store it into the specified folder,
# defaults to env[PERFORMA_BOOK]. (string value)
#book = <None>
# Tag the execution, defaults to env[PERFORMA_TAG]. (string value)
#tag = <None>

62
tools/cli_auto_doc.py Normal file
View File

@ -0,0 +1,62 @@
# Copyright (c) 2015 Mirantis Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
try:
import ConfigParser as configparser
except ImportError:
import configparser
def split_multiline(value):
value = [element for element in
(line.strip() for line in value.split('\n'))
if element]
return value
def get_entry_points(config):
if 'entry_points' not in config:
return {}
return dict((option, split_multiline(value))
for option, value in config['entry_points'].items())
def make(cfg, dest):
parser = configparser.RawConfigParser()
parser.read(cfg)
config = {}
for section in parser.sections():
config[section] = dict(parser.items(section))
entry_points = get_entry_points(config)
console_scripts = entry_points.get('console_scripts')
if console_scripts:
for item in console_scripts:
tool = item.split('=')[0].strip()
print('Running %s' % tool)
os.system('%(tool)s --help > %(dest)s/%(tool)s.txt' %
dict(tool=tool, dest=dest))
if len(sys.argv) < 2:
print('Usage: cli_auto_doc <dest folder>')
sys.exit(1)
print('Generating docs from help to console tools')
make(cfg='setup.cfg', dest=sys.argv[1])

3
tools/svg2png.sh Executable file
View File

@ -0,0 +1,3 @@
#!/bin/bash -xe
find doc/source/ -name *svg | sed "s/\.svg$//" | xargs -I% cairosvg "%.svg" -o "%.png"

14
tox.ini
View File

@ -19,6 +19,20 @@ commands = {posargs}
[testenv:pep8]
commands = flake8
[testenv:genconfig]
commands =
oslo-config-generator --config-file=config-generator.conf
python tools/cli_auto_doc.py doc/source/tools
[testenv:svg2png]
deps =
cairosvg
lxml
tinycss
cssselect
whitelist_externals = bash
commands = bash tools/svg2png.sh
[testenv:docs]
commands = python setup.py build_sphinx