performa/shaker
Code Issues Proposed changes

Pack Shaker into container

Browse Source 
Now Shaker can be run in Docker container! The container
does the whole work: it creates image, runs the scenario
and cleans everything up.

Change-Id: I9cbcb7e79d08f9526cf1ee8f3ccafab34d3935c2
This commit is contained in:
Ilya Shakhat 2016-08-03 19:11:43 +03:00
parent 62bd121a8d
commit 1f9c4d0a21
10 changed files with 424 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
.dockerignore Normal file
View File

@ -0,0 +1,19 @@
*~
*.pyc
*.local
AUTHORS
ChangeLog
MANIFEST
dist/
.venv/
build/*
build-stamp
cover/*
doc/build/
doc/source/api/
*.egg-info
*.egg
.autogenerated
.coverage
.testrepository/
.tox/

13
Dockerfile Normal file
View File

@ -0,0 +1,13 @@
FROM python:3.5
MAINTAINER Ilya Shakhat <shakhat@gmail.com>
ADD . /opt/shaker/
RUN pip install -r /opt/shaker/requirements.txt
WORKDIR /opt/shaker/
RUN python setup.py install
VOLUME /artifacts
STOPSIGNAL SIGTERM
ENTRYPOINT ["/usr/local/bin/shaker-all-in-one", "--artifacts-dir", "/artifacts"]

56
README.rst
View File

@ -10,36 +10,62 @@ Shaker is able to deploy OpenStack instances and networks in different
topologies. Shaker scenario specifies the deployment and list of tests topologies. Shaker scenario specifies the deployment and list of tests
to execute. Additionally tests may be tuned dynamically in command-line. to execute. Additionally tests may be tuned dynamically in command-line.
Features: Features
--------
* User-defined topology via Heat templates * User-defined topology via Heat templates
* Simultaneously test execution on multiple instances * Simultaneously test execution on multiple instances
* Pluggable tools
* Interactive report with stats and charts * Interactive report with stats and charts
* Built-in SLA verification * Built-in SLA verification
Requirements: Deployment Requirements
-----------------------
* Shaker server routable from OpenStack cloud * Shaker server routable from OpenStack cloud
* Admin-user access to OpenStack API * Admin-user access to OpenStack API is preferable
Setup: Run in Python Environment
-------------------------
1. ``pip install pyshaker`` - installs the tool and all its python dependencies .. code-block:: bash
2. ``shaker-image-builder`` - builds shaker image and stores it in Glance
$ pip install pyshaker
$ . openrc
$ shaker-image-builder
$ shaker --server-endpoint <host:port> --scenario <scenario> --report <report.html>``
where:
* ``host`` and ``port`` - host and port of machine where Shaker is deployed
* ``scenario`` - the scenario to execute, e.g. `openstack/perf_l2` (
`catalog <http://pyshaker.readthedocs.io/en/latest/catalog.html>`_)
* ``<report.html>`` - file to store the final report
Full list of parameters is available in `documentation <http://pyshaker.readthedocs.io/en/latest/tools.html#shaker>`_.
Run: Shaker in Container
-------------------
``shaker --server-endpoint <host:port> --scenario <scenario.yaml> --report <report.html>`` Shaker is available as container a container at Docker Hub at
`shakhat/shaker <https://hub.docker.com/r/shakhat/shaker/>`_
where: .. code-block:: bash
* ``<host:port>`` - address of machine where Shaker is deployed and any free port
* ``<scenario.yaml>`` - the scenario to execute; L2, L3 east-west and L3 north-south already included $ docker run -p <port>:<port> -v <artifacts-dir>:/artifacts shakhat/shaker --scenario <scenario> --server-endpoint <host:port>
* ``<report.html>`` - file to store the report --os-auth-url <os-auth-url> --os-username <os-username> --os-password <os-password> --os-project-name <os-project-name>
where:
* ``host`` and ``port`` - host and port on machine where Shaker is deployed
* ``artifacts-dir`` - where to store report and raw result
* ``scenario`` - the scenario to execute, e.g. `openstack/perf_l2` (
`catalog <http://pyshaker.readthedocs.io/en/latest/catalog.html>`_)
* ``os-XXX`` - OpenStack cloud credentials
Links: Links
-----
* PyPi - https://pypi.python.org/pypi/pyshaker/ * PyPi - https://pypi.python.org/pypi/pyshaker/
* Docs - http://pyshaker.readthedocs.org/ * Docker - https://hub.docker.com/r/shakhat/shaker/
* Docs - http://pyshaker.readthedocs.io/
* Bugtracker - https://launchpad.net/shaker/ * Bugtracker - https://launchpad.net/shaker/

208
doc/source/tools/shaker-all-in-one.txt Normal file
View File

@ -0,0 +1,208 @@
usage: shaker-all-in-one [-h] [--agent-join-timeout AGENT_JOIN_TIMEOUT]
[--agent-loss-timeout AGENT_LOSS_TIMEOUT]
[--artifacts-dir ARTIFACTS_DIR] [--book BOOK]
[--cleanup] [--cleanup-on-error] [--config-dir DIR]
[--config-file PATH] [--debug]
[--dns-nameservers DNS_NAMESERVERS]
[--external-net EXTERNAL_NET]
[--flavor-disk FLAVOR_DISK]
[--flavor-name FLAVOR_NAME] [--flavor-ram FLAVOR_RAM]
[--flavor-vcpus FLAVOR_VCPUS]
[--image-builder-template IMAGE_BUILDER_TEMPLATE]
[--image-name IMAGE_NAME] [--log-config-append PATH]
[--log-date-format DATE_FORMAT] [--log-dir LOG_DIR]
[--log-file PATH] [--matrix MATRIX]
[--no-report-on-error] [--nocleanup]
[--nocleanup-on-error] [--nodebug]
[--nono-report-on-error] [--noos-insecure]
[--nouse-syslog] [--noverbose] [--nowatch-log-file]
[--os-auth-url <auth-url>]
[--os-cacert <auth-cacert>] [--os-insecure]
[--os-password <auth-password>]
[--os-project-name <auth-project-name>]
[--os-region-name <auth-region-name>]
[--os-tenant-name <auth-tenant-name>]
[--os-username <auth-username>] [--output OUTPUT]
[--polling-interval POLLING_INTERVAL]
[--report REPORT] [--report-template REPORT_TEMPLATE]
[--scenario SCENARIO]
[--server-endpoint SERVER_ENDPOINT]
[--subunit SUBUNIT]
[--syslog-log-facility SYSLOG_LOG_FACILITY]
[--use-syslog] [--verbose] [--watch-log-file]
optional arguments:
-h, --help show this help message and exit
--agent-join-timeout AGENT_JOIN_TIMEOUT
Timeout to treat agent as join failed in seconds,
defaults to env[SHAKER_AGENT_JOIN_TIMEOUT] (time
between stack deployment and start of scenario
execution).
--agent-loss-timeout AGENT_LOSS_TIMEOUT
Timeout to treat agent as lost in seconds, defaults to
env[SHAKER_AGENT_LOSS_TIMEOUT]
--artifacts-dir ARTIFACTS_DIR
If specified, directs Shaker to store there all its
artifacts (output, report, subunit and book). Defaults
to env[SHAKER_ARTIFACTS_DIR].
--book BOOK Generate report in ReST format and store it into the
specified folder, defaults to env[SHAKER_BOOK].
--cleanup Cleanup the image and the flavor.
--cleanup-on-error Clean up the heat-stack upon any error occured during
scenario execution.
--config-dir DIR Path to a config directory to pull *.conf files from.
This file set is sorted, so as to provide a
predictable parse order if individual options are
over-ridden. The set is parsed after the file(s)
specified via previous --config-file, arguments hence
over-ridden options in the directory take precedence.
--config-file PATH Path to a config file to use. Multiple config files
can be specified, with values in later files taking
precedence. Defaults to None.
--debug, -d If set to true, the logging level will be set to DEBUG
instead of the default INFO level.
--dns-nameservers DNS_NAMESERVERS
Comma seperated list of IPs of the DNS nameservers for
the subnets. If no value is provided defaults to
Google Public DNS.
--external-net EXTERNAL_NET
Name or ID of external network, defaults to
env[SHAKER_EXTERNAL_NET]. If no value provided then
Shaker picks any of available external networks.
--flavor-disk FLAVOR_DISK
Shaker image disk size in GB, defaults to
env[SHAKER_FLAVOR_DISK]
--flavor-name FLAVOR_NAME
Name of image flavor. The default is created by
shaker-image-builder.
--flavor-ram FLAVOR_RAM
Shaker image RAM size in MB, defaults to
env[SHAKER_FLAVOR_RAM]
--flavor-vcpus FLAVOR_VCPUS
Number of cores to allocate for Shaker image, defaults
to env[SHAKER_FLAVOR_VCPUS]
--image-builder-template IMAGE_BUILDER_TEMPLATE
Heat template containing receipt of building the
image. Can be a file name or one of aliases: "centos",
"debian", "ubuntu". Defaults to "ubuntu".
--image-name IMAGE_NAME
Name of image to use. The default is created by
shaker-image-builder.
--log-config-append PATH, --log_config PATH
The name of a logging configuration file. This file is
appended to any existing logging configuration files.
For details about logging configuration files, see the
Python logging module documentation. Note that when
logging configuration files are used then all logging
configuration is set in the configuration file and
other logging configuration options are ignored (for
example, logging_context_format_string).
--log-date-format DATE_FORMAT
Defines the format string for %(asctime)s in log
records. Default: None . This option is ignored if
log_config_append is set.
--log-dir LOG_DIR, --logdir LOG_DIR
(Optional) The base directory used for relative
log_file paths. This option is ignored if
log_config_append is set.
--log-file PATH, --logfile PATH
(Optional) Name of log file to send logging output to.
If no default is set, logging will go to stderr as
defined by use_stderr. This option is ignored if
log_config_append is set.
--matrix MATRIX Set the matrix of parameters for the scenario. The
value is specified in YAML format. E.g. to override
the scenario duration one may provide: "{time: 10}",
or to override list of hosts: "{host:[ping.online.net,
iperf.eenet.ee]}". When several parameters are
overridden all combinations are tested
--no-report-on-error Do not generate report for failed scenarios
--nocleanup The inverse of --cleanup
--nocleanup-on-error The inverse of --cleanup-on-error
--nodebug The inverse of --debug
--nono-report-on-error
The inverse of --no-report-on-error
--noos-insecure The inverse of --os-insecure
--nouse-syslog The inverse of --use-syslog
--noverbose The inverse of --verbose
--nowatch-log-file The inverse of --watch-log-file
--os-auth-url <auth-url>
Authentication URL, defaults to env[OS_AUTH_URL].
--os-cacert <auth-cacert>
Location of CA Certificate, defaults to
env[OS_CACERT].
--os-insecure When using SSL in connections to the registry server,
do not require validation via a certifying authority,
defaults to env[OS_INSECURE].
--os-password <auth-password>
Authentication password, defaults to env[OS_PASSWORD].
--os-project-name <auth-project-name>
Another way to specify tenant name. This option is
mutually exclusive with --os-tenant-name. Defaults to
env[OS_PROJECT_NAME].
--os-region-name <auth-region-name>
Authentication region name, defaults to
env[OS_REGION_NAME].
--os-tenant-name <auth-tenant-name>
Authentication tenant name, defaults to
env[OS_TENANT_NAME].
--os-username <auth-username>
Authentication username, defaults to env[OS_USERNAME].
--output OUTPUT File for output in JSON format, defaults to
env[SHAKER_OUTPUT]. If it is empty, then output will
be saved to /tmp/shaker_<time_execution>.json
--polling-interval POLLING_INTERVAL
How frequently the agent polls server, in seconds
--report REPORT Report file name, defaults to env[SHAKER_REPORT].
--report-template REPORT_TEMPLATE
Template for report. Can be a file name or one of
aliases: "interactive", "json". Defaults to
"interactive".
--scenario SCENARIO Scenario to play. Can be a file name or one of
aliases: "misc/instance_metadata",
"misc/static_agent", "misc/static_agents_pair",
"openstack/cross_az/full_l2",
"openstack/cross_az/full_l3_east_west",
"openstack/cross_az/full_l3_north_south",
"openstack/cross_az/perf_l2",
"openstack/cross_az/perf_l3_east_west",
"openstack/cross_az/perf_l3_north_south",
"openstack/cross_az/udp_l2",
"openstack/cross_az/udp_l2_mss8950",
"openstack/cross_az/udp_l3_east_west",
"openstack/dense_l2", "openstack/dense_l3_east_west",
"openstack/dense_l3_north_south",
"openstack/external/dense_l3_north_south_no_fip",
"openstack/external/dense_l3_north_south_with_fip",
"openstack/external/full_l3_north_south_no_fip",
"openstack/external/full_l3_north_south_with_fip",
"openstack/external/perf_l3_north_south_no_fip",
"openstack/external/perf_l3_north_south_with_fip",
"openstack/full_l2", "openstack/full_l3_east_west",
"openstack/full_l3_north_south", "openstack/perf_l2",
"openstack/perf_l3_east_west",
"openstack/perf_l3_north_south",
"openstack/qos/perf_l2", "openstack/udp_l2",
"openstack/udp_l3_east_west",
"openstack/udp_l3_north_south", "spot/ping",
"spot/tcp", "spot/udp". Defaults to
env[SHAKER_SCENARIO].
--server-endpoint SERVER_ENDPOINT
Address for server connections (host:port), defaults
to env[SHAKER_SERVER_ENDPOINT].
--subunit SUBUNIT Subunit stream file name, defaults to
env[SHAKER_SUBUNIT].
--syslog-log-facility SYSLOG_LOG_FACILITY
Syslog facility to receive log lines. This option is
ignored if log_config_append is set.
--use-syslog Use syslog for logging. Existing syslog format is
DEPRECATED and will be changed later to honor RFC5424.
This option is ignored if log_config_append is set.
--verbose, -v If set to false, the logging level will be set to
WARNING instead of the default INFO level.
--watch-log-file Uses logging handler designed to watch file system.
When log file is moved or removed this handler will
open a new log file with specified path
instantaneously. It makes sense only if log_file
option is specified and Linux platform is used. This
option is ignored if log_config_append is set.

14
doc/source/tools/shaker-cleanup.txt
View File

@ -1,15 +1,15 @@
usage: shaker-cleanup [-h] [--cleanup-on-error] [--config-dir DIR] usage: shaker-cleanup [-h] [--cleanup] [--cleanup-on-error] [--config-dir DIR]
[--config-file PATH] [--debug] [--config-file PATH] [--debug]
[--dns-nameservers DNS_NAMESERVERS] [--dns-nameservers DNS_NAMESERVERS]
[--external-net EXTERNAL_NET] [--external-net EXTERNAL_NET]
[--flavor-name FLAVOR_NAME] [--image-name IMAGE_NAME] [--flavor-name FLAVOR_NAME] [--image-name IMAGE_NAME]
[--log-config-append PATH] [--log-config-append PATH]
[--log-date-format DATE_FORMAT] [--log-dir LOG_DIR] [--log-date-format DATE_FORMAT] [--log-dir LOG_DIR]
[--log-file PATH] [--nocleanup-on-error] [--nodebug] [--log-file PATH] [--nocleanup] [--nocleanup-on-error]
[--noos-insecure] [--nouse-syslog] [--noverbose] [--nodebug] [--noos-insecure] [--nouse-syslog]
[--nowatch-log-file] [--os-auth-url <auth-url>] [--noverbose] [--nowatch-log-file]
[--os-cacert <auth-cacert>] [--os-insecure] [--os-auth-url <auth-url>] [--os-cacert <auth-cacert>]
[--os-password <auth-password>] [--os-insecure] [--os-password <auth-password>]
[--os-project-name <auth-project-name>] [--os-project-name <auth-project-name>]
[--os-region-name <auth-region-name>] [--os-region-name <auth-region-name>]
[--os-tenant-name <auth-tenant-name>] [--os-tenant-name <auth-tenant-name>]
@ -19,6 +19,7 @@ usage: shaker-cleanup [-h] [--cleanup-on-error] [--config-dir DIR]
optional arguments: optional arguments:
-h, --help show this help message and exit -h, --help show this help message and exit
--cleanup Cleanup the image and the flavor.
--cleanup-on-error Clean up the heat-stack upon any error occured during --cleanup-on-error Clean up the heat-stack upon any error occured during
scenario execution. scenario execution.
--config-dir DIR Path to a config directory to pull *.conf files from. --config-dir DIR Path to a config directory to pull *.conf files from.
@ -68,6 +69,7 @@ optional arguments:
If no default is set, logging will go to stderr as If no default is set, logging will go to stderr as
defined by use_stderr. This option is ignored if defined by use_stderr. This option is ignored if
log_config_append is set. log_config_append is set.
--nocleanup The inverse of --cleanup
--nocleanup-on-error The inverse of --cleanup-on-error --nocleanup-on-error The inverse of --cleanup-on-error
--nodebug The inverse of --debug --nodebug The inverse of --debug
--noos-insecure The inverse of --os-insecure --noos-insecure The inverse of --os-insecure

3
etc/shaker.conf
View File

@ -237,3 +237,6 @@
# Shaker image disk size in GB, defaults to env[SHAKER_FLAVOR_DISK] (integer # Shaker image disk size in GB, defaults to env[SHAKER_FLAVOR_DISK] (integer
# value) # value)
#flavor_disk = 3 #flavor_disk = 3
# Cleanup the image and the flavor. (boolean value)
#cleanup = true

5
setup.cfg
View File

@ -27,9 +27,10 @@ console_scripts =
shaker = shaker.engine.server:main shaker = shaker.engine.server:main
shaker-report = shaker.engine.report:main shaker-report = shaker.engine.report:main
shaker-agent = shaker.agent.agent:main shaker-agent = shaker.agent.agent:main
shaker-image-builder = shaker.engine.image_builder:build_image shaker-image-builder = shaker.engine.image_builder:build_image_entry_point
shaker-cleanup = shaker.engine.image_builder:cleanup shaker-cleanup = shaker.engine.image_builder:cleanup_entry_point
shaker-spot = shaker.engine.spot:main shaker-spot = shaker.engine.spot:main
shaker-all-in-one = shaker.engine.all_in_one:main
oslo.config.opts = oslo.config.opts =
oslo_log = oslo_log._options:list_opts oslo_log = oslo_log._options:list_opts

93
shaker/engine/all_in_one.py Normal file
View File

@ -0,0 +1,93 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import errno
import os
import tempfile
from oslo_config import cfg
from oslo_log import log as logging
from shaker.engine import config
from shaker.engine import image_builder
from shaker.engine import server
from shaker.engine import utils
LOG = logging.getLogger(__name__)
def _make_filename(folder, prefix, ext=None):
tmp_report = prefix
if ext:
tmp_report = '%s.%s' % (tmp_report, ext)
return os.path.join(folder, tmp_report)
def _mkdir_p(path):
try:
os.makedirs(path)
except OSError as exc:
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
else:
raise
def _configure_log_file(log_file):
cfg.CONF.set_override('log_file', log_file)
logging.setup(cfg.CONF, 'shaker')
cfg.CONF.log_opt_values(LOG, logging.DEBUG)
def main():
utils.init_config_and_logging(
config.COMMON_OPTS + config.OPENSTACK_OPTS + config.SERVER_OPTS +
config.REPORT_OPTS + config.IMAGE_BUILDER_OPTS + config.CLEANUP_OPTS +
config.ALL_IN_ONE_OPTS
)
artifacts_dir = cfg.CONF.artifacts_dir
if artifacts_dir:
_mkdir_p(artifacts_dir)
else:
artifacts_dir = tempfile.mkdtemp(prefix='shaker')
# image-builder
_configure_log_file(_make_filename(artifacts_dir, 'image_builder', 'log'))
LOG.info('Building the image')
image_builder.build_image()
# core
scenario = cfg.CONF.scenario
prefix = utils.strict(scenario)
_configure_log_file(_make_filename(artifacts_dir, prefix, 'log'))
LOG.info('Executing scenario: %s', scenario)
cfg.CONF.set_override('output',
_make_filename(artifacts_dir, prefix, 'json'))
cfg.CONF.set_override('report',
_make_filename(artifacts_dir, prefix, 'html'))
cfg.CONF.set_override('subunit',
_make_filename(artifacts_dir, prefix, 'subunit'))
cfg.CONF.set_override('book', _make_filename(artifacts_dir, prefix))
server.act()
# cleanup
_configure_log_file(_make_filename(artifacts_dir, 'cleanup', 'log'))
LOG.info('Cleaning up')
image_builder.cleanup()
if __name__ == "__main__":
main()

16
shaker/engine/config.py
View File

@ -233,8 +233,22 @@ IMAGE_BUILDER_OPTS = [
'env[SHAKER_FLAVOR_DISK]'), 'env[SHAKER_FLAVOR_DISK]'),
] ]
CLEANUP_OPTS = [
cfg.BoolOpt('cleanup',
default=(utils.env('SHAKER_CLEANUP') or True),
help='Cleanup the image and the flavor.'),
]
# very specific, should not be listed in list_opts()
ALL_IN_ONE_OPTS = [
cfg.StrOpt('artifacts-dir', default=utils.env('SHAKER_ARTIFACTS_DIR'),
help='If specified, directs Shaker to store there all its '
'artifacts (output, report, subunit and book). '
'Defaults to env[SHAKER_ARTIFACTS_DIR].'),
]
def list_opts(): def list_opts():
all_opts = (COMMON_OPTS + OPENSTACK_OPTS + SERVER_OPTS + REPORT_OPTS + all_opts = (COMMON_OPTS + OPENSTACK_OPTS + SERVER_OPTS + REPORT_OPTS +
INPUT_OPTS + AGENT_OPTS + IMAGE_BUILDER_OPTS) INPUT_OPTS + AGENT_OPTS + IMAGE_BUILDER_OPTS + CLEANUP_OPTS)
yield (None, copy.deepcopy(all_opts)) yield (None, copy.deepcopy(all_opts))

27
shaker/engine/image_builder.py
View File

@ -30,9 +30,7 @@ from shaker.openstack.clients import openstack
LOG = logging.getLogger(__name__) LOG = logging.getLogger(__name__)
def init(cfg_options): def init():
utils.init_config_and_logging(cfg_options)
openstack_params = utils.pack_openstack_params(cfg.CONF) openstack_params = utils.pack_openstack_params(cfg.CONF)
try: try:
return openstack.OpenStackClient(openstack_params) return openstack.OpenStackClient(openstack_params)
@ -43,7 +41,7 @@ def init(cfg_options):
def build_image(): def build_image():
openstack_client = init(config.OPENSTACK_OPTS + config.IMAGE_BUILDER_OPTS) openstack_client = init()
flavor_name = cfg.CONF.flavor_name flavor_name = cfg.CONF.flavor_name
image_name = cfg.CONF.image_name image_name = cfg.CONF.image_name
dns_nameservers = cfg.CONF.dns_nameservers dns_nameservers = cfg.CONF.dns_nameservers
@ -119,10 +117,14 @@ def build_image():
def cleanup(): def cleanup():
openstack_client = init(config.OPENSTACK_OPTS) openstack_client = init()
flavor_name = cfg.CONF.flavor_name flavor_name = cfg.CONF.flavor_name
image_name = cfg.CONF.image_name image_name = cfg.CONF.image_name
if not cfg.CONF.cleanup:
LOG.info('Skip cleanup')
return
image = glance.get_image(openstack_client.glance, image_name) image = glance.get_image(openstack_client.glance, image_name)
if image: if image:
openstack_client.glance.images.delete(image.id) openstack_client.glance.images.delete(image.id)
@ -131,5 +133,18 @@ def cleanup():
if flavor: if flavor:
openstack_client.nova.flavors.delete(flavor.id) openstack_client.nova.flavors.delete(flavor.id)
if __name__ == "__main__":
def build_image_entry_point():
utils.init_config_and_logging(
config.OPENSTACK_OPTS + config.IMAGE_BUILDER_OPTS
)
build_image() build_image()
def cleanup_entry_point():
utils.init_config_and_logging(config.OPENSTACK_OPTS + config.CLEANUP_OPTS)
cleanup()
if __name__ == "__main__":
build_image_entry_point()