OpenStack Messaging (Zaqar)
Go to file
Flavio Percoco 3f12d76118 Add ZAQAR_TEST_EVERYTHING to enable all tests
We've a way to enable some tests that are environment dependant but
there's no good way, other than setting all those variables, to enable
all tests in a specific environment.

This patch adds a new variable to do the above. It also DRYes a bit the
check of env variables that have been set.

Change-Id: Ia75925b4e22e27dab4d67caf67db80c191717caf
2014-09-06 15:08:39 +02:00
doc Merge "First patch and first review docs" 2014-09-03 07:50:28 +00:00
etc Implements queue, message and claim controllers for Redis 2014-08-30 15:45:27 -05:00
tests Switch Redis driver modules over to oslo.utils.timeutils 2014-09-03 13:59:23 -05:00
tools/config Fix config generator in Zaqar 2014-08-22 15:33:32 +02:00
zaqar Add ZAQAR_TEST_EVERYTHING to enable all tests 2014-09-06 15:08:39 +02:00
.coveragerc Rename Marconi to Zaqar 2014-08-04 10:36:50 +02:00
.gitignore adds docs directory with dev guide 2014-03-18 14:48:54 -04:00
.gitreview Fix .gitreview due to the repo rename/move 2014-08-16 21:30:22 +04:00
.testr.conf Use testr instead of nosetest 2014-01-17 15:43:49 +01:00
AUTHORS.rst refactor: Rename AUTHORS so that it doesn't keep getting overwritten 2013-03-19 16:33:43 -04:00
babel.cfg Prepare marconi for localization 2014-06-04 22:31:55 +02:00
bench-requirements.txt Add observer role to benchmark tool 2014-08-29 10:30:51 -05:00
doc-test.conf Rename Marconi to Zaqar 2014-08-04 10:36:50 +02:00
HACKING.rst Rename Marconi to Zaqar 2014-08-04 10:36:50 +02:00
LICENSE Include full license text 2014-03-21 10:16:28 +01:00
MANIFEST.in Initial import. 2012-11-01 09:52:20 +01:00
openstack-common.conf Rename Marconi to Zaqar 2014-08-04 10:36:50 +02:00
README.rst Add observer role to benchmark tool 2014-08-29 10:30:51 -05:00
requirements-py3.txt Updated from global requirements 2014-08-26 20:55:33 +00:00
requirements.txt Updated from global requirements 2014-08-26 20:55:33 +00:00
setup.cfg Adds a manual garbage collector script 2014-09-02 21:52:05 -03:00
setup.py Updated from global requirements 2014-04-30 02:40:38 +00:00
test-requirements-py3.txt Updated from global requirements 2014-08-24 17:15:48 +00:00
test-requirements.txt Updated from global requirements 2014-08-24 17:15:48 +00:00
tox.ini Work toward Python 3.4 support and testing 2014-09-03 20:06:17 +00:00

Zaqar

Message queuing service for OpenStack. To find more information read our wiki.

Running a local Zaqar server with MongoDB

Note: These instructions are for running a local instance of Zaqar and not all of these steps are required. It is assumed you have MongoDB installed and running.

  1. From your home folder create the ~/.zaqar folder and clone the repo:

    $ cd
    $ mkdir .zaqar
    $ git clone https://github.com/openstack/zaqar.git
  2. Copy the Zaqar config files to the directory ~/.zaqar:

    $ cp zaqar/etc/zaqar.conf.sample ~/.zaqar/zaqar.conf
    $ cp zaqar/etc/logging.conf.sample ~/.zaqar/logging.conf
  3. Find [drivers] section in ~/.zaqar/zaqar.conf and specify to use mongodb storage:

    storage = mongodb

    Then find the [drivers:storage:mongodb] section and modify the URI to point to your local mongod instance:

    uri = mongodb://$MONGODB_HOST:$MONGODB_PORT

    By default, you will have:

    uri = mongodb://127.0.0.1:27017
  4. For logging, find the [DEFAULT] section in ~/.zaqar/zaqar.conf and modify as desired:

    log_file = server.log
  5. Change directories back to your local copy of the repo:

    $ cd zaqar
  6. Run the following so you can see the results of any changes you make to the code without having to reinstall the package each time:

    $ pip install -e .
  7. Start the Zaqar server with logging level set to INFO so you can see the port on which the server is listening:

    $ zaqar-server -v
  8. Test out that Zaqar is working by creating a queue:

    $ curl -i -X PUT http://127.0.0.1:8888/v1/queues/samplequeue -H
    "Content-type: application/json"

You should get an HTTP 201 along with some headers that will look similar to this:

HTTP/1.0 201 Created
Date: Fri, 25 Oct 2013 15:34:37 GMT
Server: WSGIServer/0.1 Python/2.7.3
Content-Length: 0
Location: /v1/queues/samplequeue

Running tests

First install additional requirements:

$ pip install tox

And then run tests:

$ tox -e py27

You can read more about running functional tests in separate TESTS_README.

Running the benchmarking tool

First install and run zaqar-server (see above).

Then install additional requirements:

$ pip install -r bench-requirements.txt

Copy the configuration file to ~/.zaqar:

$ cp etc/zaqar-benchmark.conf.sample ~/.zaqar/zaqar-benchmark.conf

In the configuration file specify where zaqar-server can be found:

server_url = http://localhost:8888

The benchmarking tool needs a set of messages to work with. Specify the path to the file with messages in the configuration file. Alternatively, put it in the directory with the configuration file and name it zaqar-benchmark- messages.json. As a starting point, you can use the sample file from the etc directory:

$ cp etc/zaqar-benchmark-messages.json ~/.zaqar/

If the file is not found or no file is specified, a single hard-coded message is used for all requests.

Run the benchmarking tool using the following command:

$ zaqar-bench

By default, the command will run a performance test for 5 seconds, using one producer process with 10 greenlet workers, and one observer process with 5 workers. The consumer role is disabled by default.

You can override these defaults in the config file or on the command line using a variety of options. For example, the following command runs a performance test for 30 seconds using 4 producer processes with 20 workers each, plus 4 consumer processes with 20 workers each. Note that the observer role is also disabled in this example by setting its number of workers to zero:

$ zaqar-bench -pp 4 -pw 10 -cw 4 -cw 20 -ow 0 -t 30

By default, the results are in JSON. For more human-readable output add the --verbose flag. Verbose output looks similar to the following:

$ zaqar-bench --verbose

Starting producer (pp=1 , pw=10)...

Starting observer (op=1 , ow=5)...

Producer
========
duration_sec: 5.1
ms_per_req: 2.9
reqs_per_sec: 344.5
successful_reqs: 1742.0
total_reqs: 1742.0

Observer
========
duration_sec: 5.0
ms_per_req: 2.9
reqs_per_sec: 339.3
successful_reqs: 1706.0
total_reqs: 1706.0