Merge "Restructure Rally docs"

doc/feature_request/README.rst

@@ -2,10 +2,8 @@
 Feature requests
 ================
 
-To request a new feature you should create a document similar to other feature
-requests. And contribute it to this directory using next instruction_.
+To request a new feature, you should create a document similar to other feature requests. And contribute it to this directory using the next instruction_.
 
-If you don't have time to contribute via gerrit,
-please contact Boris Pavlovic (boris@pavlovic.me)
+If you don't have time to contribute your feature request via gerrit, please contact Boris Pavlovic (boris@pavlovic.me)
 
-.. _instruction:  https://wiki.openstack.org/wiki/Rally/Develop#How_to_contribute
+.. _instruction:  http://rally.readthedocs.org/en/latest/contribute.html#how-to-contribute

doc/source/contribute.rst

@@ -0,0 +1,178 @@
+..
+      Copyright 2015 Mirantis Inc. All Rights Reserved.
+
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+
+
+.. _contribute:
+
+Contribute to Rally
+===================
+
+Where to begin
+--------------
+
+Please take a look `our Roadmap <https://docs.google.com/a/mirantis.com/spreadsheets/d/16DXpfbqvlzMFaqaXAcJsBzzpowb_XpymaK2aFY2gA2g/edit#gid=0>`_ to get information about our current work directions.
+
+In case you have questions or want to share your ideas, be sure to contact us at the ``#openstack-rally`` IRC channel on **irc.freenode.net**.
+
+If you are going to contribute to Rally, you will probably need to grasp a better understanding of several main design concepts used throughout our project (such as **benchmark scenarios**, **contexts** etc.). To do so, please read :ref:`this article <main_concepts>`.
+
+
+How to contribute
+-----------------
+
+1. You need a `Launchpad <https://launchpad.net/>`_ account and need to be joined to the `Openstack team <https://launchpad.net/openstack>`_. You can also join the `Rally team <https://launchpad.net/rally>`_ if you want to. Make sure Launchpad has your SSH key, Gerrit (the code review system) uses this.
+
+2. Sign the CLA as outlined in the `account setup <http://docs.openstack.org/infra/manual/developers.html#development-workflow>`_ section of the developer guide.
+
+3. Tell git your details:
+
+.. code-block:: none
+
+    git config --global user.name "Firstname Lastname"
+    git config --global user.email "your_email@youremail.com"
+
+4. Install git-review. This tool takes a lot of the pain out of remembering commands to push code up to Gerrit for review and to pull it back down to edit it. It is installed using:
+
+.. code-block:: none
+
+    pip install git-review
+
+Several Linux distributions (notably Fedora 16 and Ubuntu 12.04) are also starting to include git-review in their repositories so it can also be installed using the standard package manager.
+
+5. Grab the Rally repository:
+
+.. code-block:: none
+
+    git clone git@github.com:stackforge/rally.git
+
+6. Checkout a new branch to hack on:
+
+.. code-block:: none
+
+    git checkout -b TOPIC-BRANCH
+
+7. Start coding
+
+8. Run the test suite locally to make sure nothing broke, e.g. (this will run py26/py27/pep8 tests):
+
+.. code-block:: none
+
+    tox
+
+**(NOTE: you should have installed tox<=1.6.1 )**
+
+If you extend Rally with new functionality, make sure you have also provided unit and/or functional tests for it.
+
+9. Commit your work using:
+
+.. code-block:: none
+
+    git commit -a
+
+
+Make sure you have supplied your commit with a neat commit message, containing a link to the corresponding blueprint / bug, if appropriate.
+
+10. Push the commit up for code review using:
+
+.. code-block:: none
+
+    git review -R
+
+That is the awesome tool we installed earlier that does a lot of hard work for you.
+
+11. Watch your email or `review site <http://review.openstack.org/>`_, it will automatically send your code for a battery of tests on our `Jenkins setup <http://jenkins.openstack.org/>`_ and the core team for the project will review your code. If there are any changes that should be made they will let you know.
+
+12. When all is good the review site  will automatically merge your code.
+
+
+(This tutorial is based on: http://www.linuxjedi.co.uk/2012/03/real-way-to-start-hacking-on-openstack.html)
+
+Testing
+-------
+
+Please, don't hesitate to write tests ;)
+
+
+Unit tests
+^^^^^^^^^^
+
+*Files: /tests/unit/**
+
+The goal of unit tests is to ensure that internal parts of the code work properly.
+All internal methods should be fully covered by unit tests with a reasonable mocks usage.
+
+
+About Rally unit tests:
+
+- All `unit tests <http://en.wikipedia.org/wiki/Unit_testing>`_ are located inside /tests/unit/*
+- Tests are written on top of: *testtools*, *fixtures* and *mock* libs
+- `Tox <https://tox.readthedocs.org/en/latest/>`_ is used to run unit tests
+
+
+To run unit tests locally::
+
+  $ pip install tox
+  $ tox
+
+To run py26, py27 or pep8 only::
+
+  $ tox -e <name>
+
+  #NOTE: <name> is one of py26, py27 or pep8
+
+To get test coverage::
+
+  $ tox -e cover
+
+  #NOTE: Results will be in /cover/index.html
+
+To generate docs::
+
+  $ tox -e docs
+
+  #NOTE: Documentation will be in doc/source/_build/html/index.html
+
+Functional tests
+^^^^^^^^^^^^^^^^
+
+*Files: /tests/functional/**
+
+The goal of `functional tests <https://en.wikipedia.org/wiki/Functional_testing>`_ is to check that everything works well together.
+Fuctional tests use Rally API only and check responses without touching internal parts.
+
+To run functional tests locally::
+
+  $ source openrc
+  $ rally deployment create --fromenv --name testing
+  $ tox -e cli
+
+  #NOTE: openrc file with OpenStack admin credentials
+
+Rally CI scripts
+^^^^^^^^^^^^^^^^
+
+*Files: /tests/ci/**
+
+This directory contains scripts and files related to the Rally CI system.
+
+Rally Style Commandments
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+*Files: /tests/hacking/*
+
+This module contains Rally specific hacking rules for checking commandments.
+
+For more information about Style Commandments, read the `OpenStack Style Commandments manual <http://docs.openstack.org/developer/hacking/>`_.

doc/source/feature_requests.rst

@@ -1,6 +1,31 @@
-.. include:: feature_request/README.rst
+..
+      Copyright 2015 Mirantis Inc. All Rights Reserved.
+
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+.. _feature_requests:
+
+Request New Features
+====================
+
+To request a new feature, you should create a document similar to other feature requests and then contribute it to the **doc/feature_request** directory of the Rally repository (see the :ref:`How-to-contribute tutorial <contribute>`).
+
+If you don't have time to contribute your feature request via gerrit, please contact Boris Pavlovic (boris@pavlovic.me)
+
+Active feature requests:
 
 .. toctree::
    :glob:
+   :maxdepth: 1
 
    feature_request/*

doc/source/index.rst

@@ -1,5 +1,5 @@
 ..
-      Copyright 2014 Mirantis Inc. All Rights Reserved.
+      Copyright 2015 Mirantis Inc. All Rights Reserved.
 
       Licensed under the Apache License, Version 2.0 (the "License"); you may
       not use this file except in compliance with the License. You may obtain
@@ -14,45 +14,26 @@
       under the License.
 
 What is Rally?
-=================================
-
-If you are here, you are probably familiar with OpenStack and you also know that it's a really huge ecosystem of cooperative services. When something fails, performs slowly or doesn't scale, it's really hard to answer different questions on "what", "why" and "where" has happened. Another reason why you could be here is that you would like to build an OpenStack CI/CD system that will allow you to improve SLA, performance and stability of OpenStack continuously.
-
-The OpenStack QA team mostly works on CI/CD that ensures that new patches don't break some specific single node installation of OpenStack. On the other hand it's clear that such CI/CD is only an indication and does not cover all cases (e.g. if a cloud works well on a single node installation it doesn't mean that it will continue to do so on a 1k servers installation under high load as well). Rally aims to fix this and help us to answer the question "How does OpenStack work at scale?". To make it possible, we are going to automate and unify all steps that are required for benchmarking OpenStack at scale: multi-node OS deployment, verification, benchmarking & profiling.
+==============
 
+**OpenStack** is, undoubtedly, a really *huge* ecosystem of cooperative services. **Rally** is a **benchmarking tool** that answers the question: **"How does OpenStack work at scale?"**. To make this possible, Rally **automates** and **unifies** multi-node OpenStack deployment, cloud verification, benchmarking & profiling. Rally does it in a **generic** way, making it possible to check whether OpenStack is going to work well on, say, a 1k-servers installation under high load. Thus it can be used as a basic tool for an *OpenStack CI/CD system* that would continuously improve its SLA, performance and stability.
 
 .. image:: ./images/Rally-Actions.png
-   :width: 50%
+   :width: 100%
    :align: center
 
-* Deploy engine is not yet another deployer of OpenStack, but just a pluggable mechanism that allows to unify & simplify work with different deployers like: DevStack, Fuel, Anvil on hardware/VMs that you have.
-* Verification - (work in progress) uses tempest to verify the functionality of a deployed OpenStack cloud. In future Rally will support other OS verifiers.
-* Benchmark engine - allows to create parameterized load on the cloud based on a big repository of benchmarks.
 
-
-Deeper in Rally:
-----------------
+Contents
+--------
 .. toctree::
    :maxdepth: 2
 
    overview
-   concepts
-   deploy_engines
-   server_providers
-   verify
-   installation
-   usage
-   testing
-   feature_requests
+   install
+   tutorial
    user_stories
-
-Development information:
-------------------------
-
-.. toctree::
-   :maxdepth: 2
-
-   cmds
-   implementation
-   improve_rally
-   rally_gatejob
+   plugins
+   contribute
+   gates
+   feature_requests
+   project_info

doc/user_stories/keystone/authenticate.rst

@@ -1,12 +1,15 @@
+===================================================================================
+4x performance increase in Keysone inside Apache using the token creation benchmark
+===================================================================================
 
-Keystone token creation benchmark
-=================================
-Authenticate users with keystone to get tokens.
+*(Contributed by Neependra Khare, Red Hat)*
+
+Below we describe how we were able to get and verify a 4x better performance of Keysone inside Apache. To do that, we ran a Keystone token creation benchmark with Rally under different load (this benchmark scenario essentially just authenticates users with keystone to get tokens).
 
 Goal
 ----
-- To get data about performance of token creation under different load.
-- To ensure that keystone with increased  public_workers/admin_workers values
+- Get the data about performance of token creation under different load.
+- Ensure that keystone with increased public_workers/admin_workers values
   and under Apache works better than the default setup.
 
 Summary

doc/user_stories/nova/boot_server.rst

@@ -1,8 +1,10 @@
-=========================================================
-Testing how 20 node HA cloud performs on creating 400 VMs
-=========================================================
+==========================================================================================
+Finding a Keystone bug while benchmarking 20 node HA cloud performance at creating 400 VMs
+==========================================================================================
 
-Boot significant amount of servers on a cluster and ensure that we have reasonable performance and there are no errors.
+*(Contributed by Alexander Maretskiy, Mirantis)*
+
+Below we describe how we found a `bug in keystone <https://bugs.launchpad.net/keystone/+bug/1360446>`_ and achieved 2x average performance increase at booting Nova servers after fixing that bug. Our initial goal was to benchmark the booting of a significant amount of servers on a cluster (running on a custom build of `Mirantis OpenStack <https://software.mirantis.com/>`_ v5.1) and to ensure that this operation has reasonable performance and completes with no errors.
 
 Goal
 ----
@@ -37,7 +39,7 @@ Cluster
 This cluster was created via Fuel Dashboard interface.
 
 +----------------------+-----------------------------------------------------------------------------+
-| Deployment           | Custom build of `MirantisOpenStack <https://software.mirantis.com/>`_  v5.1 |
+| Deployment           | Custom build of `Mirantis OpenStack <https://software.mirantis.com/>`_  v5.1 |
 +----------------------+-----------------------------------------------------------------------------+
 | OpenStack release    | Icehouse                                                                    |
 +----------------------+-----------------------------------------------------------------------------+
@@ -65,7 +67,7 @@ https://review.openstack.org/#/c/96300/
 
 **Deployment**
 
-Rally was deployed for cluster using `ExistingCloud <https://github.com/stackforge/rally/blob/master/doc/samples/deployments/existing.json>`_ type of deployment.
+Rally was deployed for cluster using `ExistingCloud <https://github.com/stackforge/rally/blob/master/samples/deployments/existing.json>`_ type of deployment.
 
 **Server flavor** ::
 
@@ -165,7 +167,7 @@ That is how a `bug in keystone <https://bugs.launchpad.net/keystone/+bug/1360446
 
 **Second run, with bugfix:**
 
-After a patch was applied (using RPC instead of neutron client in metadata agent), we got **100% success and 2x improved avg perfomance**:
+After a patch was applied (using RPC instead of neutron client in metadata agent), we got **100% success and 2x improved average perfomance**:
 
 +------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
 | action           | min (sec) | avg (sec) | max (sec) | 90 percentile | 95 percentile | success | count |