Development overhaul

doc/source/develop/extending.rst

@@ -0,0 +1,13 @@
+`Home <index.html>`_ OpenStack-Salt Development Documentation
+
+Chapter 3. Extending
+=====================
+
+.. toctree::
+
+   extending-formulas.rst
+   extending-contribute.rst
+
+--------------
+
+.. include:: navigation.txt

doc/source/develop/index.rst

@@ -1,17 +1,36 @@
-Developer Documentation
-=======================
+Development documentation
+=============================
 
 In this section, you will find documentation relevant to developing
 openstack-salt.
 
 Contents:
 
+
+Quick start
+^^^^^^^^^^^
+
 .. toctree::
    :maxdepth: 2
 
-   quickstart
-   formulas
-   contribute
+   quickstart.rst
+
+Testing
+^^^^^^^
+
+.. toctree::
+   :maxdepth: 2
+
+   testing.rst
+
+
+Extending
+^^^^^^^^^
+
+.. toctree::
+   :maxdepth: 2
+
+   extending.rst
 
 
 Indices and tables

doc/source/develop/quickstart-heat.rst

@@ -0,0 +1,72 @@
+`Home <index.html>`_ OpenStack-Salt Development Documentation
+
+Heat Stack deployment
+=====================
+
+All-in-one (AIO) deployments are a great way to setup an OpenStack-Salt cloud for:
+
+* a service development environment
+* an overview of how all of the OpenStack services and roles play together
+* a simple lab deployment for testing
+
+It is possible to run full size proof-of-concept deployment on OpenStack with `Heat template`, the stack has following requirements: 
+
+* At least 200GB disk space
+* 70GB RAM
+
+.. _Heat template: https://github.com/tcpcloud/heat-templates
+
+Environment setup 
+-----------------
+
+To install heat client, it's recommended to setup Python virtualenv and
+install tested versions of openstack clients that are defined in
+`requirements.txt` file.
+
+Install build tools (eg. on Ubuntu):
+  .. code-block:: bash
+
+     apt-get install python-dev python-pip python-virtualenv build-essential libffi-dev libssl-dev
+
+Create and activate virtualenv named `venv-heat`:
+  .. code-block:: bash
+
+     virtualenv venv-heat
+     source ./venv-heat/bin/activate
+
+Install requirements:
+  .. code-block:: bash
+
+     pip install -r requirements.txt
+
+
+List of available stacks
+-------------------------
+
+.. list-table::
+   :stub-columns: 1
+
+   *  - salt_single_public
+      - Base stack which deploys network and single-node Salt master
+   *  - openstack_cluster_public
+      - Deploy OpenStack cluster with OpenContrail, requires
+        ``salt_single_public``
+
+
+Launching the Heat stack
+------------------------
+
+#. Setup environment file, eg. ``env/salt_single_public.env``, look at example
+   file first
+#. Source credentials and required environment variables. You can download
+   openrc file from Horizon dashboard.
+
+   .. code-block:: bash
+
+     source my_tenant-openrc.sh
+
+#. Deploy the actual stack
+
+   .. code-block:: bash
+
+     ./create_stack.sh salt_single_public

doc/source/develop/quickstart-vagrant.rst

@@ -0,0 +1,23 @@
+`Home <index.html>`_ OpenStack-Salt Development Documentation
+
+Vagrant deployment
+==================
+
+All-in-one (AIO) deployments are a great way to setup an OpenStack-Salt cloud for:
+
+* a service development environment
+* an overview of how all of the OpenStack services and roles play together
+* a simple lab deployment for testing
+
+Although AIO builds aren't recommended for large production deployments,
+they're great for smal proof-of-concept deployments.
+
+It's strongly recommended to have hardware that meets the following
+requirements before starting an AIO build:
+
+* CPU that supports `hardware-assisted virtualization`_
+* At least 80GB disk space
+* 16GB RAM
+
+.. _hardware-assisted virtualization: https://en.wikipedia.org/wiki/Hardware-assisted_virtualization
+

doc/source/develop/quickstart.rst

@@ -1,30 +1,14 @@
 `Home <index.html>`_ OpenStack-Salt Development Documentation
 
-Quick Start
-===========
+Chapter 1. Quick start
+=======================
 
-All-in-one (AIO) deployments are a great way to setup an OpenStack-Salt cloud for:
 
-* a service development environment
-* an overview of how all of the OpenStack services and roles play together
-* a simple lab deployment for testing
+.. toctree::
 
-Although AIO builds aren't recommended for large production deployments,
-they're great for smal proof-of-concept deployments.
+   quickstart-vagrant.rst
+   quickstart-heat.rst
 
-It's strongly recommended to have hardware that meets the following
-requirements before starting an AIO build:
-
-* CPU that supports `hardware-assisted virtualization`_
-* At least 80GB disk space
-* 16GB RAM
-
-It is possible to run full size proof-of-concept deployment on OpenStack with `Heat template`, the stack has following
-requirements: 
-
-* At least 200GB disk space
-* 70GB RAM
-
-.. _hardware-assisted virtualization: https://en.wikipedia.org/wiki/Hardware-assisted_virtualization
-.. _Heat template: https://github.com/tcpcloud/heat-templates
+--------------
 
+.. include:: navigation.txt

doc/source/develop/testing-coding-style.rst

@@ -0,0 +1,38 @@
+
+Salt Formula coding style
+=============================
+
+
+
+https://github.com/johanek/salt-lint.git
+
+
+Using double quotes with no variables
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In general - it's a bad idea. All the strings which does not contain dynamic content ( variables ) should use single quote instead of double.
+
+
+Line length above 80 characters
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+As a 'standard code width limit' and for historical reasons - [IBM punch card](http://en.wikipedia.org/wiki/Punched_card) had exactly 80 columns.
+
+Found single line declaration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Avoid extending your code by adding single-line declarations. It makes your code much cleaner and easier to parse / grep while searching for those declarations.
+
+No newline at the end of the file
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Each line should be terminated in a newline character, including the last one. Some programs have problems processing the last line of a file if it isn't newline terminated. [Stackoverflow thread](http://stackoverflow.com/questions/729692/why-should-files-end-with-a-newline)
+
+Trailing whitespace character found
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Trailing whitespaces take more spaces than necessary, any regexp based searches won't return lines as a result due to trailing whitespace(s).
+
+--------------
+
+.. include:: navigation.txt

doc/source/develop/testing-integration.rst

@@ -0,0 +1,66 @@
+
+Integration testing
+=====================
+
+There are requirements, in addition to Salt's requirements, which need to be installed in order to run the test suite. Install the line below.
+
+.. code-block:: bash
+
+   pip install -r requirements/dev_python27.txt
+
+Once all require requirements are set, use ``tests/runtests.py`` to run all of the tests included in Salt's test suite. For more information, see --help.
+
+Running the tests
+-----------------
+
+An alternative way of invoking the test suite is available in setup.py:
+
+.. code-block:: bash
+
+   ./setup.py test
+
+Instead of running the entire test suite, there are several ways to run only specific groups of tests or individual tests:
+
+* Run unit tests only: ./tests/runtests.py --unit-tests
+* Run unit and integration tests for states: ./tests/runtests.py --state
+* Run integration tests for an individual module: ./tests/runtests.py -n integration.modules.virt
+* Run unit tests for an individual module: ./tests/runtests.py -n unit.modules.virt_test
+* Run an individual test by using the class and test name (this example is for the test_default_kvm_profile test in the integration.module.virt)
+
+Running Unit tests without integration test daemons
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Since the unit tests do not require a master or minion to execute, it is often useful to be able to run unit tests individually, or as a whole group, without having to start up the integration testing daemons. Starting up the master, minion, and syndic daemons takes a lot of time before the tests can even start running and is unnecessary to run unit tests. To run unit tests without invoking the integration test daemons, simple remove the /tests portion of the runtests.py command:
+
+.. code-block:: bash
+
+   ./runtests.py --unit
+
+All of the other options to run individual tests, entire classes of tests, or entire test modules still apply.
+
+
+Destructive integration tests
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Salt is used to change the settings and behavior of systems. In order to effectively test Salt's functionality, some integration tests are written to make actual changes to the underlying system. These tests are referred to as "destructive tests". Some examples of destructive tests are changes may be testing the addition of a user or installing packages. By default, destructive tests are disabled and will be skipped.
+
+Generally, destructive tests should clean up after themselves by attempting to restore the system to its original state. For instance, if a new user is created during a test, the user should be deleted after the related test(s) have completed. However, no guarantees are made that test clean-up will complete successfully. Therefore, running destructive tests should be done with caution.
+
+To run tests marked as destructive, set the ``--run-destructive`` flag:
+
+.. code-block:: bash
+
+   ./tests/runtests.py --run-destructive
+
+Automated test runs
+-------------------
+
+Jenkins server executes series of tests across supported platforms. The tests executed from OpenStack-Salt's Jenkins server create fresh virtual machines for each test run, then execute destructive tests on the new, clean virtual machines.
+
+When a pull request is submitted to OpenStack-Salt's repository, Jenkins runs Salt's test suite on a couple of virtual machines to gauge the pull request's viability to merge into OpenStack-Salt's develop branch. If these initial tests pass, the pull request can then merged into OpenStack-Salt's develop branch by one of OpenStack-Salt's core developers, pending their discretion. If the initial tests fail, core developers may request changes to the pull request. If the failure is unrelated to the changes in question, core developers may merge the pull request despite the initial failure.
+
+Once the pull request is merged into OpenStack-Salt's develop branch, a new set of Jenkins virtual machines will begin executing the test suite. The develop branch tests have many more virtual machines to provide more comprehensive results.
+
+--------------
+
+.. include:: navigation.txt

doc/source/develop/testing-metadata.rst

@@ -0,0 +1,20 @@
+
+Metadata validation
+===================
+
+Pillars are tree-like structures of data defined on the Salt Master and passed through to minions. They allow confidential, targeted data to be securely sent only to the relevant minion. Pillar is therefore one of the most important systems when using Salt.
+
+
+Testing scenarios
+-----------------
+
+Our testing plan is to test each state with the example pillar:
+
+#. Run ``state.show_sls`` to ensure that it parses properly and have some debugging output,
+#. Run ``state.sls`` to run the state we’re on,
+#. Run ``state.sls again, capturing output, asserting that ``^Not Run:`` is not present in the output, because if it is then it means that a state cannot detect by itself whether it has to be run or not and thus is not idempotent.
+
+
+--------------
+
+.. include:: navigation.txt

doc/source/develop/testing.rst

@@ -0,0 +1,14 @@
+`Home <index.html>`_ OpenStack-Salt Development Documentation
+
+Chapter 2. Testing
+=====================
+
+.. toctree::
+
+   testing-coding-style.rst
+   testing-metadata.rst
+   testing-integration.rst
+
+--------------
+
+.. include:: navigation.txt

doc/source/index.rst

@@ -8,7 +8,7 @@ Welcome to OpenStack-Salt's documentation!
 Overview
 ========
 
-This project provides Salt configuration templates (formulas) with required metadata (pillars) for installing and operating OpenStack cloud.
+This project provides complete set of Salt configuration templates for installing and operating robust OpenStack cloud deployments.
 
 Contents:
 

doc/source/install/index.rst

@@ -1,5 +1,5 @@
 ==================================
-OpenStack-Salt Installation Manual
+OpenStack-Salt installation manual
 ==================================
 
 `Home <index.html>`_ OpenStack-Salt Installation Manual
@@ -41,7 +41,6 @@ Installation
 
 .. toctree::
 
-   install.rst
    install-config.rst
    install-infrastructure.rst
    install-openstack.rst

doc/source/operate/index.rst

@@ -1,5 +1,5 @@
 ================================
-OpenStack-Salt Operations Manual
+OpenStack-Salt operations manual
 ================================
 
 `Home <index.html>`_ OpenStack-Salt Operations Manual

doc/source/operate/overview.rst

@@ -5,8 +5,6 @@ Chapter 1. Overview
 
 .. toctree::
 
-   overview-salt.rst
-   overview-security.rst
 
 --------------
 

doc/source/operate/troubleshoot-networking.rst

@@ -0,0 +1,8 @@
+
+Troubleshooting networking
+============================
+
+
+--------------
+
+.. include:: navigation.txt

doc/source/operate/troubleshoot.rst

@@ -1,7 +1,12 @@
+
 ====================================
 Chapter 3. Troubleshooting OpenStack
 ====================================
 
+.. toctree::
+
+   troubleshoot-networking.rst
+
 --------------
 
 .. include:: navigation.txt