More doc fixups/cleanups

docs/README.rst

@@ -0,0 +1,55 @@
+=================
+Building the docs
+=================
+
+Dependencies
+============
+
+Sphinx_
+  You'll need sphinx (the python one) and if you are
+  using the virtualenv you'll need to install it in the virtualenv
+  specifically so that it can load the nova modules.
+
+  ::
+
+    pip install Sphinx
+
+Graphviz_
+  Some of the diagrams are generated using the ``dot`` language
+  from Graphviz.
+
+  ::
+
+    sudo apt-get install graphviz
+
+.. _Sphinx: http://sphinx.pocoo.org
+
+.. _Graphviz: http://www.graphviz.org/
+
+
+Use `make`
+==========
+
+Just type make::
+
+  % make
+
+Look in the Makefile for more targets.
+
+
+Manually
+========
+
+  1. Generate the code.rst file so that Sphinx will pull in our docstrings::
+     
+      % ./generate_autodoc_index.sh > source/code.rst
+
+  2. Run `sphinx_build`::
+
+      % sphinx-build -b html source build/html
+
+
+The docs have been built
+========================
+
+Check out the `build` directory to find them. Yay!

docs/source/index.rst

@@ -9,4 +9,11 @@ DEVSTACKpy Documentation
 .. toctree::
    :maxdepth: 2
 
-   topics/index
+   topics/goals
+   topics/features
+   topics/warning
+   topics/basics
+   topics/qanda
+   topics/bugshugscode
+   topics/advanced/index
+   

docs/source/topics/advanced/addingyourowndistro.rst

@@ -0,0 +1,55 @@
+==========
+Adding your own distribution
+==========
+
+
+Your mission...
+=============
+
+So you have decided you want to venture into the bowels of DEVSTACKpy
+and want to get support for your latest and greatest distribution. This
+wiki will hopefully make that adventure simpler by listing out the key
+places, files and configs that may have to be adjusted to get that to
+work. This tape will self-destruct in 5 seconds. 1..2..3..4..5, boom!
+
+Steps
+=====
+
+Snapshot
+--------
+
+One of the most useful things to do will be to get a virtual machine
+with your distribution and setup a stable state. Create a snapshot of
+that stable state just in-case you *bork* your machine later on.
+
+Logging
+-------
+
+Now turn ensure you run ``DEBUG/VERBOSE`` logging using ``-vv``. This
+will be useful to see exactly what actions and commands are being ran
+instead of the default ``INFO`` level logging which is just meant for
+simple informational messages about the underlying actions which are
+occurring.
+
+Configs
+-------
+
+By looking at the config folder ``distros`` you should exactly which
+packages and pips and commands are needed for each component by looking
+at a similar distribution. So you first task is to determine exactly
+what versions are available for your distribution. If a version doesn’t
+exist the you may need to resort to either using the `pypi`_ index or
+having to package it yourself. If a version is to new, this is usually
+ok (your mileage may vary) and if its to old then that might also not be
+ok (your mileage may vary).
+
+Try it
+------
+
+Now that you have provided a new `YAML`_ distro file you should be able
+to run through the simple setup wiki and see if the install will pass.
+If that does try starting and then seeing if everything has started up
+correctly.
+
+.. _pypi: http://pypi.python.org
+.. _YAML: http://yaml.org/

docs/source/topics/advanced/addingyourownpersona.rst

@@ -0,0 +1,60 @@
+==========
+Adding your own persona
+==========
+
+
+Your mission...
+=============
+
+So you have decided you want to venture into the bowels of DEVSTACKpy
+and want to alter what is installed/started/stopped, the order of what
+is installed/started/stopped, what subsystems are activated (or the
+component options). This wiki will hopefully make that adventure simpler
+by listing out the key places, files and configs that may have to be
+adjusted to get that to work. This tape will self-destruct in 5 seconds.
+1..2..3..4..5, boom!
+
+Steps
+=====
+
+Snapshot
+--------
+
+One of the most useful things to do will be to get a virtual machine
+with your distribution and setup a stable state. Create a snapshot of
+that stable state just in-case you *bork* your machine later on.
+
+Logging
+-------
+
+Now turn ensure you run ``DEBUG/VERBOSE`` logging using ``-vv``. This
+will be useful to see exactly what actions and commands are being ran
+instead of the default ``INFO`` level logging which is just meant for
+simple informational messages about the underlying actions which are
+occurring.
+
+Configs
+-------
+
+By looking at the config folder ``personas`` you should a file called
+``devstack.sh.yaml``. This file contains the component order of
+installation (ie the ``db`` before ``keystone``), a nice useful
+description of the persona and subsystems for the previously specified
+components and any options these components may have. So you first task
+is to determine exactly what of these you wish to change (if any). Note
+that changing the component order may not always work (ie typically
+starting components are dependent, ie the message queue needs to be
+started before nova). To add in new components check the ``distros``
+folder to determine exactly what that component is named (typically this
+is common) and alter the persona file as desired. To alter the
+``subsystems`` or ``options`` section you will have to jump in the code
+and check for what these values could be (TODO make that better).
+
+Try it
+------
+
+Now that you have provided a new `YAML`_ persona file you should be able
+to run the ``stack`` program with that persona through the ``-p``
+option.
+
+.. _YAML: http://yaml.org/

docs/source/topics/advanced/design.rst

@@ -0,0 +1,163 @@
+========
+Design
+========
+
+How it works
+------------
+
+DEVSTACKpy is based along the following system design
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+-  Having shared components/actions be shared (using object oriented
+   practices)
+-  Having specific actions be isolated to its component (and easily
+   readable)
+-  Being simple enough to read yet following standard python software
+   development practices and patterns
+
+Directory structure is the following
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+-  Parent classes located in *devstack/component.py*, it contains the
+   root install/uninstall/start/stop classes
+-  Subclasses in *devstack/components/*, it contains the individual
+   install classes for each openstack component
+-  Running modes implementations in *devstack/runners/* (ie fork,
+   screen, upstart)
+-  Packaging implementations in *devstack/packaging/*
+-  Image uploading/registry/management (for glance) in *devstack/image/*
+-  Shared classes and utils in *devstack/*
+-  Main entry point/s in *devstack/progs/*
+-  Other various tools in *tools/*
+-  Configuration in *conf/* (see below)
+
+Example
+~~~~~~~
+
+Install object model
+^^^^^^^^^^^^^^^^^^^^
+
+Here is the **install** components (default set only) class hierarchy:
+
+.. figure:: http://farm8.staticflickr.com/7043/6894250521_d882b770d1_o.png
+   :align: center
+
+Classes
+'''''''
+
+From this example the root classes job are:
+
+-  Python install component
+-  Ensures *pips* are installed (child classes specify which pips)
+-  Ensures python directories (ie *setup.py*) are setup (child classes
+   specify which directories)
+-  Package install component
+-  Installs packages that are required for install (child classes
+   specify which packages)
+-  Sets up and performs parameter replacement on config files (child
+   classes specify which config files)
+-  Sets up symlinks to config or other files (child classes specify
+   these symlinks)
+-  Tracks what was setup so that it can be removed/uninstalled
+-  Component base (used by **all** install/uninstall/start/stop
+   component classes)
+-  Holds configuration object, component name, packaging and other
+   shared members…
+-  Allows for overriding of dependency function and pre-run verification
+
+Functions
+'''''''''
+
+For a install class the following functions are activated (in the
+following order by *devstack/progs/actions.py*):
+
+::
+
+      download()
+
+    Performs the main git download (or other download type) to the
+    application target directory.
+
+::
+
+      configure()
+
+    Configures the components files (symlinks, configuration and logging
+    files…)
+
+::
+
+      pre_install()
+
+    Child class specific function that can be used to do anything before
+    an install (ie set a ubuntu mysql pre-install root password)
+
+::
+
+      install()
+
+    Installs distribution packages, python packages (*pip*), sets up
+    python directories (ie *python setup.py develop*) and any other
+    child class specific actions.
+
+::
+
+      post_install()
+
+    Child class specific function that can be used to do anything after
+    an install (ie run *nova-manage db sync*)
+
+Other object models
+^^^^^^^^^^^^^^^^^^^
+
+-  `Start object model (for default set only)`_
+-  `Stop object model (for default set only)`_
+-  `Uninstall object model (for default set only)`_
+
+Configuring
+-----------
+
+For those of you that are brave enough to change *stack* here are some
+starting points.
+
+conf/stack.ini
+~~~~~~~~~~~~~~
+
+Check out *conf/stack.ini* for various configuration settings applied
+(branches, git repositories…). Check out the header of that file for how
+the customized configuration values are parsed and what they may result
+in.
+
+conf/distros
+~~~~~~~~~~~~
+
+Check out *conf/distros* for the `YAML`_ files that describe
+pkgs/cmds/pips for various distributions which are required by the
+different OpenStack components to run correctly. The versions and pip
+names listed for each distribution should be the correct version that is
+known to work with a given OpenStack release.
+
+conf/templates/
+~~~~~~~~~~~~~~~
+
+Check out *conf/templates/* for various component specific settings and
+files. All of these files are *templates* with sections or text that
+needs to be filled in by the ``stack`` script to become a *complete*
+file.
+
+These files may have strings of the format ``%NAME%`` where ``NAME``
+will most often be adjusted to a real value by the *stack* script.
+
+An example where this is useful is say for the following line:
+
+::
+
+       admin_token = %SERVICE_TOKEN% 
+
+Since the script will either prompt for this value (or generate it for
+you) we can not have this statically set in a configuration file.
+
+.. _Start object model (for default set only): http://farm8.staticflickr.com/7046/6894981327_a583bcb4fc_o.png
+.. _Stop object model (for default set only): http://farm8.staticflickr.com/7059/6894981341_e6d4901b20_o.png
+.. _Uninstall object model (for default set only): http://farm8.staticflickr.com/7177/6894981357_fef65b28d3_o.png
+.. _YAML: http://yaml.org/

docs/source/topics/advanced/index.rst

@@ -0,0 +1,11 @@
+===============
+Advanced
+===============
+
+.. toctree::
+   :maxdepth: 2
+
+   addingyourowndistro
+   addingyourownpersona
+   design
+

docs/source/topics/basics.rst

@@ -1,4 +1,5 @@
-Topics
+=========
+Basics
 =========
 
 .. toctree::
@@ -6,6 +7,5 @@ Topics
 
    gettingstarted
    usageexamples
-   bugshugscode
-   qanda
    solvedproblems
+   

docs/source/topics/bugshugscode.rst

@@ -34,6 +34,12 @@ Bugs/Features
 
 Please use `github’s issue tracking system`_ or `launchpad’s issue tracking system`_ to report or follow bugs or to discuss features and get support.
 
+Hacking
+=============
+
+Feel free to hack but please try to follow the `hacking guidelines`_
+
+
 Discussions
 ===========
 
@@ -43,3 +49,4 @@ Please either use `launchpad’s email system`_ or find us on ``irc.freenode.net
 .. _github’s issue tracking system: https://github.com/yahoo/Openstack-Devstackpy/issues
 .. _launchpad’s issue tracking system: http://launchpad.net/~devstackpy
 .. _launchpad’s email system: https://launchpad.net/%7Edevstackpy/+contactuser
+.. _hacking guidelines: https://github.com/yahoo/Openstack-DevstackPy/blob/master/HACKING.md

docs/source/topics/features.rst

@@ -0,0 +1,34 @@
+========
+Features
+========
+
+-  Supports more than one distribution.
+
+   -  Currently RHEL 6.2 (with `epel`_), Ubuntu 11.10, Fedora 16
+
+-  Supports dry-run mode (to see what *would* happen)
+-  Supports varying installation *personas*
+-  A single ``stack.ini`` file that shows common configuration
+-  Supports install/uninstall/starting/stopping of OpenStack components.
+
+   -  In various styles (daemonizing via `forking`_, `screen`_, `upstart`_)
+
+-  Written in python so it matches the style of other `OpenStack`_ components.
+-  Extensively documented distribution specifics
+
+   -  Packages and pip (with versions known to work!) dependencies
+   -  Any needed distribution specific actions (ie service names…)
+
+-  Follows standard software development practices (for everyones sanity).
+
+   -  Functions, classes, objects and more (oh my!)
+   -  Still *readable* by someone with limited python knowledge.
+
+-  The ability to be unit-tested!
+-  Extensive logging
+
+.. _epel: http://fedoraproject.org/wiki/EPEL
+.. _forking: http://users.telenet.be/bartl/classicperl/fork/all.html
+.. _screen: http://www.manpagez.com/man/1/screen/
+.. _upstart: http://upstart.ubuntu.com/
+.. _OpenStack: http://openstack.org/

docs/source/topics/goals.rst

@@ -0,0 +1,23 @@
+===============
+Goals
+===============
+
+-  To aid developers getting involved with `OpenStack`_!
+-  To quickly build developer `OpenStack`_ environments in a clean
+   environment (as well as start, stop, and uninstall those
+   environments) with as little baggage as possible.
+-  To describe working configurations of `OpenStack`_.
+
+   -  Which code branches work together?
+   -  What do config files look like for those branches?
+   -  What packages are needed for installation for a given
+      distribution?
+
+-  To make it easier for developers to dive into `OpenStack`_ so that
+   they can productively contribute without having to understand every
+   part of the system at once.
+-  To make it easy to prototype cross-project features.
+
+
+.. _OpenStack: http://openstack.org/
+

docs/source/topics/warning.rst

@@ -0,0 +1,15 @@
+==========
+Important!
+==========
+
+**Warning:** Be sure to carefully read ``stack`` and any other scripts
+you execute before you run them, as they install software and may alter
+your networking configuration. We strongly recommend that you run
+``stack`` in a clean and disposable virtual machine when you are first
+getting started.
+
+.. _epel: http://fedoraproject.org/wiki/EPEL
+.. _forking: http://users.telenet.be/bartl/classicperl/fork/all.html
+.. _screen: http://www.manpagez.com/man/1/screen/
+.. _upstart: http://upstart.ubuntu.com/
+