From 1970eaa56afcbd4689d51fb079ef96e7674e5457 Mon Sep 17 00:00:00 2001 From: Joshua Harlow Date: Fri, 30 Mar 2012 13:59:29 -0700 Subject: [PATCH] More doc fixups/cleanups --- docs/README.rst | 55 ++++++ docs/source/index.rst | 9 +- .../topics/advanced/addingyourowndistro.rst | 55 ++++++ .../topics/advanced/addingyourownpersona.rst | 60 +++++++ docs/source/topics/advanced/design.rst | 163 ++++++++++++++++++ docs/source/topics/advanced/index.rst | 11 ++ docs/source/topics/{index.rst => basics.rst} | 8 +- docs/source/topics/bugshugscode.rst | 7 + docs/source/topics/features.rst | 34 ++++ docs/source/topics/goals.rst | 23 +++ docs/source/topics/warning.rst | 15 ++ 11 files changed, 435 insertions(+), 5 deletions(-) create mode 100644 docs/README.rst create mode 100644 docs/source/topics/advanced/addingyourowndistro.rst create mode 100644 docs/source/topics/advanced/addingyourownpersona.rst create mode 100644 docs/source/topics/advanced/design.rst create mode 100644 docs/source/topics/advanced/index.rst rename docs/source/topics/{index.rst => basics.rst} (60%) create mode 100644 docs/source/topics/features.rst create mode 100644 docs/source/topics/goals.rst create mode 100644 docs/source/topics/warning.rst diff --git a/docs/README.rst b/docs/README.rst new file mode 100644 index 00000000..7e5229a1 --- /dev/null +++ b/docs/README.rst @@ -0,0 +1,55 @@ +================= +Building the docs +================= + +Dependencies +============ + +Sphinx_ + You'll need sphinx (the python one) and if you are + using the virtualenv you'll need to install it in the virtualenv + specifically so that it can load the nova modules. + + :: + + pip install Sphinx + +Graphviz_ + Some of the diagrams are generated using the ``dot`` language + from Graphviz. + + :: + + sudo apt-get install graphviz + +.. _Sphinx: http://sphinx.pocoo.org + +.. _Graphviz: http://www.graphviz.org/ + + +Use `make` +========== + +Just type make:: + + % make + +Look in the Makefile for more targets. + + +Manually +======== + + 1. Generate the code.rst file so that Sphinx will pull in our docstrings:: + + % ./generate_autodoc_index.sh > source/code.rst + + 2. Run `sphinx_build`:: + + % sphinx-build -b html source build/html + + +The docs have been built +======================== + +Check out the `build` directory to find them. Yay! diff --git a/docs/source/index.rst b/docs/source/index.rst index c2ffe44b..d527a04e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -9,4 +9,11 @@ DEVSTACKpy Documentation .. toctree:: :maxdepth: 2 - topics/index + topics/goals + topics/features + topics/warning + topics/basics + topics/qanda + topics/bugshugscode + topics/advanced/index + diff --git a/docs/source/topics/advanced/addingyourowndistro.rst b/docs/source/topics/advanced/addingyourowndistro.rst new file mode 100644 index 00000000..722dfe15 --- /dev/null +++ b/docs/source/topics/advanced/addingyourowndistro.rst @@ -0,0 +1,55 @@ +========== +Adding your own distribution +========== + + +Your mission... +============= + +So you have decided you want to venture into the bowels of DEVSTACKpy +and want to get support for your latest and greatest distribution. This +wiki will hopefully make that adventure simpler by listing out the key +places, files and configs that may have to be adjusted to get that to +work. This tape will self-destruct in 5 seconds. 1..2..3..4..5, boom! + +Steps +===== + +Snapshot +-------- + +One of the most useful things to do will be to get a virtual machine +with your distribution and setup a stable state. Create a snapshot of +that stable state just in-case you *bork* your machine later on. + +Logging +------- + +Now turn ensure you run ``DEBUG/VERBOSE`` logging using ``-vv``. This +will be useful to see exactly what actions and commands are being ran +instead of the default ``INFO`` level logging which is just meant for +simple informational messages about the underlying actions which are +occurring. + +Configs +------- + +By looking at the config folder ``distros`` you should exactly which +packages and pips and commands are needed for each component by looking +at a similar distribution. So you first task is to determine exactly +what versions are available for your distribution. If a version doesn’t +exist the you may need to resort to either using the `pypi`_ index or +having to package it yourself. If a version is to new, this is usually +ok (your mileage may vary) and if its to old then that might also not be +ok (your mileage may vary). + +Try it +------ + +Now that you have provided a new `YAML`_ distro file you should be able +to run through the simple setup wiki and see if the install will pass. +If that does try starting and then seeing if everything has started up +correctly. + +.. _pypi: http://pypi.python.org +.. _YAML: http://yaml.org/ \ No newline at end of file diff --git a/docs/source/topics/advanced/addingyourownpersona.rst b/docs/source/topics/advanced/addingyourownpersona.rst new file mode 100644 index 00000000..8240c3ae --- /dev/null +++ b/docs/source/topics/advanced/addingyourownpersona.rst @@ -0,0 +1,60 @@ +========== +Adding your own persona +========== + + +Your mission... +============= + +So you have decided you want to venture into the bowels of DEVSTACKpy +and want to alter what is installed/started/stopped, the order of what +is installed/started/stopped, what subsystems are activated (or the +component options). This wiki will hopefully make that adventure simpler +by listing out the key places, files and configs that may have to be +adjusted to get that to work. This tape will self-destruct in 5 seconds. +1..2..3..4..5, boom! + +Steps +===== + +Snapshot +-------- + +One of the most useful things to do will be to get a virtual machine +with your distribution and setup a stable state. Create a snapshot of +that stable state just in-case you *bork* your machine later on. + +Logging +------- + +Now turn ensure you run ``DEBUG/VERBOSE`` logging using ``-vv``. This +will be useful to see exactly what actions and commands are being ran +instead of the default ``INFO`` level logging which is just meant for +simple informational messages about the underlying actions which are +occurring. + +Configs +------- + +By looking at the config folder ``personas`` you should a file called +``devstack.sh.yaml``. This file contains the component order of +installation (ie the ``db`` before ``keystone``), a nice useful +description of the persona and subsystems for the previously specified +components and any options these components may have. So you first task +is to determine exactly what of these you wish to change (if any). Note +that changing the component order may not always work (ie typically +starting components are dependent, ie the message queue needs to be +started before nova). To add in new components check the ``distros`` +folder to determine exactly what that component is named (typically this +is common) and alter the persona file as desired. To alter the +``subsystems`` or ``options`` section you will have to jump in the code +and check for what these values could be (TODO make that better). + +Try it +------ + +Now that you have provided a new `YAML`_ persona file you should be able +to run the ``stack`` program with that persona through the ``-p`` +option. + +.. _YAML: http://yaml.org/ \ No newline at end of file diff --git a/docs/source/topics/advanced/design.rst b/docs/source/topics/advanced/design.rst new file mode 100644 index 00000000..032a3546 --- /dev/null +++ b/docs/source/topics/advanced/design.rst @@ -0,0 +1,163 @@ +======== +Design +======== + +How it works +------------ + +DEVSTACKpy is based along the following system design +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Having shared components/actions be shared (using object oriented + practices) +- Having specific actions be isolated to its component (and easily + readable) +- Being simple enough to read yet following standard python software + development practices and patterns + +Directory structure is the following +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Parent classes located in *devstack/component.py*, it contains the + root install/uninstall/start/stop classes +- Subclasses in *devstack/components/*, it contains the individual + install classes for each openstack component +- Running modes implementations in *devstack/runners/* (ie fork, + screen, upstart) +- Packaging implementations in *devstack/packaging/* +- Image uploading/registry/management (for glance) in *devstack/image/* +- Shared classes and utils in *devstack/* +- Main entry point/s in *devstack/progs/* +- Other various tools in *tools/* +- Configuration in *conf/* (see below) + +Example +~~~~~~~ + +Install object model +^^^^^^^^^^^^^^^^^^^^ + +Here is the **install** components (default set only) class hierarchy: + +.. figure:: http://farm8.staticflickr.com/7043/6894250521_d882b770d1_o.png + :align: center + +Classes +''''''' + +From this example the root classes job are: + +- Python install component +- Ensures *pips* are installed (child classes specify which pips) +- Ensures python directories (ie *setup.py*) are setup (child classes + specify which directories) +- Package install component +- Installs packages that are required for install (child classes + specify which packages) +- Sets up and performs parameter replacement on config files (child + classes specify which config files) +- Sets up symlinks to config or other files (child classes specify + these symlinks) +- Tracks what was setup so that it can be removed/uninstalled +- Component base (used by **all** install/uninstall/start/stop + component classes) +- Holds configuration object, component name, packaging and other + shared members… +- Allows for overriding of dependency function and pre-run verification + +Functions +''''''''' + +For a install class the following functions are activated (in the +following order by *devstack/progs/actions.py*): + +:: + + download() + + Performs the main git download (or other download type) to the + application target directory. + +:: + + configure() + + Configures the components files (symlinks, configuration and logging + files…) + +:: + + pre_install() + + Child class specific function that can be used to do anything before + an install (ie set a ubuntu mysql pre-install root password) + +:: + + install() + + Installs distribution packages, python packages (*pip*), sets up + python directories (ie *python setup.py develop*) and any other + child class specific actions. + +:: + + post_install() + + Child class specific function that can be used to do anything after + an install (ie run *nova-manage db sync*) + +Other object models +^^^^^^^^^^^^^^^^^^^ + +- `Start object model (for default set only)`_ +- `Stop object model (for default set only)`_ +- `Uninstall object model (for default set only)`_ + +Configuring +----------- + +For those of you that are brave enough to change *stack* here are some +starting points. + +conf/stack.ini +~~~~~~~~~~~~~~ + +Check out *conf/stack.ini* for various configuration settings applied +(branches, git repositories…). Check out the header of that file for how +the customized configuration values are parsed and what they may result +in. + +conf/distros +~~~~~~~~~~~~ + +Check out *conf/distros* for the `YAML`_ files that describe +pkgs/cmds/pips for various distributions which are required by the +different OpenStack components to run correctly. The versions and pip +names listed for each distribution should be the correct version that is +known to work with a given OpenStack release. + +conf/templates/ +~~~~~~~~~~~~~~~ + +Check out *conf/templates/* for various component specific settings and +files. All of these files are *templates* with sections or text that +needs to be filled in by the ``stack`` script to become a *complete* +file. + +These files may have strings of the format ``%NAME%`` where ``NAME`` +will most often be adjusted to a real value by the *stack* script. + +An example where this is useful is say for the following line: + +:: + + admin_token = %SERVICE_TOKEN% + +Since the script will either prompt for this value (or generate it for +you) we can not have this statically set in a configuration file. + +.. _Start object model (for default set only): http://farm8.staticflickr.com/7046/6894981327_a583bcb4fc_o.png +.. _Stop object model (for default set only): http://farm8.staticflickr.com/7059/6894981341_e6d4901b20_o.png +.. _Uninstall object model (for default set only): http://farm8.staticflickr.com/7177/6894981357_fef65b28d3_o.png +.. _YAML: http://yaml.org/ \ No newline at end of file diff --git a/docs/source/topics/advanced/index.rst b/docs/source/topics/advanced/index.rst new file mode 100644 index 00000000..60c6b1ca --- /dev/null +++ b/docs/source/topics/advanced/index.rst @@ -0,0 +1,11 @@ +=============== +Advanced +=============== + +.. toctree:: + :maxdepth: 2 + + addingyourowndistro + addingyourownpersona + design + diff --git a/docs/source/topics/index.rst b/docs/source/topics/basics.rst similarity index 60% rename from docs/source/topics/index.rst rename to docs/source/topics/basics.rst index f6153906..2d67f1da 100644 --- a/docs/source/topics/index.rst +++ b/docs/source/topics/basics.rst @@ -1,4 +1,5 @@ -Topics +========= +Basics ========= .. toctree:: @@ -6,6 +7,5 @@ Topics gettingstarted usageexamples - bugshugscode - qanda - solvedproblems \ No newline at end of file + solvedproblems + \ No newline at end of file diff --git a/docs/source/topics/bugshugscode.rst b/docs/source/topics/bugshugscode.rst index e23b61fb..cbe9e707 100644 --- a/docs/source/topics/bugshugscode.rst +++ b/docs/source/topics/bugshugscode.rst @@ -34,6 +34,12 @@ Bugs/Features Please use `github’s issue tracking system`_ or `launchpad’s issue tracking system`_ to report or follow bugs or to discuss features and get support. +Hacking +============= + +Feel free to hack but please try to follow the `hacking guidelines`_ + + Discussions =========== @@ -43,3 +49,4 @@ Please either use `launchpad’s email system`_ or find us on ``irc.freenode.net .. _github’s issue tracking system: https://github.com/yahoo/Openstack-Devstackpy/issues .. _launchpad’s issue tracking system: http://launchpad.net/~devstackpy .. _launchpad’s email system: https://launchpad.net/%7Edevstackpy/+contactuser +.. _hacking guidelines: https://github.com/yahoo/Openstack-DevstackPy/blob/master/HACKING.md diff --git a/docs/source/topics/features.rst b/docs/source/topics/features.rst new file mode 100644 index 00000000..bfe6f287 --- /dev/null +++ b/docs/source/topics/features.rst @@ -0,0 +1,34 @@ +======== +Features +======== + +- Supports more than one distribution. + + - Currently RHEL 6.2 (with `epel`_), Ubuntu 11.10, Fedora 16 + +- Supports dry-run mode (to see what *would* happen) +- Supports varying installation *personas* +- A single ``stack.ini`` file that shows common configuration +- Supports install/uninstall/starting/stopping of OpenStack components. + + - In various styles (daemonizing via `forking`_, `screen`_, `upstart`_) + +- Written in python so it matches the style of other `OpenStack`_ components. +- Extensively documented distribution specifics + + - Packages and pip (with versions known to work!) dependencies + - Any needed distribution specific actions (ie service names…) + +- Follows standard software development practices (for everyones sanity). + + - Functions, classes, objects and more (oh my!) + - Still *readable* by someone with limited python knowledge. + +- The ability to be unit-tested! +- Extensive logging + +.. _epel: http://fedoraproject.org/wiki/EPEL +.. _forking: http://users.telenet.be/bartl/classicperl/fork/all.html +.. _screen: http://www.manpagez.com/man/1/screen/ +.. _upstart: http://upstart.ubuntu.com/ +.. _OpenStack: http://openstack.org/ diff --git a/docs/source/topics/goals.rst b/docs/source/topics/goals.rst new file mode 100644 index 00000000..d241035f --- /dev/null +++ b/docs/source/topics/goals.rst @@ -0,0 +1,23 @@ +=============== +Goals +=============== + +- To aid developers getting involved with `OpenStack`_! +- To quickly build developer `OpenStack`_ environments in a clean + environment (as well as start, stop, and uninstall those + environments) with as little baggage as possible. +- To describe working configurations of `OpenStack`_. + + - Which code branches work together? + - What do config files look like for those branches? + - What packages are needed for installation for a given + distribution? + +- To make it easier for developers to dive into `OpenStack`_ so that + they can productively contribute without having to understand every + part of the system at once. +- To make it easy to prototype cross-project features. + + +.. _OpenStack: http://openstack.org/ + diff --git a/docs/source/topics/warning.rst b/docs/source/topics/warning.rst new file mode 100644 index 00000000..1789c7c0 --- /dev/null +++ b/docs/source/topics/warning.rst @@ -0,0 +1,15 @@ +========== +Important! +========== + +**Warning:** Be sure to carefully read ``stack`` and any other scripts +you execute before you run them, as they install software and may alter +your networking configuration. We strongly recommend that you run +``stack`` in a clean and disposable virtual machine when you are first +getting started. + +.. _epel: http://fedoraproject.org/wiki/EPEL +.. _forking: http://users.telenet.be/bartl/classicperl/fork/all.html +.. _screen: http://www.manpagez.com/man/1/screen/ +.. _upstart: http://upstart.ubuntu.com/ +