Migrate documentation to Sphinx

.travis.yml

@@ -31,6 +31,9 @@ matrix:
     # Run coverage checks.
     - python: 3.5
       env: TOX_ENV=cover
+    # Generate docs.
+    - python: 3.5
+      env: TOX_ENV=docs
 
 install:
   # Install tox in a virtualenv to ensure we have an up to date version.

README.md

@@ -1,120 +0,0 @@
-# Tenks
-
-Tenks is a utility that manages virtual bare metal clusters for development and
-testing purposes.
-
-## Getting Started
-
-### Pre-Requisites
-
-Tenks has dependencies on Ansible roles that are hosted by Ansible Galaxy.
-Given that your virtualenv of choice is active and Ansible (>=2.6) is
-installed inside it, Tenks' role dependencies can be installed by
-`ansible-galaxy install --role-file=requirements.yml
---roles-path=ansible/roles/`.
-
-### Hosts
-
-Tenks uses Ansible inventory to manage hosts. A multi-host setup is therefore
-supported, although the default hosts configuration will deploy an all-in-one
-setup on the host where the `ansible-playbook` command is executed
-(`localhost`).
-
-* Configuration management of the Tenks cluster is always performed on
-  `localhost`.
-* The `hypervisors` group should not directly contain any hosts. Its sub-groups
-  must contain one or more system. Systems in its sub-groups will host a subset
-  of the nodes deployed by Tenks.
-
-    * The `libvirt` group is a sub-group of `hypervisors`. Systems in this
-      group will act as hypervisors using the Libvirt provider.
-
-### Configuration
-
-An override file should be created to configure Tenks. Any variables specified
-in this file will take precedence over their default settings in Tenks. This
-will allow you to set options as necessary for your setup, without needing to
-directly modify Tenks' variable files. An example override file can be found
-in `ansible/override.yml.example`.
-
-Most of the configuration you will need to do relates to variables defined in
-`ansible/host_vars/localhost`. You can set your own values for these in your
-override file (mentioned above). In addition to other options, you will need to
-define the types of node you'd like to be able to manage as a dict in
-`node_types`, as well as the desired deployment specifications in `specs`.
-Format and guidance for available options will be found within the variable
-file.
-
-Broadly, most variables in `ansible/group_vars/*` have sensible defaults which
-may be left as-is unless you have a particular need to configure them. A
-notable exception to this is the variable `physnet_mappings` in
-`ansible/group_vars/hypervisors`, which should map physical network names to
-the device to use for that network: this can be a network interface, or an
-existing OVS or Linux bridge. If these mappings are the same for all hosts in
-your `hypervisors` group, you may set a single dict `physnet_mappings` in your
-overrides file, and this will be used for all hosts. If different mappings are
-required for different hosts, you will need to individually specify them: for a
-host with hostname *myhost*, set `physnet_mappings` within the file
-`ansible/host_vars/myhost`.
-
-### Commands
-
-Tenks has a variable `cmd` which specifies the command to be run. This variable
-can be set in your override file (see above). The possible values it can take
-are:
-
-* `deploy`: create a virtual cluster to the specification given. This is the
-  default command.
-* `teardown`: tear down any existing virtual cluster with the specification
-  given.
-
-### Running Tenks
-
-Currently, Tenks does not have a CLI or wrapper. Deployment can be run by
-calling `ansible-playbook --inventory ansible/inventory ansible/deploy.yml
---extra-vars=@override.yml`, where `override.yml` is the path to your override
-file.
-
-The `deploy.yml` playbook will run deployment from start to finish;
-`teardown.yml` is `deploy.yml`'s "mirror image" to tear down a cluster. These
-playbooks automatically set `cmd` appropriately, and they contain various
-constituent playbooks which perform different parts of the deployment. An
-individual section of Tenks can be run separately by substituting
-`ansible/deploy.yml` in the command above with the path to the playbook(s) you
-want to run. The current playbooks can be seen in the Ansible structure diagram
-in the *Development* section. Bear in mind that you will have to set `cmd` in
-your override file if you are running any of the sub-playbooks individually.
-
-Once a cluster has been deployed, it can be reconfigured by modifying the Tenks
-configuration and rerunning `deploy.yml`. Node specs can be changed (including
-increasing/decreasing the number of nodes); node types can also be
-reconfigured. Existing nodes will be preserved where possible.
-
-## Limitations
-
-The following is a non-exhaustive list of current known limitations of Tenks:
-
-* When using the Libvirt provider (currently the only provider), Tenks
-  hypervisors cannot co-exist with a containerised Libvirt daemon (for example,
-  as deployed by Kolla in the nova-libvirt container). Tenks will configure an
-  uncontainerised Libvirt daemon instance on the hypervisor, and this may
-  conflict with an existing containerised daemon. A workaround is to disable
-  the Nova virtualised compute service on each Tenks hypervisor if it is
-  present (for example, `docker stop nova_libvirt`) before running Tenks.
-
-## Development
-
-A diagram representing the Ansible structure of Tenks can be seen below. Blue
-rectangles represent playbooks, green rounded rectangles represent task books,
-red ellipses represent roles and yellow rhombi represent action plugins.
-
-<!---
-This diagram will need to be updated when the Ansible structure changes. The
-original draw.io diagram can be found below. The link below contains the
-diagram's XML which can be imported into draw.io and edited, then a new PNG
-artifact can be produced.
-
-https://drive.google.com/file/d/1MlmaTvJ2BPkhrOCLin4GPH265JDJqD1E/view?usp=sharing
--->
-
-![Tenks Ansible structure](assets/tenks_ansible_structure.png)

README.rst

@@ -0,0 +1,8 @@
+Tenks
+=====
+
+Tenks is a utility that manages virtual bare metal clusters for development and
+testing purposes.
+
+Refer to the documentation in ``doc/`` for installation instructions,
+configuration guidance and other information.

doc/requirements.txt

@@ -0,0 +1 @@
+sphinx>=1.8.0 # BSD

doc/source/architecture.rst

@@ -0,0 +1,49 @@
+.. _architecture:
+
+Architecture
+============
+
+Ansible
+-------
+
+A diagram representing the Ansible structure of Tenks can be seen below. Blue
+rectangles represent playbooks, green rounded rectangles represent task books,
+red ellipses represent roles and yellow rhombi represent action plugins.
+
+.. This diagram will need to be updated when the Ansible structure changes. The
+   original draw.io diagram can be found below. The link below contains the
+   diagram's XML which can be imported into draw.io and edited, then a new PNG
+   artifact can be produced.
+   https://drive.google.com/file/d/1MlmaTvJ2BPkhrOCLin4GPH265JDJqD1E/view?usp=sharing
+
+.. image:: _static/tenks_ansible_structure.png
+   :target: _static/tenks_ansible_structure.png
+
+Networking
+----------
+
+Tenks has a concept of 'physical network' which currently must map one-to-one
+to the hardware networks plugged into the hypervisors. It requires device
+mappings to be specified on a hypervisor for each physical network that is to
+be connected to nodes on that hypervisor. This device can be an interface, a
+Linux bridge or an Open vSwitch bridge. For each physical network that is give
+a mapping on a hypervisor, a new Tenks-managed Open vSwitch bridge is created.
+If the device mapped to this physnet is an interface, it is plugged directly
+into the new bridge. If the device is an existing Linux bridge, a veth pair is
+created to connect the existing bridge to the new bridge. If the device is an
+existing Open vSwitch bridge, an Open vSwitch patch port is created to link th
+two bridges.
+
+A new veth pair is created for each physical network that each node on each
+hypervisor is connected to, and one end of the pair is plugged into the Tenks
+Open vSwitch bridge for that physical network; the other end will be plugged
+into the node itself.  Creation of these veth pairs is necessary (at least for
+the Libvirt provider) to ensure that an interface is present in Open vSwitch
+even when the node itself is powered off.
+
+An example of the networking structure of Tenks is shown below. In this
+example, one node was requested to be connected to physnet0 and physnet1, and
+two nodes were requested to be connected just to physnet1.
+
+.. image:: _static/tenks_networking_structure.png
+   :target: _static/tenks_networking_structure.png

doc/source/conf.py

@@ -0,0 +1,173 @@
+# -*- coding: utf-8 -*-
+#
+# Configuration file for the Sphinx documentation builder.
+#
+# This file does only contain a selection of the most common options. For a
+# full list see the documentation:
+# http://www.sphinx-doc.org/en/master/config
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = u'Tenks'
+copyright = u'2018, Will Miller'
+author = u'Will Miller'
+
+# The short X.Y version
+version = u''
+# The full version, including alpha/beta/rc tags
+release = u''
+
+
+# -- General configuration ---------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The master toctree document.
+master_doc = 'index'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = None
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'alabaster'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#
+# html_theme_options = {}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Custom sidebar templates, must be a dictionary that maps document names
+# to template names.
+#
+# The default sidebars (for documents that don't match any pattern) are
+# defined by theme itself.  Builtin themes are using these templates by
+# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
+# 'searchbox.html']``.
+#
+# html_sidebars = {}
+
+
+# -- Options for HTMLHelp output ---------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'Tenksdoc'
+
+
+# -- Options for LaTeX output ------------------------------------------------
+
+latex_elements = {
+    # The paper size ('letterpaper' or 'a4paper').
+    #
+    # 'papersize': 'letterpaper',
+
+    # The font size ('10pt', '11pt' or '12pt').
+    #
+    # 'pointsize': '10pt',
+
+    # Additional stuff for the LaTeX preamble.
+    #
+    # 'preamble': '',
+
+    # Latex figure (float) alignment
+    #
+    # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (master_doc, 'Tenks.tex', u'Tenks Documentation',
+     u'Will Miller', 'manual'),
+]
+
+
+# -- Options for manual page output ------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'tenks', u'Tenks Documentation',
+     [author], 1)
+]
+
+
+# -- Options for Texinfo output ----------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, 'Tenks', u'Tenks Documentation',
+     author, 'Tenks', 'One line description of project.',
+     'Miscellaneous'),
+]
+
+
+# -- Options for Epub output -------------------------------------------------
+
+# Bibliographic Dublin Core info.
+epub_title = project
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#
+# epub_identifier = ''
+
+# A unique identification for the text.
+#
+# epub_uid = ''
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']

doc/source/configuration.rst

@@ -0,0 +1,50 @@
+.. _configuration:
+
+Configuration
+=============
+
+Hosts
+-----
+
+Tenks uses Ansible inventory to manage hosts. A multi-host setup is therefore
+supported, although the default hosts configuration will deploy an all-in-one
+setup on the host where the ``ansible-playbook`` command is executed
+(*localhost*).
+
+* Configuration management of the Tenks cluster is always performed on
+  *localhost*.
+* The ``hypervisors`` group should not directly contain any hosts. Its sub-groups
+  must contain one or more system. Systems in its sub-groups will host a subset
+  of the nodes deployed by Tenks.
+
+    * The ``libvirt`` group is a sub-group of ``hypervisors``. Systems in this
+      group will act as hypervisors using the Libvirt provider.
+
+Variable Configuration
+----------------------
+
+An override file should be created to configure Tenks. Any variables specified
+in this file will take precedence over their default settings in Tenks. This
+will allow you to set options as necessary for your setup, without needing to
+directly modify Tenks' variable files. An example override file can be found
+in ``ansible/override.yml.example``.
+
+Most of the configuration you will need to do relates to variables defined in
+``ansible/host_vars/localhost``. You can set your own values for these in your
+override file (mentioned above). In addition to other options, you will need to
+define the types of node you'd like to be able to manage as a dict in
+``node_types``, as well as the desired deployment specifications in ``specs``.
+Format and guidance for available options will be found within the variable
+file.
+
+Broadly, most variables in ``ansible/group_vars/*`` have sensible defaults which
+may be left as-is unless you have a particular need to configure them. A
+notable exception to this is the variable ``physnet_mappings`` in
+``ansible/group_vars/hypervisors``, which should map physical network names to
+the device to use for that network: this can be a network interface, or an
+existing OVS or Linux bridge. If these mappings are the same for all hosts in
+your ``hypervisors`` group, you may set a single dict ``physnet_mappings`` in your
+overrides file, and this will be used for all hosts. If different mappings are
+required for different hosts, you will need to individually specify them: for a
+host with hostname *myhost*, set ``physnet_mappings`` within the file
+``ansible/host_vars/myhost``.

doc/source/development.rst

@@ -0,0 +1,10 @@
+Development
+===========
+
+Contribution
+------------
+
+Contribution to Tenks' development is welcomed. Currently, the correct workflow
+is to raise an issue in the `GitHub repository
+<https://github.com/stackhpc/tenks/>`_. Patches should be submitted for review
+using a GitHub pull request.

doc/source/index.rst

@@ -0,0 +1,20 @@
+.. Tenks documentation master file, created by
+   sphinx-quickstart on Thu Sep 20 15:43:21 2018.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to Tenks's documentation!
+=================================
+
+Tenks is a utility that manages virtual bare metal clusters for development and
+testing purposes.
+
+.. toctree::
+   :maxdepth: 2
+
+   architecture
+   configuration
+   development
+   install
+   limitations
+   run

doc/source/install.rst

@@ -0,0 +1,8 @@
+Installation
+===============
+
+Tenks has dependencies on Ansible roles that are hosted by Ansible Galaxy.
+Given that your virtualenv of choice is active and Ansible (>=2.6) is
+installed inside it, Tenks' role dependencies can be installed by
+``ansible-galaxy install --role-file=requirements.yml
+--roles-path=ansible/roles/``.

doc/source/limitations.rst

@@ -0,0 +1,12 @@
+Limitations
+===========
+
+The following is a non-exhaustive list of current known limitations of Tenks:
+
+* When using the Libvirt provider (currently the only provider), Tenks
+  hypervisors cannot co-exist with a containerised Libvirt daemon (for example,
+  as deployed by Kolla in the nova-libvirt container). Tenks will configure an
+  uncontainerised Libvirt daemon instance on the hypervisor, and this may
+  conflict with an existing containerised daemon. A workaround is to disable
+  the Nova virtualised compute service on each Tenks hypervisor if it is
+  present (for example, ``docker stop nova_libvirt``) before running Tenks.

doc/source/run.rst

@@ -0,0 +1,38 @@
+Running Tenks
+=============
+
+Commands
+--------
+
+Tenks has a variable ``cmd`` which specifies the command to be run. This
+variable can be set in your override file (see :ref:`configuration`). The
+possible values it can take are:
+
+* ``deploy``: create a virtual cluster to the specification given. This is the
+  default command.
+* ``teardown``: tear down any existing virtual cluster with the specification
+  given.
+
+Execution
+---------
+
+Currently, Tenks does not have a CLI or wrapper. Deployment can be run by
+calling ``ansible-playbook --inventory ansible/inventory ansible/deploy.yml
+--extra-vars=@override.yml``, where ``override.yml`` is the path to your
+override file.
+
+The ``deploy.yml`` playbook will run deployment from start to finish;
+``teardown.yml`` is ``deploy.yml``'s "mirror image" to tear down a cluster.
+These playbooks automatically set ``cmd`` appropriately, and they contain
+various constituent playbooks which perform different parts of the deployment.
+An individual section of Tenks can be run separately by substituting
+``ansible/deploy.yml`` in the command above with the path to the playbook(s)
+you want to run. The current playbooks can be seen in the Ansible structure
+diagram in :ref:`architecture`. Bear in mind that you will have to set
+``cmd`` in your override file if you are running any of the sub-playbooks
+individually.
+
+Once a cluster has been deployed, it can be reconfigured by modifying the Tenks
+configuration and rerunning ``deploy.yml``. Node specs can be changed
+(including increasing/decreasing the number of nodes); node types can also be
+reconfigured. Existing nodes will be preserved where possible.

setup.cfg

@@ -2,7 +2,7 @@
 name = tenks
 summary = Deployment of virtual bare metal clusters with Tenks
 description-file =
-    README.md
+    README.rst
 author = Will Miller
 author-email = willm@stackhpc.com
 home-page = https://stackhpc.com

tox.ini

@@ -29,6 +29,14 @@ basepython = python2.7
 commands =
   flake8 {posargs}
 
+[testenv:docs]
+basepython = python3
+deps =
+    -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=stable/rocky}
+    -r{toxinidir}/requirements.txt
+    -r{toxinidir}/doc/requirements.txt
+commands = sphinx-build -W -b html doc/source doc/build/html
+
 [testenv:cover]
 basepython = python3
 setenv =