diff --git a/doc/contributor-guide/source/doc-tools.rst b/doc/contributor-guide/source/doc-tools.rst index f72cfb69dd..48c53a1d0b 100644 --- a/doc/contributor-guide/source/doc-tools.rst +++ b/doc/contributor-guide/source/doc-tools.rst @@ -17,6 +17,7 @@ location, explains how to install, configure, use, and contribute to them. :maxdepth: 2 doc-tools/scripts.rst + doc-tools/template-generator.rst doc-tools/installation.rst doc-tools/cli-reference.rst doc-tools/config-reference.rst diff --git a/doc/contributor-guide/source/doc-tools/scripts.rst b/doc/contributor-guide/source/doc-tools/scripts.rst index 3e8ac723b8..137a42009a 100644 --- a/doc/contributor-guide/source/doc-tools/scripts.rst +++ b/doc/contributor-guide/source/doc-tools/scripts.rst @@ -50,12 +50,7 @@ beneficial to consolidate these into the ``openstack-doc-tools`` repository. www-generator.py Generates static, template-based HTML files for - https://docs.openstack.org/. The script reads YAML data files in - ``www/project-data`` to determine which projects exist in a given - series and how they should be displayed on the list of installation, - configuration, and other guides. The file - ``www/project-data/schema.yaml`` contains the JSONSchema settings to - describe what a valid data file looks like. + https://docs.openstack.org/. See :doc:`template-generator` for details. sync-projects.sh Synchronizes the **Glossary**, common files, and some translations diff --git a/doc/contributor-guide/source/doc-tools/template-generator.rst b/doc/contributor-guide/source/doc-tools/template-generator.rst new file mode 100644 index 0000000000..a75dde1f96 --- /dev/null +++ b/doc/contributor-guide/source/doc-tools/template-generator.rst @@ -0,0 +1,265 @@ +.. _template-generator: + +============================ + Template generator details +============================ + +The content on docs.openstack.org that is not managed as document sets +built by Sphinx is handled via a custom template rendering tool in +``tools/www-generator.py`` within the openstack-manuals git +repository. + +The script reads YAML data files in ``www/project-data`` to determine +which projects exist in a given series and how they should be +displayed on the list of installation, configuration, and other +guides. + +After the script loads the data about the external projects, it scans +``www/`` for HTML template files. It uses Jinja2_ to convert the +templates to complete HTML pages, which it writes to the +``publish-docs`` output directory. + +.. seealso:: + + The Jinja2_ documentation includes a guide for template designers + that covers the syntax of templates, inheritance between templates, + and the various filters and other features available when writing + templates. + +.. _Jinja2: http://jinja.pocoo.org + +Defining release series +======================= + +The set of release series and their individual status and other +metadata is embedded in the template generator script in the +``SERIES_INFO`` data structure. The structure is a dictionary mapping +the name of the release series to a ``SeriesInfo`` structure holding +the metadata. + +For each release series, the generator needs to know: + +``date`` + The date value should be a string containing the month name and 4 + digit year. + +``status`` + The 'status' field should be one of: + + ``obsolete`` + the release existed, but we have no more artifacts for it + + ``EOL`` + the release is closed but we have docs for it + + ``maintained`` + the release still has an open branch + + ``development`` + the current release being developed + +.. seealso:: + + :ref:`release-www-page-updates` has some additional information + about how the status values are updated at the end of a release + cycle. + +Project data file format +======================== + +The projects associated with each release series are listed in a +separate YAML file in the ``www/project-data`` directory. Each file is +named for the series (``austin.yaml``, ``bexar.yaml``, etc.) except +for the series currently under development which is always kept in +``latest.yaml``. + +The schema for the project data files is defined in +``www/project-data/schema.yaml``. + +Each file should contain an array or list of entries. Each entry must +define the name, service, and type properties. + +The ``name`` should be the base name of a git repository. + +The ``service`` string should be taken from the governance repository +definition of the project. + +The ``type`` must be one of the values listed below: + +``service`` + A REST API service. + +``client`` + A library for talking to a service. + +``library`` + Another type of library. + +``tool`` + A command line tool or other project that is used with, or used to + build, OpenStack. + +``networking`` + A plugin for the networking service. + +``baremetal`` + A subproject for the bare metal project, Ironic. + +``deployment`` + A tool for deploying OpenStack. + +``other`` + A project that does run in a cloud but does not provide + a REST API. + +An entry can also optionally define ``service_type``, which must match +the value associated with the name in the `service-types-authority +repository +`_. + +Entries with ``type`` set to ``client`` should include a ``description`` +field with a short description, such as "keystone client". + +Entries may optionally set flags to indicate that the repository +includes particular types of documentation in an expected location, to +include a link to that documentation on the templated landing pages. + +``has_install_guide`` + produces a link to docs.o.o/name/latest/install/ + +``has_api_guide`` + produces a link to developer.o.o/api-guide/service_type/ + +``has_api_ref`` + produces a link to developer.o.o/api-ref/service_type/ + +``has_config_ref`` + produces a link to docs.o.o/name/latest/configuration/ + +``has_in_tree_api_docs`` + produces a link to docs.o.o/name/latest/api/ + +``has_admin_guide`` + produces a link to docs.o.o/name/latest/admin/ + +.. note:: + + The documentation associated with the flags must exist before the + flags are set. + +Template variables +================== + +The template generator uses the input data to set several variables +visible within the template. This allows us to reuse the same template +to generate content for multiple pages of the same style, filling in +different data. + +By convention, all of the variables defined in the template generator +use all uppercase names. This makes it easy to differentiate the +generator variables from variables defined within templates (such as +loop contexts). + +``TEMPLATE_FILE`` + The name of the template file being rendered, with the ``www`` + prefix removed. For example, ``pike/index.html``. + +``PROJECT_DATA`` + All of the project data loaded from the data files in a dictionary + mapping the series name to the parsed data file. Most template pages + will assign a local variable using ``PROJECT_DATA[SERIES]`` to + extract the correct subset of the data. + +``TOPDIR`` + The relative path to the top of the build output. This is useful for + building paths between output pages in a way that allows those pages + to move around later. + +``SCRIPTDIR`` + The relative path to the location of the JavaScript directory in the + build output. This is useful for building links to JavaScript files. + +``CSSDIR`` + The relative path to the location of the directory containing the + CSS files in the build output. This is useful for building links to + CSS files. + +``IMAGEDIR`` + The relative path to the location of the directory containing image + files in the build output. This is useful for building links to + images. + +``SERIES`` + A string containing the name of the series usable in URLs. For the + series under development, this is ``"latest"``. For other series, it + is the series name in lower case. + + This value is derived from the path to the template file. If the + file is under a directory that matches one of the known series + names, that value is used to set ``SERIES``. + +``SERIES_TITLE`` + A string containing the name of the series usable in text visible to + the reader. It is always the actual name of the series in "title + case" (the first letter of each word is uppercase). For example, + ``"Pike"``. + + This value is derived from the path to the template file. If the + file is under a directory that matches one of the known series + names, that value is used to set ``SERIES``. + +``ALL_SERIES`` + A list of all of the series names for all OpenStack releases, in + order of release. + + This list is derived from the keys of the ``SERIES_INFO`` dictionary + defined in the template generator. + +``PAST_SERIES`` + A list of the series names for OpenStack releases with a status + other than ``"development"``. + + This list is derived from the values in the ``SERIES_INFO`` + dictionary defined in the template generator. + +``RELEASED_SERIES`` + A string containing the lowercase name of the most recent series to + be released. For example, during the Pike series this value was + ``"ocata"``. + + This value is derived from the values in the ``SERIES_INFO`` + dictionary defined in the template generator. + +``SERIES_IN_DEVELOPMENT`` + A string containing the lowercase name of the series under active + development. For example, during the Pike series this value was + ``"pike"``. + + This value is derived from the values in the ``SERIES_INFO`` + dictionary defined in the template generator. + +``SERIES_INFO`` + The ``SeriesInfo`` object associated with the current series. This + provides access to the ``date`` and ``status`` values for the + series. + + This value is taken from the ``SERIES_INFO`` dictionary defined in + the template generator. + +``REGULAR_REPOS`` + A list of all of the names of regular repositories for official + OpenStack projects. Here "regular" differentiates the repositories + from infrastructure team repositories, which have their + documentation published to a different location and therefore need + some different URLs for redirects in the ``.htaccess`` template. See + ``INFRA_REPOS``. + + This value is derived from data published from the governance + repository. + +``INFRA_REPOS`` + A list of all of the names of repositories for the infrastructure + team. See ``REGULAR_REPOS``. + + This value is derived from data published from the governance + repository. diff --git a/doc/contributor-guide/source/landing-pages.rst b/doc/contributor-guide/source/landing-pages.rst index 551de83192..d17f66540d 100644 --- a/doc/contributor-guide/source/landing-pages.rst +++ b/doc/contributor-guide/source/landing-pages.rst @@ -11,4 +11,4 @@ To add your project content to these guides, change the file `_ in the openstack-manuals repository. -.. include:: ../../../www/project-data/README.rst +See :ref:`template-generator` for details. diff --git a/doc/contributor-guide/source/release/taskdetail.rst b/doc/contributor-guide/source/release/taskdetail.rst index 384bcee6f5..062dc373f6 100644 --- a/doc/contributor-guide/source/release/taskdetail.rst +++ b/doc/contributor-guide/source/release/taskdetail.rst @@ -62,6 +62,8 @@ notes is `openstack-manuals/releasenotes/source/RELEASENAME` and they are published to `https://docs.openstack.org/releasenotes/openstack-manuals/RELEASENAME.html`. +.. _release-www-page-updates: + Update www pages for end of release ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -81,58 +83,45 @@ Make the following changes in the **openstack-manuals** repository: $ cp -a www/RELEASE www/NEXT_SERIES -#. Update the ``PAST_SERIES``, ``RELEASED_SERIES``, - ``SERIES_IN_DEVELOPMENT``, and ``FUTURE_SERIES`` values at the top - of the source file for the template generator - (``tools/www-generator.py``). +#. Update the ``SERIES_INFO`` data structure at the top of the source + file for the template generator (``tools/www-generator.py``). See + :ref:`template-generator` for details about the structure. - * Add the existing value of the ``RELEASED_SERIES`` variable to the - ``PAST_SERIES`` list. - * Set the ``RELEASED_SERIES`` variable to the name of the series - being released. - * Set the ``SERIES_IN_DEVELOPMENT`` variable to the name of the - next series. - * Add any additional known names to the ``FUTURE_SERIES`` list. + * Change the series with status ``'development'`` to have status + ``'maintained'``. + * Add a new entry for the new series, giving the estimated release + date and setting the status to ``'development'``. For example, at the end of the Pike cycle, the variables will contain: .. code-block:: python - PAST_SERIES = [ - 'kilo', - 'liberty', - 'mitaka', - 'newton', - ] - RELEASED_SERIES = 'ocata' - SERIES_IN_DEVELOPMENT = 'pike' - FUTURE_SERIES = [ - 'queens', - ] + SERIES_INFO = { + 'austin': SeriesInfo(date='October 2010', status='obsolete'), + # ... + 'mitaka': SeriesInfo(date='April 2016', status='EOL'), + 'newton': SeriesInfo(date='October 2016', status='maintained'), + 'ocata': SeriesInfo(date='February 2017', status='maintained'), + 'pike': SeriesInfo(date='August 2017', status='development'), + } To update the settings: - * ``'ocata'`` is added to ``PAST_SERIES`` - * ``RELEASED_SERIES`` changes from ``'ocata'`` to ``'pike'`` - * ``SERIES_IN_DEVELOPMENT`` becomes ``'queens'`` - * ``'queens'`` is removed from the ``FUTURE_SERIES`` list - * ``'rocky'`` is added to the ``FUTURE_SERIES`` list + * the status for pike is changed to ``'maintained'`` + * a new entry for queens is added .. code-block:: python - PAST_SERIES = [ - 'kilo', - 'liberty', - 'mitaka', - 'newton', - 'ocata', - ] - RELEASED_SERIES = 'pike' - SERIES_IN_DEVELOPMENT = 'queens' - FUTURE_SERIES = [ - 'rocky', - ] + SERIES_INFO = { + 'austin': SeriesInfo(date='October 2010', status='obsolete'), + # ... + 'mitaka': SeriesInfo(date='April 2016', status='EOL'), + 'newton': SeriesInfo(date='October 2016', status='maintained'), + 'ocata': SeriesInfo(date='February 2017', status='maintained'), + 'pike': SeriesInfo(date='August 2017', status='maintained'), + 'queens': SeriesInfo(date='August 2017', status='development'), + } This will cause docs.openstack.org to redirect to the series-specific landing page for the current release, and the diff --git a/www/project-data/README.rst b/www/project-data/README.rst index fe5e8b1da5..719a7f5fca 100644 --- a/www/project-data/README.rst +++ b/www/project-data/README.rst @@ -1,77 +1,3 @@ -Project Data -============ - -The projects associated with each release series are listed in a -separate YAML file, all following the same schema defined in -``www/project-data/schema.yaml``. - -The file should contain an array or list of entries. Each entry must -define the name, service, and type properties. - -The ``name`` should be the base name of a git repository. - -The ``service`` string should be taken from the governance repository -definition of the project. - -The ``type`` must be one of the values listed below: - -``service`` - A REST API service. - -``client`` - A library for talking to a service. - -``library`` - Another type of library. - -``tool`` - A command line tool or other project that is used with, or used to - build, OpenStack. - -``networking`` - A plugin for the networking service. - -``baremetal`` - A subproject for the bare metal project, Ironic. - -``deployment`` - A tool for deploying OpenStack. - -``other`` - A project that does run in a cloud but does not provide - a REST API. - -An entry can also optionally define ``service_type``, which must match -the value associated with the name in the `service-types-authority -repository -`_. - -Entries with ``type`` set to ``client`` should include a ``description`` -field with a short description, such as "keystone client". - -Entries may optionally set flags to indicate that the repository -includes particular types of documentation in an expected location, to -include a link to that documentation on the templated landing pages. - -``has_install_guide`` - produces a link to docs.o.o/name/latest/install/ - -``has_api_guide`` - produces a link to developer.o.o/api-guide/service_type/ - -``has_api_ref`` - produces a link to developer.o.o/api-ref/service_type/ - -``has_config_ref`` - produces a link to docs.o.o/name/latest/configuration/ - -``has_in_tree_api_docs`` - produces a link to docs.o.o/name/latest/api/ - -``has_admin_guide`` - produces a link to docs.o.o/name/latest/admin/ - -.. note:: - - The documentation associated with the flags must exist before the - flags are set. +See +https://docs.openstack.org/contributor-guide/doc-tools/template-generator.html +for details.