openstack/governance
Code Issues Proposed changes
f51a0239b3
governance/reference/project-testing-interface.rst
Zane Bitter f06b84b3ec Get rid of popularity discussion in PTI 
Now that we no longer use 'popularity' as a criteria for including
distros in the PTI, there's no need to include a boring disclaimer about
the relative popularity of RHEL vs. CentOS.

Change-Id: I4c566ff4f9d7fa20eb755061af5dae01ac6e8edb
2019-02-25 16:24:31 -05:00

5.2 KiB
Raw Blame History

Project Testing Interface

OpenStack has a lot of projects. For each project, the OpenStack CI system needs to be able to perform a lot of tasks. If each project has a slightly different way to accomplish those tasks, it makes the management of a consistent testing infrastructure very difficult to deal with. Additionally, because of the high volume of development changes and testing, the testing infrastructure has to be able to pre-cache artifacts that are normally fetched over the internet. To that end, each project should support a consistent interface for driving tests and other necessary tasks.

The following tasks are required for every project. Every project must:

  • Execute tests
  • Enforce code style
  • Generate a code coverage report
  • Generate a source tarball
  • Generate documentation
  • Generate releasenotes

The following are other common tasks, which may not be relevant for every project:

  • Enforce code coverage
  • Generate a release artifact
  • Publish a release artifact
  • Import translation strings
  • Export translation strings

Tools and approaches vary by language, please choose which language is relevant to you.

Documentation

OpenStack has decided to standardize on using Sphinx for project documentation, regardless of programming language.

Note

The use of sphinx for documentation is intended for documentation that is not written inside of docstrings or code comments. Languages, such as Go, that natively support a system for documenting the code through code comments, should use those native systems. Sphinx is intended to be used for documentation that is not written inline with the code.

To support documentation generation, projects should:

  • Have sphinx documentation source in doc/source
  • List python dependencies needed for documentation in doc/requirements.txt
  • List distro package pre-reqs for dependencies in bindep.txt using the doc tag.
  • Depend on openstackdocstheme for documentation and configure it to be used in doc/source/conf.py.
  • Have a docs environment set up in a tox.ini file within the repository.

Assuming non-Python requirements have been properly installed as indicated by bindep.txt, the following command should work with no additional setup and should result in the documentation being emitted into doc/build/html.

tox -e docs

Note

We strongly discourage project teams from adding commands to the docs environment beyond:

sphinx-build -W -b html doc/source doc/build/html

Additional logic needed around Sphinx generation should go into Sphinx plugins.

Language specific instructions supplement these and are in addition to them.

Release Notes

OpenStack uses reno for generating release notes regardless of programming language.

To support releasenotes generation, projects should:

  • Have releasenotes documentation source in releasenotes/
  • Configure openstackdocstheme to be used in releasenotes/source/conf.py.
  • Optionally list distro package pre-reqs for dependencies in bindep.txt using the releasenotes tag.

Assuming requirements have been properly installed, the following command should work with no additional setup and should result in the releasenotes being emitted into releasenotes/build/html.

sphinx-build -a -E -W -d releasenotes/build/doctrees -b html \
    releasenotes/source releasenotes/build/html

Language specific instructions supplement these and are in addition to them.

Linux Distributions

The following free operating systems are representative of platforms regularly used to deploy OpenStack on:

Each project should run some functional tests on these platforms so we make sure OpenStack works with distros used in production. The scope of these functional tests are discussed for every project, and may adjust their coverage depending of resources and support investment. These tests are run by using existing tooling, which comes with a reasonable expectation that it's viable on the indicated distributions.

Sometimes, these distributions might not support all dependencies required by new features in OpenStack. Development of these features should not be blocked, though it has to be documented in project release notes, and some tests might have to be skipped on these distributions.

pti/*

Tested Runtimes

In order to focus development efforts and prevent breaking changes midway through a development cycle, the policy for officially tested runtimes is based on the LTS or stable release of the pti-linux-distros at the start of the development cycle.

The officially tested runtimes for each cycle can be found here:

runtimes/*