Added JavaScript to Common Testing Interface
This patch expands the CTI documentation to support multiple languages. It creates a single page that describes overall CTI goals, and then links to documents that go into specifics on each language supported in OpenStack. The Javascript section is new, and includes documentation for the current state of the Javascript CTI. It also explicitly calls out areas where work needs to be done, in order to encourage discussion and contributions. Change-Id: I971b9271443e4fd227b4a6b347dfc385b67eb3e4 Depends-On: I00186c9af441c1778a22379634cdb2f918324bd8
This commit is contained in:
Michael Krotscheck
148
reference/cti/javascript-cti.rst
Normal file
148
reference/cti/javascript-cti.rst
Normal file
@@ -0,0 +1,148 @@
|
||||
========================================
|
||||
Consistent Testing Interface: JavaScript
|
||||
========================================
|
||||
|
||||
This document outlines common ways to meet the Consistent Testing Interface
|
||||
requirements for JavaScript. Each JavaScript project must be able to do:
|
||||
|
||||
- Codestyle checks.
|
||||
- Execute Tests and Code Coverage
|
||||
- Package Tarball Generation
|
||||
- Documentation Generation
|
||||
- Validate dependency licenses
|
||||
|
||||
Projects which are browser based must also be able to do:
|
||||
|
||||
- Unit tests in Firefox and Chromium.
|
||||
|
||||
Projects which are server based must also be able to do:
|
||||
|
||||
- Unit tests in Node.js.
|
||||
|
||||
Projects which require translation must also be able to do:
|
||||
|
||||
- Translation import/export and merge for translated objects.
|
||||
|
||||
Specific commands
|
||||
-----------------
|
||||
|
||||
The following commands must be supported at the root of a clean tree, in
|
||||
order to initialize your project.
|
||||
|
||||
:code:`npm install`
|
||||
This command installs all of the project's dependencies.
|
||||
|
||||
To drive the above required steps, the following commands should be
|
||||
supported at the root of an initialized tree.
|
||||
|
||||
:code:`npm test`
|
||||
This command executes all available test suites, and generate
|
||||
appropriate code coverage reports.
|
||||
:code:`npm run lint`
|
||||
This command performs codestyle checks against the project.
|
||||
:code:`npm pack`
|
||||
This command generates a release tarball.
|
||||
:code:`npm publish <tarball> --no-scripts`
|
||||
This command will publish a release tarball to npm. It may not be
|
||||
necessary for all projects.
|
||||
:code:`npm run document`
|
||||
This command builds documentation for the project.
|
||||
|
||||
The following commands are still under discussion:
|
||||
|
||||
:code:`npm run license`
|
||||
This command ensures that no incompatible licenses have accidentally been
|
||||
included.
|
||||
:code:`npm run translate`
|
||||
This command imports translations into this project, if necessary.
|
||||
|
||||
|
||||
Project Setup
|
||||
-------------
|
||||
|
||||
node and npm version
|
||||
====================
|
||||
We support the current version of node.js and npm available in the LTS
|
||||
releases of Ubuntu. As of this writing, these are Node v0.10.29 and
|
||||
npm v1.4.21. While these versions are no longer supported, this restriction is
|
||||
imposed by our package maintainers.
|
||||
|
||||
npm scripts
|
||||
===========
|
||||
All JavaScript specific testing tools are invoked via `NPM package scripts`_.
|
||||
These are useful because they provide a 'virtual' runtime environment
|
||||
whose dependencies are contained entirely in the project directory. They also
|
||||
allow us to create a consistent interface between the commands that are
|
||||
invoked by our build, and the tools required by the project.
|
||||
|
||||
Requirements Listing
|
||||
====================
|
||||
Each project should list its runtime, peer, and development dependencies
|
||||
in package.json and (if applicable) bower.json.
|
||||
|
||||
:code:`dependencies`
|
||||
Packages required by your project to run in production. These should
|
||||
never use fuzzy version matching.
|
||||
:code:`devDependencies`
|
||||
Packages that are required by your project during the test and build
|
||||
phase. These should never use fuzzy version matching.
|
||||
:code:`peerDependencies`
|
||||
Packages that are used to run your project, but whose version does not
|
||||
strictly matter. For example, eslint-config-openstack has eslint as a
|
||||
peer dependency.
|
||||
|
||||
Virtual Environment Management
|
||||
==============================
|
||||
|
||||
To support sensible testing, we use npm's environment management, as it
|
||||
permits the installation of dependencies by project.
|
||||
|
||||
Build Step Details
|
||||
------------------
|
||||
The following describes each individual command, what it should do, and its
|
||||
expected output.
|
||||
|
||||
Codestyle Checks
|
||||
================
|
||||
:Command: :code:`npm run lint`
|
||||
|
||||
OpenStack requires the custom npm script 'lint' to execute our codestyle
|
||||
checks. The tool we use is called `ESLint`_, and our rules are published to npm
|
||||
as eslint-config-openstack_.
|
||||
|
||||
Executing Tests and Code Coverage
|
||||
=================================
|
||||
:Command: :code:`npm test`
|
||||
|
||||
OpenStack requires a sane testing and code coverage strategy for each
|
||||
project, though we do not prescribe the tools and coverage threshold, as
|
||||
these may differ based on circumstance and project type. Generated test
|
||||
reports should be placed in :code:`./reports` in your projects' root directory.
|
||||
Generated coverage output should similarly be placed in :code:`./cover`.
|
||||
|
||||
Package Tarball Generation
|
||||
==========================
|
||||
:Command: :code:`npm pack`
|
||||
|
||||
OpenStack uses :code:`npm pack` to generate a release tarball, which will
|
||||
compile all files listed in :code:`package.json`. If your project requires
|
||||
concatenation, minification, or any other preprocessing to create a valid
|
||||
tarball, you may use the npm :code:`prepublish` hook to trigger these steps.
|
||||
|
||||
All packages should include:
|
||||
|
||||
- A README
|
||||
- A LICENSE file
|
||||
- All source code
|
||||
|
||||
Generate Documentation
|
||||
======================
|
||||
:Command: :code:`npm run document`
|
||||
|
||||
In order to reuse existing templates, styles, and tooling, OpenStack uses
|
||||
Sphinx to generate our JavaScript Project documentation. All documentation
|
||||
output should be placed in the :code:`publish-docs` directory.
|
||||
|
||||
.. _NPM package scripts: https://docs.npmjs.com/misc/scripts
|
||||
.. _ESLint: http://eslint.org
|
||||
.. _eslint-config-openstack: http://git.openstack.org/cgit/openstack/eslint-config-openstack
|
||||
115
reference/cti/python_cti.rst
Normal file
115
reference/cti/python_cti.rst
Normal file
@@ -0,0 +1,115 @@
|
||||
====================================
|
||||
Consistent Testing Interface: Python
|
||||
====================================
|
||||
|
||||
Each python project must be able to do:
|
||||
|
||||
- Unit tests for python2.7
|
||||
- Codestyle checks
|
||||
- Testing Coverage Report
|
||||
- Source Tarball Generation
|
||||
- Translations import/export and merge for translated projects
|
||||
- Documentation generation
|
||||
|
||||
Projects which are compatible with Python 3 must also be able to do:
|
||||
|
||||
- Unit tests for python3.4
|
||||
|
||||
Projects may optionally also support:
|
||||
|
||||
- Constrained unit tests for python2.7
|
||||
- Constrained unit tests for python3.4
|
||||
|
||||
Specific commands
|
||||
-----------------
|
||||
|
||||
To drive the above tasks, the following commands should be supported in a clean tree:
|
||||
|
||||
- tox -epy27
|
||||
- tox -epep8
|
||||
- tox -ecover
|
||||
- tox -evenv python setup.py sdist
|
||||
- tox -evenv python setup.py build_sphinx
|
||||
|
||||
Projects that are translated should also support:
|
||||
|
||||
- tox -evenv python setup.py extract_messages
|
||||
- tox -evenv python setup.py update_catalog
|
||||
|
||||
Projects which are compatible with Python 3 must also be able to do:
|
||||
|
||||
- tox -epy34
|
||||
|
||||
Projects desiring to support constrained unit tests must support:
|
||||
|
||||
- tox -epy27-constraints
|
||||
- tox -epy34-constraints
|
||||
|
||||
Requirements Listing
|
||||
--------------------
|
||||
|
||||
Each project should list its operations dependencies in requirements.txt
|
||||
and additional dependencies required for testing in test-requirements.txt.
|
||||
If there are requirements that are specific to python3 or pypy support,
|
||||
those may be listed in requirements.txt or test-requirements.txt using
|
||||
environment makers.
|
||||
|
||||
Virtual Environment Management
|
||||
------------------------------
|
||||
|
||||
To support sensible testing across multiple python versions, we use tox
|
||||
config files in the projects.
|
||||
|
||||
unittest running
|
||||
----------------
|
||||
|
||||
OpenStack uses testrepository as its test runner, which supports a number
|
||||
of things, most importantly to the expanded project is the subunit output
|
||||
stream collection. This is useful for aggregating and displaying test output.
|
||||
In support of that, the oslotest library is built on top of testtools,
|
||||
testscenarios and fixtures.
|
||||
|
||||
Constraints
|
||||
===========
|
||||
|
||||
The requirements project maintains a set of constraints with packages pinned
|
||||
to specific package versions that are known to be working. The goal is to
|
||||
ease the diagnosis of breakage caused by projects upstream to OpenStack and
|
||||
to provide a set of packages known to work together. Running constraints
|
||||
enabled jobs in addition to the main tests is optional and left up to
|
||||
individual projects to decide.
|
||||
|
||||
Project Configuration
|
||||
---------------------
|
||||
|
||||
All OpenStack projects use `pbr` for consistent operation of setuptools.
|
||||
To accomplish this, all setup.py files only contain a simple setup function
|
||||
that setup_requires on an unversioned pbr, and a directive to pass processing
|
||||
to the pbr library. Actual project configuration is then handled in setup.cfg.
|
||||
|
||||
Generated Files
|
||||
---------------
|
||||
|
||||
ChangeLog and AUTHORS files are generated at setup.py sdist time. This is
|
||||
handled by pbr.
|
||||
|
||||
.mailmap files should exist where a developer has more than one email address
|
||||
or identity, and should map to the developer's canonical identity.
|
||||
|
||||
Translations
|
||||
------------
|
||||
|
||||
To support translations processing, projects should have a valid babel config.
|
||||
There should be a locale package inside of the top project module, and in that
|
||||
dir should be the $project.pot file. For instance, for nova, there should be
|
||||
nova/locale/nova.pot. Babel commands should be configured out output their .mo
|
||||
files in to $project/locale as well.
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
Developer docs are generated from Sphinx sources in the tree. Additionally,
|
||||
there are end user docs and API docs which are maintained outside of the
|
||||
context of a project's repo. To support documentation generation, projects
|
||||
should have sphinx documentation source in doc/source and build_sphinx should
|
||||
output the documentation to doc/build.
|
||||
@@ -11,117 +11,28 @@ infrastructure has to be able to pre-cache artifacts that are normally fetched
|
||||
over the internet. To that end, each project should support a consistent
|
||||
interface for driving tests and other necessary tasks.
|
||||
|
||||
End results needed
|
||||
------------------
|
||||
The following tasks are required for every project. Every project must:
|
||||
|
||||
Each python project must be able to do:
|
||||
- Execute tests
|
||||
- Enforce code style
|
||||
- Generate a code coverage report
|
||||
- Generate a source tarball
|
||||
- Generate documentation
|
||||
|
||||
- Unit tests for python2.7
|
||||
- Codestyle checks
|
||||
- Testing Coverage Report
|
||||
- Source Tarball Generation
|
||||
- Translations import/export and merge for translated projects
|
||||
- Documentation generation
|
||||
The following are other common tasks, which may not be relevant for every
|
||||
project:
|
||||
|
||||
Projects which are compatible with Python 3 must also be able to do:
|
||||
- Enforce code coverage
|
||||
- Generate a release artifact
|
||||
- Publish a release artifact
|
||||
- Import translation strings
|
||||
- Export translation strings
|
||||
|
||||
- Unit tests for python3.4
|
||||
Tools and approaches vary by language, please choose which language is
|
||||
relevant to you.
|
||||
|
||||
Projects may optionally also support:
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
- Constrained unit tests for python2.7
|
||||
- Constrained unit tests for python3.4
|
||||
|
||||
Specific commands
|
||||
-----------------
|
||||
|
||||
To drive the above tasks, the following commands should be supported in a clean tree:
|
||||
|
||||
- tox -epy27
|
||||
- tox -epep8
|
||||
- tox -ecover
|
||||
- tox -evenv python setup.py sdist
|
||||
- tox -evenv python setup.py build_sphinx
|
||||
|
||||
Projects that are translated should also support:
|
||||
|
||||
- tox -evenv python setup.py extract_messages
|
||||
- tox -evenv python setup.py update_catalog
|
||||
|
||||
Projects which are compatible with Python 3 must also be able to do:
|
||||
|
||||
- tox -epy34
|
||||
|
||||
Projects desiring to support constrained unit tests must support:
|
||||
|
||||
- tox -epy27-constraints
|
||||
- tox -epy34-constraints
|
||||
|
||||
Requirements Listing
|
||||
--------------------
|
||||
|
||||
Each project should list its operations dependencies in requirements.txt
|
||||
and additional dependencies required for testing in test-requirements.txt.
|
||||
If there are requirements that are specific to python3 or pypy support,
|
||||
those may be listed in requirements.txt or test-requirements.txt using
|
||||
environment makers.
|
||||
|
||||
Virtual Environment Management
|
||||
------------------------------
|
||||
|
||||
To support sensible testing across multiple python versions, we use tox
|
||||
config files in the projects.
|
||||
|
||||
unittest running
|
||||
----------------
|
||||
|
||||
OpenStack uses testrepository as its test runner, which supports a number
|
||||
of things, most importantly to the expanded project is the subunit output
|
||||
stream collection. This is useful for aggregating and displaying test output.
|
||||
In support of that, the oslotest library is built on top of testtools,
|
||||
testscenarios and fixtures.
|
||||
|
||||
Constraints
|
||||
===========
|
||||
|
||||
The requirements project maintains a set of constraints with packages pinned
|
||||
to specific package versions that are known to be working. The goal is to
|
||||
ease the diagnosis of breakage caused by projects upstream to OpenStack and
|
||||
to provide a set of packages known to work together. Running constraints
|
||||
enabled jobs in addition to the main tests is optional and left up to
|
||||
individual projects to decide.
|
||||
|
||||
Project Configuration
|
||||
---------------------
|
||||
|
||||
All OpenStack projects use `pbr` for consistent operation of setuptools.
|
||||
To accomplish this, all setup.py files only contain a simple setup function
|
||||
that setup_requires on an unversioned pbr, and a directive to pass processing
|
||||
to the pbr library. Actual project configuration is then handled in setup.cfg.
|
||||
|
||||
Generated Files
|
||||
---------------
|
||||
|
||||
ChangeLog and AUTHORS files are generated at setup.py sdist time. This is
|
||||
handled by pbr.
|
||||
|
||||
.mailmap files should exist where a developer has more than one email address
|
||||
or identity, and should map to the developer's canonical identity.
|
||||
|
||||
Translations
|
||||
------------
|
||||
|
||||
To support translations processing, projects should have a valid babel config.
|
||||
There should be a locale package inside of the top project module, and in that
|
||||
dir should be the $project.pot file. For instance, for nova, there should be
|
||||
nova/locale/nova.pot. Babel commands should be configured out output their .mo
|
||||
files in to $project/locale as well.
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
Developer docs are generated from Sphinx sources in the tree. Additionally,
|
||||
there are end user docs and API docs which are maintained outside of the
|
||||
context of a project's repo. To support documentation generation, projects
|
||||
should have sphinx documentation source in doc/source and build_sphinx should
|
||||
output the documentation to doc/build.
|
||||
cti/*
|
||||
|
||||
Reference in New Issue
Block a user