docs: Fix indent level
Leading spaces before item lists leads to vertical line on the left side. They are completely unnecessary. Change-Id: I08c3f077e470aa593076a525de1445bc5d0bdb9a
This commit is contained in:
Akihiro Motoki
parent
a23cd43abe
commit
eee9d2ca80
34
TESTING.rst
34
TESTING.rst
@ -50,17 +50,17 @@ We will talk about three classes of tests: unit, functional and integration.
|
|||||||
Each respective category typically targets a larger scope of code. Other than
|
Each respective category typically targets a larger scope of code. Other than
|
||||||
that broad categorization, here are a few more characteristic:
|
that broad categorization, here are a few more characteristic:
|
||||||
|
|
||||||
* Unit tests - Should be able to run on your laptop, directly following a
|
* Unit tests - Should be able to run on your laptop, directly following a
|
||||||
'git clone' of the project. The underlying system must not be mutated,
|
'git clone' of the project. The underlying system must not be mutated,
|
||||||
mocks can be used to achieve this. A unit test typically targets a function
|
mocks can be used to achieve this. A unit test typically targets a function
|
||||||
or class.
|
or class.
|
||||||
* Functional tests - Run against a pre-configured environment
|
* Functional tests - Run against a pre-configured environment
|
||||||
(tools/configure_for_func_testing.sh). Typically test a component
|
(tools/configure_for_func_testing.sh). Typically test a component
|
||||||
such as an agent using no mocks.
|
such as an agent using no mocks.
|
||||||
* Integration tests - Run against a running cloud, often target the API level,
|
* Integration tests - Run against a running cloud, often target the API level,
|
||||||
but also 'scenarios' or 'user stories'. You may find such tests under
|
but also 'scenarios' or 'user stories'. You may find such tests under
|
||||||
tests/tempest/api, tests/tempest/scenario, tests/fullstack, and in the
|
tests/tempest/api, tests/tempest/scenario, tests/fullstack, and in the
|
||||||
Tempest and Rally projects.
|
Tempest and Rally projects.
|
||||||
|
|
||||||
Tests in the Neutron tree are typically organized by the testing infrastructure
|
Tests in the Neutron tree are typically organized by the testing infrastructure
|
||||||
used, and not by the scope of the test. For example, many tests under the
|
used, and not by the scope of the test. For example, many tests under the
|
||||||
@ -469,9 +469,7 @@ the tracking of long-running tests and other things.
|
|||||||
|
|
||||||
For more information on the standard Tox-based test infrastructure used by
|
For more information on the standard Tox-based test infrastructure used by
|
||||||
OpenStack and how to do some common test/debugging procedures with Testr,
|
OpenStack and how to do some common test/debugging procedures with Testr,
|
||||||
see this wiki page:
|
see this wiki page: https://wiki.openstack.org/wiki/Testr
|
||||||
|
|
||||||
https://wiki.openstack.org/wiki/Testr
|
|
||||||
|
|
||||||
.. _Testr: https://wiki.openstack.org/wiki/Testr
|
.. _Testr: https://wiki.openstack.org/wiki/Testr
|
||||||
.. _tox: http://tox.readthedocs.org/en/latest/
|
.. _tox: http://tox.readthedocs.org/en/latest/
|
||||||
@ -593,10 +591,10 @@ doc/source/devref/testing_coverage.rst. You could also rely on Zuul
|
|||||||
logs, that are generated post-merge (not every project builds coverage
|
logs, that are generated post-merge (not every project builds coverage
|
||||||
results). To access them, do the following:
|
results). To access them, do the following:
|
||||||
|
|
||||||
* Check out the latest `merge commit <https://review.openstack.org/gitweb?p=openstack/neutron.git;a=search;s=Jenkins;st=author>`_
|
* Check out the latest `merge commit <https://review.openstack.org/gitweb?p=openstack/neutron.git;a=search;s=Jenkins;st=author>`_
|
||||||
* Go to: http://logs.openstack.org/<first-2-digits-of-sha1>/<sha1>/post/neutron-coverage/.
|
* Go to: http://logs.openstack.org/<first-2-digits-of-sha1>/<sha1>/post/neutron-coverage/.
|
||||||
* `Spec <https://review.openstack.org/#/c/221494/>`_ is a work in progress to
|
* `Spec <https://review.openstack.org/#/c/221494/>`_ is a work in progress to
|
||||||
provide a better landing page.
|
provide a better landing page.
|
||||||
|
|
||||||
Debugging
|
Debugging
|
||||||
---------
|
---------
|
||||||
|
|||||||
@ -77,68 +77,68 @@ This means showing proof of adoption of practices as led by the Neutron core
|
|||||||
team. Some of these practices are typically already followed by the most
|
team. Some of these practices are typically already followed by the most
|
||||||
mature OpenStack projects:
|
mature OpenStack projects:
|
||||||
|
|
||||||
* Exhaustive documentation: it is expected that each project will have a
|
* Exhaustive documentation: it is expected that each project will have a
|
||||||
`developer <http://docs.openstack.org/developer/neutron/>`_,
|
`developer <http://docs.openstack.org/developer/neutron/>`_,
|
||||||
`user/operator <http://docs.openstack.org/mitaka/networking-guide/>`_
|
`user/operator <http://docs.openstack.org/mitaka/networking-guide/>`_
|
||||||
and `API <http://developer.openstack.org/api-ref/networking/>`_
|
and `API <http://developer.openstack.org/api-ref/networking/>`_
|
||||||
documentations available.
|
documentations available.
|
||||||
|
|
||||||
* Exhaustive OpenStack CI coverage: unit, functional, and tempest coverage
|
* Exhaustive OpenStack CI coverage: unit, functional, and tempest coverage
|
||||||
using OpenStack CI (upstream) resources so that `Grafana <http://grafana.openstack.org/dashboard/db/neutron-failure-rate>`_
|
using OpenStack CI (upstream) resources so that `Grafana <http://grafana.openstack.org/dashboard/db/neutron-failure-rate>`_
|
||||||
and `OpenStack Health <http://status.openstack.org/openstack-health/#/>`_
|
and `OpenStack Health <http://status.openstack.org/openstack-health/#/>`_
|
||||||
support is available. Access to CI resources and historical data by the
|
support is available. Access to CI resources and historical data by the
|
||||||
team is key to ensuring stability and robustness of a project.
|
team is key to ensuring stability and robustness of a project.
|
||||||
In particular, it is of paramount importance to ensure that DB models/migrations
|
In particular, it is of paramount importance to ensure that DB models/migrations
|
||||||
are tested functionally to prevent data inconsistency issues or unexpected
|
are tested functionally to prevent data inconsistency issues or unexpected
|
||||||
DB logic errors due to schema/models mismatch. For more details, please
|
DB logic errors due to schema/models mismatch. For more details, please
|
||||||
look at the following resources:
|
look at the following resources:
|
||||||
|
|
||||||
* https://review.openstack.org/#/c/346091/
|
* https://review.openstack.org/#/c/346091/
|
||||||
* https://review.openstack.org/#/c/346272/
|
* https://review.openstack.org/#/c/346272/
|
||||||
* https://review.openstack.org/#/c/346083/
|
* https://review.openstack.org/#/c/346083/
|
||||||
|
|
||||||
More Database related information can be found on:
|
More Database related information can be found on:
|
||||||
|
|
||||||
* http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html
|
* http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html
|
||||||
* http://docs.openstack.org/developer/neutron/devref/db_layer.html
|
* http://docs.openstack.org/developer/neutron/devref/db_layer.html
|
||||||
|
|
||||||
Bear in mind that many projects have been transitioning their codebase and
|
Bear in mind that many projects have been transitioning their codebase and
|
||||||
tests to fully support Python 3+, and it is important that each Stadium
|
tests to fully support Python 3+, and it is important that each Stadium
|
||||||
project supports Python 3+ the same way Neutron core does. For more
|
project supports Python 3+ the same way Neutron core does. For more
|
||||||
information on how to do testing, please refer to the
|
information on how to do testing, please refer to the
|
||||||
`Neutron testing documentation <http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron>`_.
|
`Neutron testing documentation <http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron>`_.
|
||||||
|
|
||||||
* Good release footprint, according to the chosen `release model <http://governance.openstack.org/reference/tags/#release-management-tags>`_.
|
* Good release footprint, according to the chosen `release model <http://governance.openstack.org/reference/tags/#release-management-tags>`_.
|
||||||
|
|
||||||
* Adherence to deprecation and `stable backports policies <http://governance.openstack.org/reference/tags/#stable-maintenance-tags>`_.
|
* Adherence to deprecation and `stable backports policies <http://governance.openstack.org/reference/tags/#stable-maintenance-tags>`_.
|
||||||
|
|
||||||
* Demonstrated ability to do `upgrades <http://governance.openstack.org/reference/tags/assert_supports-upgrade.html>`_
|
* Demonstrated ability to do `upgrades <http://governance.openstack.org/reference/tags/assert_supports-upgrade.html>`_
|
||||||
and/or `rolling upgrades <http://governance.openstack.org/reference/tags/assert_supports-rolling-upgrade.html>`_,
|
and/or `rolling upgrades <http://governance.openstack.org/reference/tags/assert_supports-rolling-upgrade.html>`_,
|
||||||
where applicable. This means having grenade support on top of the CI
|
where applicable. This means having grenade support on top of the CI
|
||||||
coverage as described above.
|
coverage as described above.
|
||||||
|
|
||||||
* Client bindings and CLI developed according to the OpenStack Client `plugin model <http://docs.openstack.org/developer/python-openstackclient/plugins.html>`_.
|
* Client bindings and CLI developed according to the OpenStack Client `plugin model <http://docs.openstack.org/developer/python-openstackclient/plugins.html>`_.
|
||||||
|
|
||||||
On top of the above mentioned criteria, the following also are taken into
|
On top of the above mentioned criteria, the following also are taken into
|
||||||
consideration:
|
consideration:
|
||||||
|
|
||||||
* A project must use, adopt and implement open software and technologies.
|
* A project must use, adopt and implement open software and technologies.
|
||||||
|
|
||||||
* A project must integrate with Neutron via one of the supported, advertised
|
* A project must integrate with Neutron via one of the supported, advertised
|
||||||
and maintained public Python APIs. REST API does not qualify (the project
|
and maintained public Python APIs. REST API does not qualify (the project
|
||||||
python-neutronclient is an exception).
|
python-neutronclient is an exception).
|
||||||
|
|
||||||
* It adopts neutron-lib (with related hacking rules applied), and has proof
|
* It adopts neutron-lib (with related hacking rules applied), and has proof
|
||||||
of good decoupling from Neutron core internals.
|
of good decoupling from Neutron core internals.
|
||||||
|
|
||||||
* It provides an API that adopts API guidelines as set by the Neutron core
|
* It provides an API that adopts API guidelines as set by the Neutron core
|
||||||
team, and that relies on an open implementation.
|
team, and that relies on an open implementation.
|
||||||
|
|
||||||
* It adopts modular interfaces to provide networking services: this means
|
* It adopts modular interfaces to provide networking services: this means
|
||||||
that L2/7 services are provided in the form of ML2 mech drivers and
|
that L2/7 services are provided in the form of ML2 mech drivers and
|
||||||
service plugins respectively. A service plugin can expose a driver
|
service plugins respectively. A service plugin can expose a driver
|
||||||
interface to support multiple backend technologies, and/or adopt the
|
interface to support multiple backend technologies, and/or adopt the
|
||||||
flavor framework as necessary.
|
flavor framework as necessary.
|
||||||
|
|
||||||
Adding or removing projects to the Stadium
|
Adding or removing projects to the Stadium
|
||||||
------------------------------------------
|
------------------------------------------
|
||||||
@ -179,112 +179,112 @@ enhancements.
|
|||||||
Checklist
|
Checklist
|
||||||
---------
|
---------
|
||||||
|
|
||||||
* How to integrate documentation into docs.o.o: The documentation
|
* How to integrate documentation into docs.o.o: The documentation
|
||||||
website has a section for `project developer documentation <http://docs.openstack.org/developer/openstack-projects.html>`_.
|
website has a section for `project developer documentation <http://docs.openstack.org/developer/openstack-projects.html>`_.
|
||||||
Each project in the Neutron Stadium must have an entry under the
|
Each project in the Neutron Stadium must have an entry under the
|
||||||
'Networking Sub Projects' section that points to the developer
|
'Networking Sub Projects' section that points to the developer
|
||||||
documentation for the project, available at http://docs.openstack.org/developer/<your-project>/.
|
documentation for the project, available at http://docs.openstack.org/developer/<your-project>/.
|
||||||
This is a two step process that involves the following:
|
This is a two step process that involves the following:
|
||||||
|
|
||||||
* Build the artefacts: this can be done by following example
|
* Build the artefacts: this can be done by following example
|
||||||
https://review.openstack.org/#/c/293399/.
|
https://review.openstack.org/#/c/293399/.
|
||||||
* Publish the artefacts: this can be done by following example
|
* Publish the artefacts: this can be done by following example
|
||||||
https://review.openstack.org/#/c/216448/.
|
https://review.openstack.org/#/c/216448/.
|
||||||
|
|
||||||
More information can also be found on the
|
More information can also be found on the
|
||||||
`project creator guide <http://docs.openstack.org/infra/manual/creators.html#add-link-to-your-developer-documentation>`_.
|
`project creator guide <http://docs.openstack.org/infra/manual/creators.html#add-link-to-your-developer-documentation>`_.
|
||||||
|
|
||||||
* How to integrate into Grafana: Grafana is a great tool that provides
|
* How to integrate into Grafana: Grafana is a great tool that provides
|
||||||
the ability to display historical series, like failure rates of
|
the ability to display historical series, like failure rates of
|
||||||
OpenStack CI jobs. A few examples that added dashboards over time are:
|
OpenStack CI jobs. A few examples that added dashboards over time are:
|
||||||
|
|
||||||
* `Neutron <https://review.openstack.org/#/c/278832/>`_.
|
* `Neutron <https://review.openstack.org/#/c/278832/>`_.
|
||||||
* `Networking-OVN <https://review.openstack.org/#/c/335791>`_.
|
* `Networking-OVN <https://review.openstack.org/#/c/335791>`_.
|
||||||
* `Networking-Midonet <https://review.openstack.org/#/c/315033>`_.
|
* `Networking-Midonet <https://review.openstack.org/#/c/315033>`_.
|
||||||
|
|
||||||
Any subproject must have a Grafana dashboard that shows failure
|
Any subproject must have a Grafana dashboard that shows failure
|
||||||
rates for at least Gate and Check queues.
|
rates for at least Gate and Check queues.
|
||||||
|
|
||||||
* How to integrate into neutron-lib's CI: there are a number of steps
|
* How to integrate into neutron-lib's CI: there are a number of steps
|
||||||
required to integrate with neutron-lib CI and adopt neutron-lib in
|
required to integrate with neutron-lib CI and adopt neutron-lib in
|
||||||
general. One step is to validate that neutron-lib master is working
|
general. One step is to validate that neutron-lib master is working
|
||||||
with the master of a given project that uses neutron-lib. For example
|
with the master of a given project that uses neutron-lib. For example
|
||||||
`patch <https://review.openstack.org/#/c/338603/>`_ introduced such
|
`patch <https://review.openstack.org/#/c/338603/>`_ introduced such
|
||||||
support for the Neutron project. Any subproject that wants to do the
|
support for the Neutron project. Any subproject that wants to do the
|
||||||
same would need to adopt the following few lines:
|
same would need to adopt the following few lines:
|
||||||
|
|
||||||
#. https://review.openstack.org/#/c/338603/4/jenkins/jobs/projects.yaml@4685
|
#. https://review.openstack.org/#/c/338603/4/jenkins/jobs/projects.yaml@4685
|
||||||
#. https://review.openstack.org/#/c/338603/3/zuul/layout.yaml@8501
|
#. https://review.openstack.org/#/c/338603/3/zuul/layout.yaml@8501
|
||||||
#. https://review.openstack.org/#/c/338603/4/grafana/neutron.yaml@39
|
#. https://review.openstack.org/#/c/338603/4/grafana/neutron.yaml@39
|
||||||
|
|
||||||
Line 1 and 2 respectively add a job to the periodic queue for the
|
Line 1 and 2 respectively add a job to the periodic queue for the
|
||||||
project, whereas line 3 introduced the failure rate trend for the
|
project, whereas line 3 introduced the failure rate trend for the
|
||||||
periodic job to spot failure spikes etc. Make sure your project has
|
periodic job to spot failure spikes etc. Make sure your project has
|
||||||
the following:
|
the following:
|
||||||
|
|
||||||
#. https://review.openstack.org/#/c/357086/
|
#. https://review.openstack.org/#/c/357086/
|
||||||
#. https://review.openstack.org/#/c/359143/
|
#. https://review.openstack.org/#/c/359143/
|
||||||
|
|
||||||
* How to port api-ref over to neutron-lib: to publish the subproject
|
* How to port api-ref over to neutron-lib: to publish the subproject
|
||||||
API reference into the `Networking API guide <http://developer.openstack.org/api-ref/networking/>`_
|
API reference into the `Networking API guide <http://developer.openstack.org/api-ref/networking/>`_
|
||||||
you must contribute the API documentation into neutron-lib's api-ref
|
you must contribute the API documentation into neutron-lib's api-ref
|
||||||
directory as done in the `WADL/REST transition patch <https://review.openstack.org/#/c/327510/>`_.
|
directory as done in the `WADL/REST transition patch <https://review.openstack.org/#/c/327510/>`_.
|
||||||
Once this is done successfully, a link to the subproject API will
|
Once this is done successfully, a link to the subproject API will
|
||||||
show under the published `table of content <https://github.com/openstack/neutron-lib/blob/master/api-ref/source/index.rst>`_.
|
show under the published `table of content <https://github.com/openstack/neutron-lib/blob/master/api-ref/source/index.rst>`_.
|
||||||
An RFE bug tracking this effort effectively initiates the request
|
An RFE bug tracking this effort effectively initiates the request
|
||||||
for Stadium inclusion, where all the aspects as outlined in this
|
for Stadium inclusion, where all the aspects as outlined in this
|
||||||
documented are reviewed by the PTL.
|
documented are reviewed by the PTL.
|
||||||
|
|
||||||
* How to port API definitions over the neutron-lib: the most basic
|
* How to port API definitions over the neutron-lib: the most basic
|
||||||
steps to port API definitions over to neutron-lib are demonstrated
|
steps to port API definitions over to neutron-lib are demonstrated
|
||||||
in the following patches:
|
in the following patches:
|
||||||
|
|
||||||
* https://review.openstack.org/#/c/353131/
|
* https://review.openstack.org/#/c/353131/
|
||||||
* https://review.openstack.org/#/c/353132/
|
* https://review.openstack.org/#/c/353132/
|
||||||
|
|
||||||
The `neutron-lib patch <https://review.openstack.org/#/c/353131/>`_
|
The `neutron-lib patch <https://review.openstack.org/#/c/353131/>`_
|
||||||
introduces the elements that define the API, and testing coverage
|
introduces the elements that define the API, and testing coverage
|
||||||
validates that the resource and actions maps use valid keywords.
|
validates that the resource and actions maps use valid keywords.
|
||||||
API reference documentation is provided alongside the definition to
|
API reference documentation is provided alongside the definition to
|
||||||
keep everything in one place.
|
keep everything in one place.
|
||||||
The `neutron patch <https://review.openstack.org/#/c/353132/>`_
|
The `neutron patch <https://review.openstack.org/#/c/353132/>`_
|
||||||
uses the Neutron extension framework to plug the API definition
|
uses the Neutron extension framework to plug the API definition
|
||||||
on top of the Neutron API backbone. The change can only merge when
|
on top of the Neutron API backbone. The change can only merge when
|
||||||
there is a released version of neutron-lib.
|
there is a released version of neutron-lib.
|
||||||
|
|
||||||
* How to integrate into the openstack release: every project in the
|
* How to integrate into the openstack release: every project in the
|
||||||
Stadium must have release notes. In order to set up release notes,
|
Stadium must have release notes. In order to set up release notes,
|
||||||
please see the patches below for an example on how to set up reno:
|
please see the patches below for an example on how to set up reno:
|
||||||
|
|
||||||
* https://review.openstack.org/#/c/320904/
|
* https://review.openstack.org/#/c/320904/
|
||||||
* https://review.openstack.org/#/c/243085/
|
* https://review.openstack.org/#/c/243085/
|
||||||
|
|
||||||
For release documentation related to Neutron, please check the
|
For release documentation related to Neutron, please check the
|
||||||
`Neutron policies document <http://docs.openstack.org/developer/neutron/#neutron-policies>`_.
|
`Neutron policies document <http://docs.openstack.org/developer/neutron/#neutron-policies>`_.
|
||||||
Once, everything is set up and your project is released, make sure
|
Once, everything is set up and your project is released, make sure
|
||||||
you see an entry on the release page (e.g. `Ocata <http://releases.openstack.org/ocata/index.html#other-projects>`_.
|
you see an entry on the release page (e.g. `Ocata <http://releases.openstack.org/ocata/index.html#other-projects>`_.
|
||||||
Make sure you release according to the project declared release
|
Make sure you release according to the project declared release
|
||||||
`model <http://governance.openstack.org/reference/projects/neutron.html#deliverables-and-tags>`_.
|
`model <http://governance.openstack.org/reference/projects/neutron.html#deliverables-and-tags>`_.
|
||||||
|
|
||||||
* How to port OpenStack Client over to python-neutronclient: client
|
* How to port OpenStack Client over to python-neutronclient: client
|
||||||
API bindings and client command line interface support must be
|
API bindings and client command line interface support must be
|
||||||
developed in python-neutronclient under `osc module <https://github.com/openstack/python-neutronclient/tree/master/neutronclient/osc/v2>`_.
|
developed in python-neutronclient under `osc module <https://github.com/openstack/python-neutronclient/tree/master/neutronclient/osc/v2>`_.
|
||||||
If your project requires one or both, consider looking at the
|
If your project requires one or both, consider looking at the
|
||||||
following example on how to contribute these two python-neutronclient
|
following example on how to contribute these two python-neutronclient
|
||||||
according to the OSC framework and guidelines:
|
according to the OSC framework and guidelines:
|
||||||
|
|
||||||
* https://review.openstack.org/#/c/340624/
|
* https://review.openstack.org/#/c/340624/
|
||||||
* https://review.openstack.org/#/c/340763/
|
* https://review.openstack.org/#/c/340763/
|
||||||
* https://review.openstack.org/#/c/352653/
|
* https://review.openstack.org/#/c/352653/
|
||||||
|
|
||||||
More information on how to develop python-openstackclient plugins
|
More information on how to develop python-openstackclient plugins
|
||||||
can be found on the following links:
|
can be found on the following links:
|
||||||
|
|
||||||
* http://docs.openstack.org/developer/python-openstackclient/plugins.html
|
* http://docs.openstack.org/developer/python-openstackclient/plugins.html
|
||||||
* http://docs.openstack.org/developer/python-openstackclient/humaninterfaceguide.html
|
* http://docs.openstack.org/developer/python-openstackclient/humaninterfaceguide.html
|
||||||
|
|
||||||
It is worth prefixing the commands being added with the keyword
|
It is worth prefixing the commands being added with the keyword
|
||||||
`network <https://review.openstack.org/#/c/340624/10/setup.cfg>`_ to
|
`network <https://review.openstack.org/#/c/340624/10/setup.cfg>`_ to
|
||||||
avoid potential clash with other commands with similar names. This
|
avoid potential clash with other commands with similar names. This
|
||||||
is only required if the command object name is highly likely to have
|
is only required if the command object name is highly likely to have
|
||||||
an ambiguous meaning.
|
an ambiguous meaning.
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user