Merge "Move third-party CI policy under docs/policies"
This commit is contained in:
Jenkins
Gerrit Code Review
commit
6b9fd15072
|
|
@ -0,0 +1,131 @@
|
|||
Neutron Third-party CI
|
||||
======================
|
||||
|
||||
What Is Expected of Third Party CI System for Neutron
|
||||
-----------------------------------------------------
|
||||
|
||||
As of the Liberty summit, Neutron no longer *requires* a third-party CI,
|
||||
but it is strongly encouraged, as internal neutron refactoring can break
|
||||
external plugins and drivers at any time.
|
||||
|
||||
Neutron expects any Third Party CI system that interacts with gerrit to
|
||||
follow the requirements set by the Infrastructure team[1] as well as the
|
||||
Neutron Third Party CI guidelines below. Please ping the PTL in
|
||||
#openstack-neutron or send an email to the openstack-dev ML (with subject
|
||||
[neutron]) with any questions. Be aware that the Infrastructure documentation
|
||||
as well as this document are living documents and undergo changes. Track
|
||||
changes to the infrastructure documentation using this url[2] (and please
|
||||
review the patches) and check this doc on a regular basis for updates.
|
||||
|
||||
[1] http://ci.openstack.org/third_party.html
|
||||
[2] https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:third-party,n,z
|
||||
|
||||
What Changes to Run Against
|
||||
---------------------------
|
||||
|
||||
If your code is a neutron plugin or driver, you should run against every
|
||||
neutron change submitted, except for docs, tests, tools, and top-level
|
||||
setup files. You can see a programmatic example of the exceptions here[3].
|
||||
|
||||
[3] https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L568
|
||||
|
||||
If your code is in a neutron-*aas repo, you should run against the tests
|
||||
for that repo. You may also run against every neutron change, if your service
|
||||
driver is using neutron interfaces that are not provided by your service
|
||||
plugin (e.g. loadbalancer/plugin.py). If you are using only plugin interfaces,
|
||||
it should be safe to test against only the service repo tests.
|
||||
|
||||
What Tests To Run
|
||||
-----------------
|
||||
|
||||
Network API tests (git link).
|
||||
Network scenario tests (The test_network_* tests here).
|
||||
Any tests written specifically for your setup.
|
||||
http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/network
|
||||
|
||||
Run with the test filter: 'network'. This will include all neutron specific
|
||||
tests as well as any other tests that are tagged as requiring networking. An
|
||||
example tempest setup for devstack-gate:
|
||||
export DEVSTACK_GATE_NEUTRON=1
|
||||
export DEVSTACK_GATE_TEMPEST_REGEX='(?!.*\[.*\bslow\b.*\])((network)|(neutron))'
|
||||
|
||||
... an example setup for LBaaS:
|
||||
export DEVSTACK_GATE_NEUTRON=1
|
||||
export DEVSTACK_GATE_TEMPEST_REGEX='(?!.*\[.*\bslow\b.*\])(alancer|SimpleReadOnlyNeutron|tempest.api.network)'
|
||||
|
||||
Third Party CI Voting
|
||||
---------------------
|
||||
|
||||
The Neutron team encourages you to NOT vote -1 with a third-party CI. False
|
||||
negatives are noisy to the community, and have given -1 from third-party
|
||||
CIs a bad reputation. Really bad, to the point of people ignoring them all.
|
||||
Failure messages are useful to those doing refactors, and provide you
|
||||
feedback on the state of your plugin.
|
||||
|
||||
If you insist on voting, by default, the infra team will not allow voting
|
||||
by new 3rd party CI systems. The way to get your 3rd party CI system to vote
|
||||
is to talk with the Neutron PTL, who will let infra know the system is ready
|
||||
to vote. The requirements for a new system to be given voting rights are as
|
||||
follows:
|
||||
|
||||
* A new system must be up and running for a month, with a track record of
|
||||
voting on the sandbox system.
|
||||
* A new system must correctly run and pass tests on patches for the third
|
||||
party driver/plugin for a month.
|
||||
* A new system must have a logfile setup and retention setup similar to the
|
||||
below.
|
||||
|
||||
Once the system has been running for a month, the owner of the third party CI
|
||||
system can contact the Neutron PTL to have a conversation about getting voting
|
||||
rights upstream.
|
||||
|
||||
The general process to get these voting rights is outlined here. Please follow
|
||||
that, taking note of the guidelines Neutron also places on voting for it's CI
|
||||
systems.
|
||||
|
||||
A third party system can have it's voting rights removed as well. If the
|
||||
system becomes unstable (stops running, voting, or start providing inaccurate
|
||||
results), the Neutron PTL or any core reviewer will make an attempt to contact
|
||||
the owner and copy the openstack-dev mailing list. If no response is received
|
||||
within 2 days, the Neutron PTL will remove voting rights for the third party
|
||||
CI system. If a response is received, the owner will work to correct the
|
||||
issue. If the issue cannot be addressed in a reasonable amount of time, the
|
||||
voting rights will be temporarily removed.
|
||||
|
||||
Log & Test Results Filesystem Layout
|
||||
------------------------------------
|
||||
|
||||
Third-Party CI systems MUST provide logs and configuration data to help
|
||||
developers troubleshoot test failures. A third-party CI that DOES NOT post
|
||||
logs should be a candidate for removal, and new CI systems MUST post logs
|
||||
before they can be awarded voting privileges.
|
||||
|
||||
Third party CI systems should follow the filesystem layout convention of the
|
||||
OpenStack CI system. Please store your logs as viewable in a web browser, in
|
||||
a directory structure. Requiring the user to download a giant tarball is not
|
||||
acceptable, and will be reason to not allow your system to vote from the
|
||||
start, or cancel it's voting rights if this changes while the system is
|
||||
running.
|
||||
|
||||
At the root of the results - there should be the following:
|
||||
|
||||
console.html.gz - contains the output of stdout of the test run
|
||||
local.conf / localrc - contains the setup used for this run
|
||||
logs/
|
||||
Logs must be a directory, which contains the following:
|
||||
|
||||
Log files for each screen session that DevStack creates and launches an
|
||||
OpenStack component in
|
||||
Test result files
|
||||
testr_results.html.gz
|
||||
tempest.txt.gz
|
||||
|
||||
List of existing plugins and drivers
|
||||
------------------------------------
|
||||
|
||||
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin_and_Drivers
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
[1] https://review.openstack.org/#/c/187267
|
||||
Loading…
Reference in New Issue