3.2 KiB
2018-03-07 Location of tempest tests for OpenStack Trademark Programs
Introduction
When 20160504_defcore_test_location was proposed, the
trademark program was simpler, involving only a small subset of the
projects available in OpenStack.
With the proposed "add on" and "vertical" trademark programs, more projects are involved. This growth introduces some scaling problems into the management of tests for the trademark programs. These issues were discussed at the Rocky PTG. The outcome of those discussions are reported here.
This resolution supersedes 20160504_defcore_test_location.
Updated Resolution
Whereas the Trademark Program has made it clear that they have the tooling and willingness to support trademark tests located in any of
- The
openstack/tempest - Project specific tempest plugins (e.g., image-trademark-tempest-plugin)
- Vertical specific tempest plugins (e.g., nfv-vertical-trademark-tempest-plugin)
- A global Trademark tempest plugin (e.g., openstack-trademark-tempest-plugin)
as long as they are tests that can be run by tempest.
And the QA Team, Trademark Program and individual projects have shared responsibility for the management of trademark tests, with varying degrees of bandwidth for review attention.
And trademark tests are expected to be changed rarely.
It is resolved that trademark tests (and candidates thereof) for
individual projects should live in one, and only one, of four possible
locations, each with differing strategies for review and repository
ownership (status in projects):
- The
openstack/tempest, with approval reviews performed by the QA Team, repository ownership via the QA Team (already true). - A vertical specific tempest plugin, with approval reviews performed by the QA Team and the Trademark Program, repository ownership via the Trademark Program.
- A global Trademark tempest plugin, with approval reviews performed by the QA Team and the Trademark Program, repository ownership via the Trademark Program.
- A project specific tempest plugin, specifically limited to trademark-oriented tests, with approval reviews performed by the project, the QA Team, and the Trademark Program, repository ownership via the Trademark Program if they accept, otherwise the project team.
The choice is up to the individual projects, in consultation with the QA Team and the Trademark Program.
Note
No overarching direction is provided here on what tooling must be used in the tests but it is generally expected that the tooling should be agreed by the people who are reviewing the code. If a project wants to use tooling other than the core tempest libraries (e.g., gabbi) they should prefer a project specific plugin and be aware that their choice of tooling may limit the ability for non-project parties to review tests. Any tooling chosen must be able to produce tests that are run by tempest.