governance/resolutions/20180307-trademark-program-test-location.rst
Ghanshyam Mann b3e4cc67ec Fix hardcoded extlinks link error in sphinx 4.4.0
governance doc job is failing because sphinx 4.4.0
raise error for hardcoded extlinks link
- https://github.com/sphinx-doc/sphinx/pull/9800

Warning, treated as error:
/home/zuul/src/opendev.org/openstack/governance/doc/source/goals/completed/pike/deploy-api-in-wsgi.rst:269:hardcoded link '6ef5fa9adc'

https://zuul.opendev.org/t/openstack/build/b84dd8d2989243bfa0cebee1a21dc4ff

we have 'repo' as extlinks
- 16302d8a2f/doc/source/conf.py (L50)

This fixes the hardcoded extlinks link and unblock the gate.

Change-Id: I61873080e0ebaa679fee191e1ae968a488bd637b
2022-01-19 12:25:29 -06:00

3.2 KiB

2018-03-07 Location of tempest tests for OpenStack Trademark Programs

Introduction

When 20160504_defcore_test_location was proposed, the trademark program was simpler, involving only a small subset of the projects available in OpenStack.

With the proposed "add on" and "vertical" trademark programs, more projects are involved. This growth introduces some scaling problems into the management of tests for the trademark programs. These issues were discussed at the Rocky PTG. The outcome of those discussions are reported here.

This resolution supersedes 20160504_defcore_test_location.

Updated Resolution

Whereas the Trademark Program has made it clear that they have the tooling and willingness to support trademark tests located in any of

  • The openstack/tempest
  • Project specific tempest plugins (e.g., image-trademark-tempest-plugin)
  • Vertical specific tempest plugins (e.g., nfv-vertical-trademark-tempest-plugin)
  • A global Trademark tempest plugin (e.g., openstack-trademark-tempest-plugin)

as long as they are tests that can be run by tempest.

And the QA Team, Trademark Program and individual projects have shared responsibility for the management of trademark tests, with varying degrees of bandwidth for review attention.

And trademark tests are expected to be changed rarely.

It is resolved that trademark tests (and candidates thereof) for individual projects should live in one, and only one, of four possible locations, each with differing strategies for review and repository ownership (status in projects):

  • The openstack/tempest, with approval reviews performed by the QA Team, repository ownership via the QA Team (already true).
  • A vertical specific tempest plugin, with approval reviews performed by the QA Team and the Trademark Program, repository ownership via the Trademark Program.
  • A global Trademark tempest plugin, with approval reviews performed by the QA Team and the Trademark Program, repository ownership via the Trademark Program.
  • A project specific tempest plugin, specifically limited to trademark-oriented tests, with approval reviews performed by the project, the QA Team, and the Trademark Program, repository ownership via the Trademark Program if they accept, otherwise the project team.

The choice is up to the individual projects, in consultation with the QA Team and the Trademark Program.

Note

No overarching direction is provided here on what tooling must be used in the tests but it is generally expected that the tooling should be agreed by the people who are reviewing the code. If a project wants to use tooling other than the core tempest libraries (e.g., gabbi) they should prefer a project specific plugin and be aware that their choice of tooling may limit the ability for non-project parties to review tests. Any tooling chosen must be able to produce tests that are run by tempest.