From b8e877300689605564c3f4e51d05223e7ab26ae2 Mon Sep 17 00:00:00 2001 From: Bruce Jones Date: Thu, 7 Nov 2019 11:45:22 -0800 Subject: [PATCH] Update the sample specification to clarify test plan requirements Update the sample specification to 1) remove OpenStack-isms and 2) clarify the testing needed for changes to StarlingX. This review only changes the template for StarlingX 4.0 and forward. For reasons I don't understand git is not letting me remove the now obsolete soft link to the specs/example spec from the 2019.03 version. Change-Id: Ia51cd327432655b600911ed35a0f43f40a80fbf8 Signed-off-by: Bruce Jones --- doc/source/specs/stx-4.0/template.rst | 39 ++++++++++++++++++--------- 1 file changed, 27 insertions(+), 12 deletions(-) diff --git a/doc/source/specs/stx-4.0/template.rst b/doc/source/specs/stx-4.0/template.rst index 3b2f7a9..d7e393e 100644 --- a/doc/source/specs/stx-4.0/template.rst +++ b/doc/source/specs/stx-4.0/template.rst @@ -294,21 +294,36 @@ Dependencies Testing ======= -Please discuss the important scenarios needed to test here, as well as specific -edge cases we should be ensuring work correctly. For each scenario please -specify if this requires specialized hardware, a full openstack environment, or -can be simulated inside the project tree. +Please discuss your plans for testing this feature. Your plans should +include: -Please discuss how the change will be tested. We especially want to know what -tempest tests will be added. It is assumed that unit test coverage will be -added so that doesn't need to be mentioned explicitly, but discussion of why -you think unit tests are sufficient and we don't need to add more tests would -need to be included. +* A discussion of whether or not you plan to create unit tests for the + feature. -Is this untestable in gate given current limitations (specific hardware / -software configurations available)? If so, are there mitigation plans (3rd -party testing, gate enhancements, etc). +* A discussion of how the new functionality you are adding + will be tested by you. Describe at a high level the test scenarios that + are needed and which (if any) can be covered by existing tests. Identify + any areas outside of your changes that could be impacted by them, and + thus would need to be tested. +* Once we have an infrastructure for + automated functional testing for StarlingX, + this section should include a brief description of the kinds of test + cases that can be automated within that framework for this feature. + +* any existing tests that should be run to test this feature. + +Your test plan should cover the important scenarios that +need to be tested. Please think about +edge cases as well. Your test plan should cover all of +the StarlingX physical (e.g. simplex, duplex, standard) +configurations that might be impacted by the changes you are making. + +Your changes may impact several parts of the operation of a StarlingX +deployment. Please work with your sub-project PL, TL and sub-project +team to prepare and review your test plan. Successful completion of +the feature test plan is a gate to the code being approved +by the project's core reviewers. Documentation Impact ====================