From 235c26445763ffae04a3cbde7e1f13b426dbf27c Mon Sep 17 00:00:00 2001 From: Ghanshyam Mann Date: Thu, 16 Apr 2020 16:55:56 -0500 Subject: [PATCH] Document the policy for Tempest & its plugins release Tempest and its plugin are very tightly coupled for testing Upstream or Production Cloud. Due to its branchless model, one of the challenges we faced/facing is to provide their compatible version set. Since Queens release and with operator request, we started doing a coordinated release of plugins along with Tempest which will be marked as the First compatible version for that particular release. For example https://releases.openstack.org/train/index.html#tempest-plugins We test our stable branch with master version of Tempest & plugins until it is moved to Extended Maintainance state. One stable release is in EM state, Tempest officially declares the end of support tag so that we can use that as the last compatible version for that EM stable testing. But we do not have the correct last compatible version from plugins which can be used for testing the EM branch. It should be with hash from the master at the time stable release is moved to EM not the hash when the stable release was done otherwise it will be incompatible with Tempest. We have faced and fixed many issue due to that during py2-drop work where we have to pick and try all different versions from plugin to make it work against EM testing. Change-Id: I99a6df0a2c02e19efaf484fcd97b875887f08d8e --- doc/source/index.rst | 10 +++- ..._and_plugins_compatible_version_policy.rst | 54 +++++++++++++++++++ 2 files changed, 63 insertions(+), 1 deletion(-) create mode 100644 doc/source/tempest_and_plugins_compatible_version_policy.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index ab994d12b9..a72e783f12 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -80,6 +80,14 @@ Plugins plugin plugin-registry +Tempest & Plugins Compatible Version Policy +------------------------------------------- + +.. toctree:: + :maxdepth: 2 + + tempest_and_plugins_compatible_version_policy + Library ------- @@ -104,4 +112,4 @@ Search * :ref:`Tempest document search `: Search the contents of this document. * `OpenStack wide search `_: Search the wider - set of OpenStack documentation, including forums. \ No newline at end of file + set of OpenStack documentation, including forums. diff --git a/doc/source/tempest_and_plugins_compatible_version_policy.rst b/doc/source/tempest_and_plugins_compatible_version_policy.rst new file mode 100644 index 0000000000..942b1bdf06 --- /dev/null +++ b/doc/source/tempest_and_plugins_compatible_version_policy.rst @@ -0,0 +1,54 @@ +Tempest and Plugins compatible version policy +============================================= + +Tempest and its plugins are responsible for the integrated +testing of OpenStack. These tools have two use cases: + +#. Testing upstream code at gate +#. Testing Production Cloud + +Upstream code is tested by the master version of branchless Tempest & plugins +for all supported stable branches in `Maintained phase`_. + +Production Cloud can be tested by using the compatible version or using +master version. It depends on the testing strategy of cloud. To provide +the compatible version of Tempest and its Plugins per OpenStack release, +we started the coordinated release of all plugins and Tempest per OpenStack +release. +These versions are the first set of versions from Tempest and its Plugins to +officially start the support of a particular OpenStack release. For example: +OpenStack Train release first compatible versions `Tempest plugins version`_. + +Because of branchless nature of Tempest and its plugins, first version +released during OpenStack release is not the last version to support that +OpenStack release. This means the next (or master) versions can also be used +for upstream testing as well as in production testing. + +Since the `Extended Maintenance policy`_ for stable branch, Tempest +started releasing the ``end of support`` version once stable release +is moved to EM state, which used to happen on EOL of stable release. This is +the last compatible version of Tempest for the OpenStack release moved to EM. + +Because of branchless nature as explained above, we have a range of versions +which can be considered a compatible version for particular OpenStack release. +How we should release those versions is mentioned in the below table. + + +-----------------------------+-----------------+------------------------------------+ + | First compatible version -> | OpenStack 'XYZ' | <- Last compatible version | + +=============================+=================+====================================+ + |This is the latest version | |This is the version released | + |released when OpenStack | |when OpenStack 'XYZ' is moved to | + |'XYZ' is released. | |EM state. Hash used for this should | + |Example: | |be the hash from master at the time | + |`Tempest plugins version`_ | |of branch is EM not the one used for| + | | |First compatible version | + +-----------------------------+-----------------+------------------------------------+ + +Tempest & the Plugins should follow the above mentioned policy for the +``First compatible version`` and the ``Last compatible version.`` +so that we provide the right set of compatible versions to Upstream as well as to +Production Cloud testing. + +.. _Maintained phase: https://docs.openstack.org/project-team-guide/stable-branches.html#maintained +.. _Extended Maintenance policy: https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.html +.. _Tempest plugins version: https://releases.openstack.org/train/#tempest-plugins