48 Commits

Author SHA1 Message Date
Martin Kopec
d76178e682 General doc updates
Added missing doc, for example for serial tests, fixed 404 links,
spellings, formatting and etc

Change-Id: I07a8cf9508d3cba33bd04afdc301bb7145153d59
2024-01-31 09:00:16 +01:00
Ghanshyam Mann
672eee1d31 Add documentation on the usage of keystone's scope & roles
As Tempest support the keystone's scope and new default roles
like reader, this commit document that how those can be
requested and used in Tempest or its plugins tests.

Change-Id: Iebacbeda231f82d6d16dbdcde635f19ae862181f
2021-03-12 09:54:39 -06:00
Lukas Piwowarski
a9055010fc Create documentation for tests
Generate documentation for each test from its docstring.

The sphinx-apidoc tool is used to generate the .rst
files necessary for the generation of the documentation.

When the documentation is being build only functions that
start with "test" are passed to the resulting documentation.

Change-Id: I1878f67e10df9bb1f8289beda8e59a56b5057fe4
2020-09-02 09:46:00 +02:00
Ghanshyam Mann
d89122be41 Document the process to pin the constraint in tox.ini
As discussed in Victoria PTG[1], we need to start pining
the constraint in tox.ini on Tempest new release so that our
tox.ini can match with compatible constraints with Tempest version.

[1] https://etherpad.opendev.org/p/qa-victoria-ptg

Change-Id: Ib705f59ef3aea93b2778cc87fc44d390b5a62c94
2020-07-07 13:11:42 +00:00
Ghanshyam Mann
da3bb61f1f Add index page for plugins documents
Now we have more documents for plugins on stable release,
support and their testing policy. Let's prepare a
index page for it will be easy for plugins maintainer.

Also renaming the stable support policy section to have a clear
name.

Adding redirect links for moved documents.

Change-Id: I17a9a6e0e88605a0ee5047678d6ed168c31396ec
2020-05-05 02:12:05 +00:00
Ghanshyam Mann
96800bb833 Add document for stable branch testing for tempest plugins
As Tempest and plugins master version is used to test the supported
stable branches, we need to test them on master gate.

Adding document for that so that we can test the stable branches
more consistently on plugins side too.

Change-Id: Ied3c6bb4032fd3740ae0d6d185e4a138fd335b02
2020-05-05 01:41:11 +00:00
Zuul
e357481ffb Merge "[ussuri][goal] Update contributor documentation" 2020-04-20 17:36:37 +00:00
Ghanshyam Mann
95c410afc0 [ussuri][goal] Update contributor documentation
This patch updates/adds the contributor documentation to follow
the guidelines of the Ussuri cycle community goal[1].

[1] https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-contrib-docs.html
Story: #2007236
Task: #38554

Change-Id: Ib54b16543eb428087555cd3411bae51244d1aa65
2020-04-19 17:44:59 -05:00
Ghanshyam Mann
235c264457 Document the policy for Tempest & its plugins release
Tempest and its plugin are very tightly coupled for testing
Upstream or Production Cloud. Due to its branchless model,
one of the challenges we faced/facing is to provide their
compatible version set.

Since Queens release and with operator request, we started
doing a coordinated release of plugins along with Tempest
which will be marked as the First compatible version for
that particular release.

For example https://releases.openstack.org/train/index.html#tempest-plugins

We test our stable branch with master version of Tempest &
plugins until it is moved to Extended Maintainance state.
One stable release is in EM state, Tempest officially declares
the end of support tag so that we can use that as the last compatible
version for that EM stable testing. But we do not have the correct
last compatible version from plugins which can be used for testing
the EM branch. It should be with hash from the master at the time
stable release is moved to EM not the hash when the stable release
was done otherwise it will be incompatible with Tempest.

We have faced and fixed many issue due to that during py2-drop work
where we have to pick and try all different versions from plugin to
make it work against EM testing.

Change-Id: I99a6df0a2c02e19efaf484fcd97b875887f08d8e
2020-04-17 15:23:18 +00:00
Ghanshyam Mann
68600b18fa Add doc for supported OpenStack release & py version
Tempest is branchless and support many OpenStack stable branches
and python versions. Users of Tempest should have a clear doc about
what all openstack releases and python versions are supported.

Change-Id: Idc76e07efdd3710a59c773c564ca532419989ed5
2020-01-23 14:39:36 +00:00
Masayuki Igawa
7e4ef4c823
Add PDF building
This commit add PDF building tox.ini environment and options for LaTeX
output. overview.rst is copied from README.rst. And, the original
README.rst file is shrunk because we don't need such a long information
in the README file. People can see the same contents in overview.rst
now.

Change-Id: Id654c814988e78704726d2ba8bea9a03ce8596f8
2019-09-11 16:42:28 +09:00
ghanshyam
247841ecf6 Move Stable branch Policy Doc outside of coding guide
Current doc explaining the stable branches support policy is under
coding guide. But support policy should be at higher level which
can be useful for user or maintainer of Tempest.

Change-Id: Id46e14080559062b4d89e5f7a9acbcd5df09691e
2018-06-20 04:53:16 +00:00
Masayuki Igawa
e88caa8544
Fix minor header wordings
This commit fixes minor header wordings which were mentioned in the
previous patch review[1].

[1] Ia1e8e16e5f5ddb06e4c4ccb4979b0750b88a4db4

Change-Id: I4db09bda5628371186e3548143c59d0841fcb7b0
2017-06-06 12:00:24 +09:00
Masayuki Igawa
d2a657de0e
Update document theme and section headers
This commit updates the document theme. It is clear, simple, up-to-date
and beautiful. And this commit also updates section headers to fix the
document index.

Change-Id: Ia1e8e16e5f5ddb06e4c4ccb4979b0750b88a4db4
2017-05-09 15:34:45 -04:00
Ken'ichi Ohmichi
d8386a3fb4 Rename test-removal.rst
All other rst files don't contain "-" in the file names instead of '_'.
This patch renames test-removal.rst for consistent format on rst files.

TrivialFix

Change-Id: I229870baa1954e1f6b2277318ae5bae332f660ad
2017-04-04 18:00:52 +00:00
Matthew Treinish
e8ab5f9b08
Add tempest test writing guide
This commit is the start of a new doc guide for helping people figure
out how to write good tempest tests. This is just a starting point for
the document and doesn't cover every aspect of writing a test case in
tempest. Right now it just covers the setupClass phase for writing
TestCases. It will be expanded upon in future commits to give a complete
guide to writing tests in Tempest.

Change-Id: Ie0db2079163f536ae72668c8fc9129ae4c6389e2
2017-04-03 11:53:35 -04:00
Jordan Pittier
2e7ae7c6e5 Remove the Stress framework
It's not used anymore. There was general consensus in Feb 2016
to deprecate it (see [1]) and remove it in Newton.

[1]  [qa] deprecating Tempest stress framework

Change-Id: Ib229985ea2a1fee495c9492c9ce1781e6bac1dc6
2016-10-19 14:31:13 +02:00
ghanshyam
2bff1a133a
Remove deprecated Javelin CLI utilities from Tempest
Javelin CLI utilities has been deprecated in Jan 2016
- Ie4ab5258946271a16e96a070abeedfafa5423714
which is over 6 months of Tempest deprecation period.

This patch removes the javelin CLI and corresponding doc/sample
files etc.

Change-Id: Ia59d3e81f245f29a7bf0f56b690586dfc55f16a0
2016-08-01 21:14:42 -04:00
Masayuki Igawa
d787bc29e3 Tidy up document index page
This commit tidies up the tempest document index page. The tempest docs
are increasing day by day. It's really good however I feel it's a little
messy these days. So let's tidy up :)

Change-Id: I4d34934b8d1a157bf38a8f2cafe5de1ba0e6098c
2016-06-29 14:30:24 +09:00
Jenkins
396593432c Merge "Add subunit-describe-calls" 2016-06-27 18:26:45 +00:00
Stephen Lowrie
c8548fc93c Add subunit-describe-calls
Adds new command subunit-describe-calls, documentation,
and unittests.

subunit-describe-calls is a parser for subunit v1 & v2 streams to
determine what REST API calls are made inside of a test and in what
order they are called.

Later commits will add additional functionality relating to request
& response headers & body data along with a stdout output option and
stdin input.

Change-Id: I468d0d3e3b6098da95a81cc86d9bdd1b47ee1f03
2016-06-22 14:22:22 -05:00
Masayuki Igawa
b8532432f8 Add test removal procedure doc
This commit copies the Tempest test removal procedure doc from wiki[1]
to the tempest repo. Because this document is tightly coupled with
tempest so we don't need to manage it in the wiki. And we can see and
review this through the gerrit when we would like to update it.
And I'll update the wiki page to be just a link after merging this.

[1] https://wiki.openstack.org/wiki/QA/Tempest-test-removal

Change-Id: I5f2322a5de7b73252e0162165eeaec120ec94948
2016-06-22 17:31:02 +09:00
Matthew Treinish
a051c22ad0
Add tempest run command
This commit adds a new run command to the unified cli endpoint. The
intent here is for tempest to control it's own run story. This
implements the basic runner and selection functionality to use the
command, however it's not necessarily the end state of the command.
The functionality in this patch is just a starting point to add the
command and the basic functionality needed. It is starting with a
limited feature set with the intent to add additional, more complex
functionality in self contained patches after the command exists.

Co-Authored-by: David Paterson <davpat2112@yahoo.com>
Co-Authored-by: Stephen Lowrie <stephen.lowrie@rackspace.com>

Partially-Implements bp tempest-run-cmd

Depends-On: I09299043e536521d48dbe10632621138e3a366e0
Change-Id: I24588b5c00d005320e8719cf82b5dd95662572cf
2016-06-09 15:49:12 -04:00
step6829
80c14eca47 Centralized Workspaces
Create a consistent means for creation and management of Tempest
workspaces.

Creates a file located at ~/.tempest/workspaces.yaml which stores existing
workspaces.

Available subcommands: list, register, rename, move, remove

bp centralized-workspaces

Change-Id: I9595e3ba809e457951a0ffdf4b15f641f2fec4f4
2016-06-07 21:20:44 -05:00
Jenkins
438084d0cf Merge "Prepare for dynamic generation of tempest plugin registry" 2016-04-08 01:43:38 +00:00
Clint Adams
a413652c25 Prepare for dynamic generation of tempest plugin registry
This is based on the devstack plugin detection code, modified
to check setup.cfg for a tempest_plugins string.  It is to be
executed from a periodic proposal job, generating a list of
plugins in the same format.

Change-Id: I53d082319b6af6fb47083ea42c3ff6b109882e72
2016-03-28 20:11:23 -04:00
ghanshyam
bd460acdc7 Add microversion testing doc in tempest
This commit adds the doc about how to implement microversion
tests in Tempest.

Also this doc will keep record of what all microversion tests
have been implemented in Tempest.

Partially implements blueprint api-microversions-testing-support

Change-Id: I68e6d2a4a681e6588e43e02378a111fd00391f08
2016-03-23 10:49:53 +09:00
Matthew Treinish
55511d98f1
Add documentation for the library interface
This commit adds documentation for the library interface, formerly
known as tempest-lib, and the release versioning.

Partially implements bp tempest-lib-reintegration

Change-Id: I623222a17fe3563bbe767fa803e8f771a21687a1
2016-02-29 15:35:26 -05:00
Matthew Treinish
42d69512d9
Remove the ec2 api tests from tempest
With the introduction of tempest plugins we can now remove the third
party tests for the ec2 api using boto. They've always been the ugly
duckling in tempest (except that it never turned into a swan) where
they go against some of the fundamental principles of tempest tests.
For exaple, like having it's own client implementation, oh and testing
OpenStack APIs. This patch removes all the pieces of the third party
test dir and deprecates all the config options related to boto. A
plugin implementation is being worked [1] that can be used to fill
the coverage hole left by removing these from the tempest tree

[1] https://github.com/mtreinish/tempest_ec2

Change-Id: Ib5e24e19bcba9808a9f49fe7f328668df77fe4f9
2015-12-17 17:59:20 -05:00
Matthew Treinish
f45ba2e604
Cleanup tempest docs a bit
This commit fixes a few things in the tempest docs. First it fixes all
of the sphinx warnings and enables fail on warn to ensure we're using
valid sphinx everywhere. It also adds a link from the configuration
guide to the sample config file.

Change-Id: I3ad645a6bbfa46b4498e1732410743e46d6eb0cb
2015-09-02 11:06:44 -04:00
Jenkins
8a5a3e7582 Merge "Include sample configuration in dev docs" 2015-08-05 00:53:03 +00:00
Matthew Treinish
3a851dc08b
Add plugin interface documentation
This commit starts the docs section on creating and using tempest
plugins. It describes how to create a plugin class, an entrypoint, and
a recommended dir structure. As well notes how to use tempest and
tempest-lib in creating a plugin. It'll likely be expanded in the
future as the plugin interface evolves.

Partially Implements bp external-plugin-interface

Change-Id: If2e0c47d0a719d869a3301ddbca07495c81cc498
2015-08-04 12:31:59 -04:00
Mathew Odden
b389faa22f Include sample configuration in dev docs
Since including the sample configuration in tree is
becoming difficult to maintain, lets try moving it to
the developer docs. We still have the ability to view it
easily for users, and it can be auto-generated on doc build,
so it should also always be up to date.

Change-Id: I7ec2d76448feef5ecb9af7c238bbd12619fa31d8
2015-08-04 10:59:38 -04:00
Jane Zadorozhna
00fc3dc40c Added documentation to tempest-account-generator
Added docstring to cmd/account_generator.py.
Fixed help messages.
Added sphinx documentation for tempest-account-generator utility.

Change-Id: I6640ba8a0eaaea187cf38582ab537394dec020b6
2015-06-03 18:17:06 +03:00
Matthew Treinish
bc1b15b800 Add tempest config guide doc
This commit adds a new doc page to the tempest documentation to
provide a guide to configuring tempest. It currently contains one
section to elaborate on the credentials and auth options in the config
file. This doc will be expanded in the future to add details around
other pieces of tempest configuration.

Change-Id: Iaf7cefc6a5e508437e4b4620ba5bfaa221f3611b
2015-02-24 11:24:18 -05:00
Joe H. Rahme
61469fd194 Adds documentation for Javelin
I followed what's done for the cleanup.py script and re-used parts of
what was described in the javelin spec.

Change-Id: I57adc3764e75784d4614b6c03996433a4ee4332f
2014-12-18 20:06:48 +02:00
David Paterson
ce781498cf Add tempest post-run cleanup script
Implemented all features defined in blueprint.

Change-Id: If3c9b82f85095eeb22bbf32841be534024213567
Implements: blueprint post-run-cleanup
2014-09-19 01:58:58 -04:00
Matthew Treinish
16dd51b50f Add REVIEWING.rst
This commit adds a file to outline what to look for as a tempest code
reviewer.

Change-Id: I945469842183d0e5022cd41f823d4892adff333f
2014-08-21 20:54:10 -04:00
Jenkins
9d63c38e98 Merge "cleaning up index.rst file" 2014-07-17 08:45:06 +00:00
Matthew Treinish
077a563bb2 Fix warnings from build_sphinx
This commit fixes a couple of issues with build_sphinx in the tempest
docs.

Change-Id: Id8572455616e9be94a8c5794328eb9c1791db5b3
2014-06-04 11:45:14 -04:00
Christian Berendt
7e571e6eea cleaning up index.rst file
Removed notes about the generation of the file.

Change-Id: I70bf5438a8fea9ca4f785313db4975b328f3a749
2014-05-29 17:15:36 +02:00
Matthew Treinish
de8378b65b Remove Whitebox tests
This commit removes the whitebox tests from tempest. The proper place
for this type of testing is in nova unit tests not in tempest.

Fixes bug 1222852

Change-Id: I0fac7b030c51985f9d6d93129bf9bab75c18cd11
2013-09-12 15:40:15 +00:00
Matthew Treinish
32d3570c18 Add unittest framework + tests for wrapper scripts
This commit adds a framework for running unittests on tempest to
verify that tempest works as expect. The first tests added are to
verify the response codes of the test runner wrapper scripts we use
on jenkins jobs.

Change-Id: If8e91238054593999e7b5bc34da499bd0ac02311
2013-08-27 11:55:13 -04:00
Attila Fazekas
6c7e8d9a23 Generate temepst API doc from source
This change adds an rst file for every tempest module, with the
'automodule' macro.

The root of the API documetation will be in the 'API and test cases'
section on the main page.

https://review.openstack.org/39427

Change-Id: I4065a24328dc85bb158b15bdfddb10ee0839916b
2013-08-01 23:14:16 +00:00
Michael Chapman
4410ca1c56 Fix index link in footer bar
bug 1192692

Sphinx isn't configured to load the tempest modules,
so the easiest fix is to remove the links to the
general and module indices (which don't exist).

Change-Id: Ic33e0b6ce56cb31fa51b4cdce25524d40191235d
2013-06-28 10:54:32 +10:00
Attila Fazekas
23fdf1de42 Tempest Coding Guide
* Adding HACKING.rst as Tempest Coding Guide
* Fixing the section/title formater markup's length

Change-Id: Ic61d287cd3bbf3adc5f7d98b17957a5be125f613
2013-06-09 22:02:19 +02:00
Sean Dague
7679b488ed update docs for consistency
adjust the index file to be more readable for the field guide

adjust titles to make more sense when displayed in top level index

change theme to 'nature', which is closer to the openstack default
style. Will eventually write our own style here.

Change-Id: I4f5fed65f9e26a4e4d333b1be4ed02c9ec246b9a
2013-05-24 08:49:44 -04:00
Sean Dague
b56052b475 initial seed for tempest doc directory
This is a first bit of experiment to get a doc tree for tempest
that will publish our doc content to the web as part of a larger
QA doc guide.

Update some of the base READMEs to be a little more accurate,
now that we're going to be pushing them to the web.

Added tox [venv] to ensure that docs get built

Change-Id: Id54e8fbc1d884525087630106e17b37dc1206cc9
2013-05-22 21:06:55 -04:00