Commit Graph

94 Commits

Author SHA1 Message Date
Ghanshyam Mann
da3bb61f1f Add index page for plugins documents
Now we have more documents for plugins on stable release,
support and their testing policy. Let's prepare a
index page for it will be easy for plugins maintainer.

Also renaming the stable support policy section to have a clear
name.

Adding redirect links for moved documents.

Change-Id: I17a9a6e0e88605a0ee5047678d6ed168c31396ec
2020-05-05 02:12:05 +00:00
Andreas Jaeger
f27a334e21 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Update local hacking checks for new flake8, remove vi check, it's now in
hacking as H106.

The tempest plugin is used on older branches as well.
We really only need hacking on this repo anyways,
where we no longer support python 2, so here we
make the requirement specific to python 3.

Change-Id: I3f4216f66606fbc450a46c93de306399b7f3cd65
2020-04-04 10:33:23 +02:00
caoyuan
349ba75410 Replace git.openstack.org URLs with opendev.org URLs
Thorough replacement of git.openstack.org and review.openstack.org URLs
with their opendev.org counterparts.

Change-Id: I88e894db7b854d32593c770f5aa9b8a91fad7866
2019-05-23 10:19:31 +00:00
Attila Fazekas
c07edb5f80 Mention autopep8 in Hacking.rst
The scripts in the tools now is able to distinguish,
the autopep8 error codes better.

Change-Id: I9ea19009118ed418e3666fad45d2e0eb676204c8
2019-03-29 20:50:23 +00:00
Sergey Vilgelm
eac094a8cf
docs: Fix incorrect docstrings in reStructuredText format
Several docstrings have incorrect format, what prevents to generate
a documentation using sphinx-apidoc extension.

Change-Id: I66431c7030089173d7635c1e2a72765db21f9780
2018-11-21 19:06:43 -06:00
Zuul
8cdc064f69 Merge "docs: Include information about the list of supported projects" 2018-11-20 09:50:38 +00:00
sunqingliang6
8606c83b4b change http links to https links
Change-Id: I9875b8db497129baebadc8e74cf602ea1c063cbd
2018-11-09 14:26:19 +08:00
Felipe Monteiro
f22e6ec651 docs: Include information about the list of supported projects
This documentation is included in HACKING at the top to let
newcomers know that Tempest's testing scope is confined to the
following projects:

* Cinder
* Glance
* Keystone
* Neutron
* Nova
* Swift

The user is directed to [0] for additional Tempest plugin
repos where they can add their tests for other projects. Note
that [0] currently appears to not be rendering correctly,
but this is an unrelated issue.

[0] https://docs.openstack.org/tempest/latest/plugin-registry.html#detected-plugins

Change-Id: I800a3f6ac4c347d4322574f649cb052070127db6
2018-11-06 01:21:17 -05:00
zhufl
42bcb55faf Fix docstring example in HACKING
I61ec91ba4a17c6797f1c626c9b09e4fdb381c357 moves scenario
docstring under the relevant test method, but the reference
to that docstring kept unchanged in HACKING.rst. Besides,
this is no longer a good example of "class level docstring",
so we just pick another one for the example.

Change-Id: I313da5627d283ec4771907d83c485db103fa2702
2018-09-18 11:24:00 +08:00
Felipe Monteiro
4d011af928 Hacking checks for negative test cases
This patchset adds 2 hacking checks for making sure negative
tests have correct conventions applied.

* T117: Check that each negative test has the
  ``@decorators.attr(type=['negative'])`` applied

This patch set adds both hacking checks, adds unit tests
and updates HACKING.rst documentation with both new checks.

Closes-Bug: 1781044
Change-Id: I46df351187d22090861150c84fa0a0c1054ae3d6
2018-07-28 21:43:45 -04:00
Felipe Monteiro
46920b8af9 docs: Add HACKING information about test attributes
Tempest uses test attributes [0] in many places for different
reasons, but there is no documentation to explain why.
For example, Tempest uses these attributes to customize
Zuul jobs to selectively run only slow scenario tests. Tempest
also has a tox entrypoint for just running smoke tests.

Further, there is no documentation explaining:

* what is a smoke test
* what is a negative test
* what is a slow test

And more importantly: When should each attribute be applied?

So this information is not just useful for beginners (helping
them understand the different types of testing and what they
mean) but also for users who would like to understand how to
correctly apply attributes to new tests in either Tempest
or Tempest plugins.

[0] https://testtools.readthedocs.io/en/latest/for-test-authors.html?highlight=attr#test-attributes

Change-Id: I7be0bebaab56a418616bea01e90a437b23499234
2018-07-09 23:58:20 -04:00
ghanshyam
247841ecf6 Move Stable branch Policy Doc outside of coding guide
Current doc explaining the stable branches support policy is under
coding guide. But support policy should be at higher level which
can be useful for user or maintainer of Tempest.

Change-Id: Id46e14080559062b4d89e5f7a9acbcd5df09691e
2018-06-20 04:53:16 +00:00
Matthew Treinish
210f18464d Document branch support policy for extended maint branches
This commit adds a note to the documentation to clarify which branches
tempest will support under the new extended maintenance stable branch
policy. The previous policy of supporting all active branches will not
scale under the new policy as we do not have enough resources to gate
all propsed changes against more than 3 stable branches at a time. With
the addition of this to the documentation we'll clearly define what
branches are in scope for testing of tempest and how/when we should
handle dropping support for a branch.

Change-Id: Ide6dc56b8d37daf6fb26d6a2f20dbf61f06c0998
2018-05-31 16:08:10 +00:00
Felipe Monteiro
356f0593ff Docs: Elaborate on feature flag in HACKING file
This is to elaborate on feature flag functionality in Tempest's
HACKING.rst. Currently, there isn't much documentation on feature
flags in Tempest or how to add them. The only place feature
flag is mentioned is in this file. Since testtools.skip(Unless|If)
is a common way of using a feature flag, it is mentioned. Also,
Tempest's interaction with DevStack is an important thing to mention
when explaining how to introduce a new feature flag.

Change-Id: Ia47132fa596918e58f21ba9810c2c28ddcf0d584
2018-03-30 19:47:10 +00:00
gaofei
6ec582f111 Replace curly quotes with straight quotes
Curly quotes usually input from Chinese input method.
When read from english context, it makes some confusion.

Change-Id: I6209f92de2a5b136c2c16decfe5752f5c44096ff
2018-01-26 05:56:28 +00:00
Masayuki Igawa
bbbaad68c3
Fix docs markup consistency
This commit fixes docs consistency about markups, mainly. Originally,
some of command options were written with strong emphasis. However,
double dashes are converted to a single dash with it. And this commit
also fixes some inconsistencies and weird indentations.

Change-Id: Iff1e8e320dcb1fa69ca0fce139c58727fca7b729
2017-11-21 19:02:54 +09:00
Masayuki Igawa
b78b923e5a Fix indentation in docs
This commit fixes indentation in tempest docs. These indentations are
not necessary and it causes a weird html outputs.

Change-Id: I9c8714558a3327b7ad0b0ab0d3fdc7e770c3c75b
2017-11-20 07:39:09 +00:00
davyyy
ac670dce6d Update URL from "http" to "https"
Most of them can still be visited through http, but the following
one is necessary to update, so I change them all by this chance.

 $ git clone http://git.openstack.org/openstack/tempest

Change-Id: I2b4da0670b599666e7151bddfff0fa6655640bc4
2017-11-16 21:55:52 +08:00
mmkmmk57
ce3bb9ba1f Fix the 'service' decorator path in doc
Documentation update.

Change-Id: I3bca9de9a2e3fbc12647a1c55be0efc4434df2c0
2017-09-21 06:58:59 +00:00
Jenkins
43afce6b8c Merge "Make resource_cleanup stable" 2017-09-13 17:54:24 +00:00
Andrea Frittoli
3be574898c Make resource_cleanup stable
Add docstrings, unit tests and an helper for class resource
cleanup. Check super's resource_cleanup is invoked when
overriden to ensure the cleanup stack is processed.

Change-Id: I9c89ba4efd715634dde6b1182c2025ddf9c2f7d2
2017-09-12 12:37:30 -06:00
junboli
477fd02fb8 doc migration: update the doc link address
In the file HACKING.rst and README.rst, there still left
some bad link. This change is to fix the doc link brought
by the doc migration.

Change-Id: I7a60a5821c199afb75a47ef24412470c16991d77
2017-09-06 17:25:11 +08:00
Andrea Frittoli
cd36841ca2 Move test decorators to common
services and extension decorators are based on CONF and needed by
Tempest only, so moving them to tempest.common.
Since a few plugnis use them today, use debtcollector to avoid
breaking all dependencies.

Change-Id: Ibd52153d00b8e60fb8c89e38d94e358ddc787251
2017-08-23 17:09:33 +00:00
junboli
bc2ae8629c Unsupported 'message' Exception attribute in PY3
* Fix unsupported 'message' Exception attribute in PY3
   The 'message' attribute has been deprecated and removed from Python3.
   Use six.text_type(e) instead of e.message. For more details, please check [1]:
   [1] https://www.python.org/dev/peps/pep-0352/
 * Add hacking to prevent this from happening in the future.

Change-Id: Id40000c2c453815b04a7d2fd765e19997291d8e3
2017-07-31 07:30:28 +00:00
Jenkins
c7a403debd Merge "Doc: fix markups, capitalization and add 2 REVIEWING advices" 2017-07-22 01:47:46 +00:00
chenxing
e98720a22f Update the documention for doc migration
Change-Id: Ie6d322d1b9c93118af7c74986993010b9f4b76b2
2017-07-19 03:42:23 +00:00
Jordan Pittier
74a56aba03 Doc: fix markups, capitalization and add 2 REVIEWING advices
I've been reading our entire documentation to see where it could
be improved. It's guide good actually. While reading I've fixed
some typos, added some capitalization to project names, mostly
trivial stuff.

What's worth reviewing is the 2 paragraphs I added to the REVIEWING
guidelines.

Change-Id: I977de335119f4ff7b9aedcfbed31c264ed531ea8
2017-07-11 20:26:32 +02:00
Jenkins
345dc6cda4 Merge "Add T115 for admin test path" 2017-05-30 10:19:51 +00:00
jeremy.zhang
c0f9556c7a Use data_utils in tempest.lib.common.utils
Tempest tests should use the stable method defined in tempest.lib.
This patch is to use data_utils from tempest.lib.common.utils in
OAUTHConsumersV3Test and HACKING.rst.

Change-Id: Ia9f4cf37d8d57da34864a5eb55a8fe9d9c5722f5
2017-05-26 13:41:57 +08:00
Ken'ichi Ohmichi
f741d0b35a Add T115 for admin test path
Sometimes commiters tried to add tempest tests which require admin
credential under non-admin test path and that caused confusions to
tempest users. This patch adds some coding rule to make test path
clear for the maintenance.

NOTE: This patch adds #noqa to AbsoluteLimitsTests because the test
      class needs force_tenant_isolation which requires admin
      credential indirectly but the test itself is not admin test.
      The history is Id71a705cf9b1dd0c0d41a2fb45ab77c95430a123

Change-Id: Id11eec13f2e431af8bbb83ac4904b2047e7932a7
2017-05-23 14:00:27 -07:00
Hai Shi
6f52fc599a [DOC] Fix details of tox
We need put a space between a param's name and a param's value in tox
and it would be more clear.
eg: from `tox -epy35` to `tox -e py35`.

Change-Id: I86750f42669bb02dcfac2b62e294d557bf44bd4c
2017-04-07 00:47:00 +00:00
Masayuki Igawa
5a3ad34fe5
Change API-WG guideline link to specs.o.o
This commit changes API-WG guideline link to spec.openstack.org instead
of github.com. We should use openstack.org links as possible.

Change-Id: Id0265bd6136a0c6f4adedf5c3aef34d34bbc933e
2017-03-22 16:27:53 +09:00
Ken'ichi Ohmichi
8a08211b4b Drop allowing old test.idempotent_id
test.idempotent_id should not be allowed anymore on the check
to block containing such wrong usage into tempest anymore.

Change-Id: Ie45b5d4a28b645ad5145ac37decf70f2f08d4229
2017-03-06 16:08:28 -08:00
Jordan Pittier
2e7ae7c6e5 Remove the Stress framework
It's not used anymore. There was general consensus in Feb 2016
to deprecate it (see [1]) and remove it in Newton.

[1]  [qa] deprecating Tempest stress framework

Change-Id: Ib229985ea2a1fee495c9492c9ce1781e6bac1dc6
2016-10-19 14:31:13 +02:00
Ken'ichi Ohmichi
8db407576b Clarify the guideline of negative tests
We have discussed negative tests at OpenStack Summit Austin before
as https://etherpad.openstack.org/p/newton-qa-negative-testing

The conclusion is
* Keep the existing negative tests
  Tempest should not cover all negative test cases as integration tests.
  Such tests should be covered with function tests in each projects.
  Function tests are implemented in Nova, [Ceilometer -> Gabbi], Barbican
  36 negative tests are used in Refstack as required tests and they are valuable

So we need some guideline about negative tests for our future, and this
patch adds it.

Change-Id: I2ac1a4cf91c9357caca2d490e6bc15699b7f3dad
2016-10-04 15:52:22 -07:00
Chris Hoge
2b47841c4d Add guidance on negative tests in HACKING.rst
Adds guidance on negative tests based on work items from the Newton
design summit in Austin[1].

[1] https://etherpad.openstack.org/p/newton-qa-negative-testing

Change-Id: Ic44b0e17540a5e984006a4a74c8318ad7e74db31
2016-06-23 16:05:01 -07:00
Jenkins
55fd59831c Merge "Add hacking rule to enfore no config in tempest.lib" 2016-06-02 04:57:15 +00:00
Matthew Treinish
59d9eaabdd
Add hacking rule to enfore no config in tempest.lib
This commit adds a hacking rule to enforce that we never add a config
dependency on tempest/lib. Right now we're completely dependent on
reviewers catching this, it is a strong rule so we should ensure we
can't ever land a change that does this.

Change-Id: I1ab1ba52573c6706a50abcd021759c93dd19aa44
2016-06-01 15:05:59 -04:00
Masayuki Igawa
e63cf0fedf Fix docs errors and warnings
This commit fixes docs errors and warnings. However, 'WARNING: nonlocal
image URI found' is still remaining because it needs an ugly workaround
that is replacing it to raw html.

Change-Id: I4524c1ff4126c87979301d0fb8ac7c9eb6f2f708
2016-05-25 10:25:21 +09:00
Franklin Naval
e8896de357 Docs: Fix Hacking guide bulleting
* fix list formatting of fixtures section

Change-Id: I9b2c11c59c34cae0086e2af507aa85313b8480bc
Closes-Bug: #1582350
2016-05-16 15:21:00 -05:00
Jenkins
43d742dd6d Merge "Remove negative test framework documentation" 2016-04-30 19:12:43 +00:00
Andrea Frittoli (andreaf)
1370bafda0 Remove few refercences left to tempest-lib
A few places in docs / comments still refer to tempest-lib.
Cleaning those up.

Change-Id: I6a15a6e7e58a49a39b3955c60e16d0c196ec154c
2016-04-29 20:30:34 +00:00
Luz Cazares
e28c18f656 Remove negative test framework documentation
We have a conclusion to remove the description of negative test
framework during Austin summit, details at:
https://etherpad.openstack.org/p/newton-qa-negative-testing

Change-Id: Ia3d36b8d1c8e9721b88dbc35bc350af822b8dc6f
2016-04-29 13:25:26 -07:00
Ken'ichi Ohmichi
d079c8925d Add pep8 check to use data_utils.rand_uuid()
There is a common method for generating random uuid value.
So it is nice to avoid pointing it out on human review to reduce
reviewing workload.
In addition, this patch removes str() around the method calls
because str() is called in the method.

Change-Id: Ic8caea352d27c479fdb765a699d277b59f1c5bb4
2016-04-19 14:23:00 -07:00
Sean Dague
ed6e586b9f deprecate use of tenant in configs
OpenStack has settled (a very long time ago) on project to represent
the container which has users. We should make tempest configs always
use project versions of these names. This deprecates the old tenant
versions of these names, and makes project version the prefered
option.

Change-Id: Ied68c0326cf891a4e6eb3db51dc5f43241e4b8d2
2016-04-06 22:26:18 +01:00
Ken'ichi Ohmichi
0dc9747519 Add pep8 check for tempest.lib import
tempest.lib should not import local tempest code to avoid circular dependency,
so this patch adds pep8 check to block such kind of code.

Change-Id: I392d28b3195040a800d96171ef275c6e73f9fef4
2016-03-29 11:14:05 -07:00
Ken'ichi Ohmichi
4f525f7cca Add T111 history to HACKING.rst
On the commit I36592a91001361d816e302745fe8417aee5e7565 , I forgot
to add T111 history to HACKING.rst. So this patch fixes it.

Change-Id: Ib4fd6e6d6f20cbe6c8b2914319aca721fb223850
2016-03-25 15:21:10 -07:00
Matthew Treinish
c1802bc956
Use the idempotent id and tooling from tempest-lib
The idempotent id decorator and the check-uuid script were migrated to
tempest-lib as part of tempest-lib 0.10.0. However, tempest was never
updated to use the library version of idempotent id or check-uuid.
This commit updates tempest to remove it's local copies of the
idempotent_id decorator and the check-uuid tool and use them from
tempest-lib instead.

Also, instead of updating the import of every single test module in
tempest an alias is just added to tempest.test to let all the old
imports work seamlessly. Updating the imports and removing the alias
may or may not be worth it in the future, but that will be a follow
on patch or discussion.

Change-Id: Iae3f491ada8f7b55c1fd817e033d968c2102d11e
2015-12-03 19:20:44 -05:00
Ken'ichi Ohmichi
c0d96bec35 Add hacking rule for "GET /resources"
This patch is a prototype for "GET /resources" hacking rule.
black_list_T110.txt file contains the service client files which
are against this rule. So we need to fix them with removing them
from this file.

Partially implements blueprint consistent-service-method-names

Change-Id: I150fe2ef21d4d4d246a46d9baf2fb14cc7d79ee5
2015-11-16 05:32:01 +00:00
Andrea Frittoli (andreaf)
17209bb083 Rename isolated creds to dynamic creds
Renaming the isolated_creds module to dynamic_creds module, and rename
the IsolatedCreds class to DynamicCredentialProvider in preparation
to migration to tempest-lib.

Partially implements bp tempest-library

Change-Id: I78a4884e980ef7d0103639cb3792a54c69fb7761
2015-10-06 15:02:01 +01:00