26 Commits

Author SHA1 Message Date
Matthew Treinish
550788819a Add a test documentation section to the docs
This commit adds a new section the HACKING and REVIEWING docs. It
explains the guidelines around inline documentation when
contributing new tests.

Change-Id: Ib56723d0c748baf41439975102d1363f81085f50
2014-09-23 10:04:40 -04:00
Ghanshyam
2a180b8398 Fix "mutable" object as default value
This patch fix the coding issue where "mutable" objects like list, dict
are being used as default value of param.

Current functions with mutable object as default value can create issues
if they will be called in more random order.
For example - first call with non default value and then with default
value. In this case original default will be overridden by the first
function call (as mutable object keep their state among function calls).

This commit also add the hacking check for the same and its
corresponding test.

Closes-Bug: #1330322

Change-Id: I251b316ef6d37f4b95c5e8d73a20a39005c22870
2014-09-01 11:13:22 +09:00
Marc Koderer
313cbd554d Update documentation for negative testing
Since generate_scenario() call is not needed any longer update the
documentation to the current state.

Change-Id: I34e8b974d5c4851a27f53a20778a181985ff6b5b
2014-03-26 10:08:42 +01:00
Jenkins
47e3ac11fd Merge "Adapt documentation for negative testing" 2014-02-25 08:45:49 +00:00
Masayuki Igawa
fcacf96204 Introduce T106 rule for vi modelines
We don't need to have the vi modelines in each source file anymore.
We've already fixed them several times.
 https://review.openstack.org/#/c/66507/
 https://review.openstack.org/#/c/68552/
 https://review.openstack.org/#/c/69318/
 https://review.openstack.org/#/c/70133/
However, newly some files still have it in its header. So we should
check it automatically with our HACKING rule.
This commit introduces T106 rule for that.

Note: This code is copied from Nova's hacking rule.
   Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6

Change-Id: I5c94ef041a39c2377ba6321ace8934f324287bcf
Closes-Bug: #1229324
2014-02-20 11:42:35 +09:00
Marc Koderer
a5afb4fc3e Adapt documentation for negative testing
With the introduction of the new negative testing framework the workflow
for new test has changed.

Partially-implements: bp negative-tests
Change-Id: I522c2be227a120a6a714349fa29b31e60b5e0bf4
2014-02-11 19:02:47 +01:00
Bruce R. Montague
44a6a194ff Fix three accidentally formatted paragraphs
Three paragraphs are misformated in the output due to
a leading space in a line. The leading space causes a
line break and the indent of the line.

Change-Id: I5c0d03a47a2ca305b2b077068df26fadccd72212
2013-12-17 09:06:04 -08:00
Matthew Treinish
ecf212c49f Update tempest hacking regarding unit tests
This commit just updates some of the hacking documentation around unit
tests. It also adds a new hacking rule to ensure that setUpClass isn't
used for the unit tests.

Partially implements bp unit-tests

Change-Id: Ie8b1c9f1312a467265d53bc28ee905fa1b5fbb53
2013-12-06 20:00:13 +00:00
Matthew Treinish
6eb0585f70 Improve the UX around sample config generation
This commit just adds some documentation around using the sample config
generator. It also adds a wrapper script that sets the env variable for
running the oslo generator script with tempest.

Change-Id: I860b17defeeb85174e68b1e8b046b0e43c30683b
2013-11-26 15:47:52 +00:00
Marc Koderer
31fe483851 Add additional documentation for stress tests
In order to encourage people to add more and more stress tests we
need more documentation about it. This documents the stresstest
decorator and gives a global overview.

Change-Id: I65294fb278faba7a0d324782bf50e2fdedc5c1de
2013-11-11 08:16:03 +01:00
Matthew Treinish
8b79bb395c Add section for negative tests to HACKING.rst
This commit adds a new section to HACKING.rst to outline the
guidelines for adding negative tests to tempest.

Change-Id: Ieb3fc3670bb9229c6791e5c8c0a7ac4fbde03c4d
2013-10-10 17:11:05 -04:00
Giulio Fidente
83181a9703 introduces skip_because decorator
this change introduces a skip_because decorator which accepts two args
a bug and a condition; also updates the skip tracker accordingly

Change-Id: If53f2ef81d6bddbce284267216254b467046855f
2013-10-04 15:48:28 +02:00
Matthew Treinish
97072c8110 Update style guide link in HACKING.rst
The previous github HACKING.rst doesn't exist anymore, so this commit
just updates the link in tempest's HACKING.rst to point it to the
correct location.

Change-Id: If49f938528ac9059a8e1983b281081fc6137b700
2013-10-01 11:54:15 -04:00
DennyZhang
900f02b34f Fix typos in tempest/HACKING.rst Edit
Change-Id: I33f5872cc945ee3c6a720e0a7fd5b8140fc14794
Close-Bug: 1229189
2013-09-23 12:12:06 -05:00
Matthew Treinish
96c28d13e7 Add parallel test execution section to HACKING.rst
This commit adds a section to HACKING.rst to add some things to watch
out for now that we are running tempest in parallel by default. Part
of this documentation was taken from David Kranz's mailing list post:
http://lists.openstack.org/pipermail/openstack-dev/2013-August/014472.html

Change-Id: Ie688650fc528cf849c619e9afb64ba1f6b556ff8
2013-09-16 17:26:26 +00:00
Matthew Treinish
5e4c0f2a31 Add service tag section to HACKING.rst
This commit just adds a section to HACKING.rst explaining how to use
service tags.

Part of bp add-service-tags

Change-Id: I9724cec8e2009e9c2eb2d75d9254c2b5f10e7a85
2013-09-12 14:07:37 +00:00
Attila Fazekas
7899d312ad Updating HACKING.rst
* Adding a test case independence rule
* assertTrue/assertFalse msg argument
* Encouraging  matcher usage for assertion

Change-Id: I89612a0dd66513f10e8bf53632f6a3d81ccf4129
2013-08-26 22:15:59 +02:00
Attila Fazekas
58d23301ba Fixing man page generation
The man page generation failed with sphinx v1.1.3
and docutils 0.10 and 0.11.

Adding docutils==0.9.1 to the test-requirements.txt
in order to solve this issue.

The docutils==0.9.1 is on the common OpenStack dependency list.

Several doc creation warning and error also fixed by this change.

Change-Id: Icb412876f5a989e6b8aa4886f2a246127a76521a
2013-07-24 13:49:14 +02:00
Chris Yeoh
c2ff727a73 Updating HACKING with some test writing recommendations
Adds a recommendation against submitting changesets with only
skipped tests and some recommendations for writing tests.

Change-Id: Iaf0accdcd1d5e37658ca56528b1709d0a00928d2
2013-07-22 22:34:28 +09:30
Joe Gordon
1374f88fbf Cleanup and make HACKING.rst DRYer
Reference the OpenStack hacking guide in HACKING.rst and remove
duplicate entries.  Add section for tempest specific rules.

Change-Id: Idea236dfa7430fb49892c0d677f6f18175c1249d
2013-07-12 17:00:34 +01:00
Attila Fazekas
10fd63dfb3 Add exception handling doc
Adding  basic exception handling guideline to the HACKING.rst

Change-Id: I6c3bb2f5a7f7b8947193a005fc2fffe57ae0f0ba
2013-07-04 19:04:49 +02:00
Attila Fazekas
23fdf1de42 Tempest Coding Guide
* Adding HACKING.rst as Tempest Coding Guide
* Fixing the section/title formater markup's length

Change-Id: Ic61d287cd3bbf3adc5f7d98b17957a5be125f613
2013-06-09 22:02:19 +02:00
Matthew Treinish
997da922c9 Update hacking.py for @testtools.skip() formatting.
This commit updates the hacking rules to add a strict format for bug
skips. Previously, there was no defined rules for skip formatting
which caused a number of test skips to be added without consistent
formatting. These skips then failed to get picked up by
tools/skip_tracker.py. This commit adds a new hacking test to ensure
that any skips added conform to a format that the skip_tracker will pick
up. HACKING.rst was also updated to explain the new rules being enforced.

Change-Id: I95f3ec7de2ee5e2039d53ad9565b5cec936a7672
2013-03-22 15:40:59 -04:00
Matthew Treinish
89273ee574 Update HACKING file.
This updates the hacking file to include notes about
openstack-common and update OpenStack LLC to the Openstack
Foundation

Change-Id: I2cee5f16b9e1f69ec5a44b76c4cc053cabfc9197
2013-02-12 13:52:09 -05:00
ivan-zhu
1feeb38e17 Use testtools as the base testcase class.
Use testtools as the base unittest test case class towards to
parallelizing the test runs. Catch places where setUp and tearDown
were not being chained properly, and fix these problems.Remove
custom skip, using testtools.skip instead. Add TestResourceManager.

Part of blueprint speed-up-tempest

Change-Id: I8e7b5686b22969a0f3db96100a357c93a4d5c03f
2013-01-28 13:50:58 +08:00
Matthew Treinish
8b37289f68 Improve pep8 checks to be similar to those in nova
This adds hacking.py, based on the one in nova, to wrap around
pep8 and enforce more stringent checks on tempest. This adds
the rules that nova is currently enforcing on incoming commits.

The rules that pep8 is ignoring are copied from the pep8 check on
jenkins.

The HACKING file was also updated to indicate the inclusion
of the new rules that are being enforced.

Change-Id: I5c85829ee15d4264d8de2d9f0207d9dd3dfbbff2
2012-12-12 11:49:30 -05:00