Commit Graph

53 Commits (cd147631f8254c2f727ccfc937566a7a200349f9)

Author SHA1 Message Date
Andreas Jaeger f27a334e21 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Update local hacking checks for new flake8, remove vi check, it's now in
hacking as H106.

The tempest plugin is used on older branches as well.
We really only need hacking on this repo anyways,
where we no longer support python 2, so here we
make the requirement specific to python 3.

Change-Id: I3f4216f66606fbc450a46c93de306399b7f3cd65
2020-04-04 10:33:23 +02:00
Zuul 9c71dcfa7c Merge "Bump hacking to 1.1.0" 2018-08-21 12:46:22 +00:00
Felipe Monteiro 4d011af928 Hacking checks for negative test cases
This patchset adds 2 hacking checks for making sure negative
tests have correct conventions applied.

* T117: Check that each negative test has the
  ``@decorators.attr(type=['negative'])`` applied

This patch set adds both hacking checks, adds unit tests
and updates HACKING.rst documentation with both new checks.

Closes-Bug: 1781044
Change-Id: I46df351187d22090861150c84fa0a0c1054ae3d6
2018-07-28 21:43:45 -04:00
Stephen Finucane fc42cc6be7 Bump hacking to 1.1.0
hacking 1.0+ adds a couple of additional checks that are helpful as part
of the transition to Python 3. However, hacking 1.1.0 incorporated
flake8 2.6.x, which transitioned from pep8 to pycodestyle. Tempest
provides some hacking extensions and these use pep8 imports. As things
stand, projects that wish to use these hacking extensions must either
constrain themselves to the same older hacking versions or install pep8
manually. Neither of these approaches is particularly attractive so
instead transition to hacking 1.1.0 ourselves. This has the benefit of
giving us the additional checks for Python 3.

Change-Id: I09e3d2461366bb439bcb817149ce807a232acfbb
2018-07-18 10:42:38 +01:00
Stephen Finucane 7f4a6211bf Mark raw strings as such
This is raising warnings now and will be an error in future versions of
Python 3.

Change-Id: I8827cb36ef2ef85e8f245a2a181fb23ee75bfd16
2018-07-06 14:01:28 +01:00
Federico Ressi 2d6bcaa7e5 Fix code to pass pep8 Tox environment.
Command tox -e pep8 is failing making CI to fail: this fixes it.

Change-Id: I5b1acfc4ba2e0eb16f50bfd1cdaa8edc361923d1
2018-04-12 11:09:37 +02:00
junboli bc2ae8629c Unsupported 'message' Exception attribute in PY3
* Fix unsupported 'message' Exception attribute in PY3
   The 'message' attribute has been deprecated and removed from Python3.
   Use six.text_type(e) instead of e.message. For more details, please check [1]:
 * Add hacking to prevent this from happening in the future.

Change-Id: Id40000c2c453815b04a7d2fd765e19997291d8e3
2017-07-31 07:30:28 +00:00
Ken'ichi Ohmichi f741d0b35a Add T115 for admin test path
Sometimes commiters tried to add tempest tests which require admin
credential under non-admin test path and that caused confusions to
tempest users. This patch adds some coding rule to make test path
clear for the maintenance.

NOTE: This patch adds #noqa to AbsoluteLimitsTests because the test
      class needs force_tenant_isolation which requires admin
      credential indirectly but the test itself is not admin test.
      The history is Id71a705cf9b1dd0c0d41a2fb45ab77c95430a123

Change-Id: Id11eec13f2e431af8bbb83ac4904b2047e7932a7
2017-05-23 14:00:27 -07:00
Shu Yingya 4480707dfb Delete Savanna element in client list
There are not savanna any more.
The Savannaclient are also deprecated, so just delete it!

Change-Id: I3df64eb6ca1f6b6b663b6fe41f678ebc60ffd896
2016-09-21 03:00:15 +00:00
Matthew Treinish c9cbb0f0ec
Remove trove tests from tempest
This commit removes the trove tests from tempest as part of plugin
decomposition of tempest. Trove isn't in scope for testing in tempest
anymore and tempest testing will be done via a plugin now.

Change-Id: I0407e96f628e96bd4d3f7f42cf433b756a5bbaa9
2016-06-27 10:22:11 -04:00
Jenkins 55fd59831c Merge "Add hacking rule to enfore no config in tempest.lib" 2016-06-02 04:57:15 +00:00
Matthew Treinish 59d9eaabdd
Add hacking rule to enfore no config in tempest.lib
This commit adds a hacking rule to enforce that we never add a config
dependency on tempest/lib. Right now we're completely dependent on
reviewers catching this, it is a strong rule so we should ensure we
can't ever land a change that does this.

Change-Id: I1ab1ba52573c6706a50abcd021759c93dd19aa44
2016-06-01 15:05:59 -04:00
Jenkins c2c90574a7 Merge "Remove Ceilometer tempest tests" 2016-05-18 13:27:27 +00:00
Andrea Frittoli (andreaf) 1370bafda0 Remove few refercences left to tempest-lib
A few places in docs / comments still refer to tempest-lib.
Cleaning those up.

Change-Id: I6a15a6e7e58a49a39b3955c60e16d0c196ec154c
2016-04-29 20:30:34 +00:00
ghanshyam e4796f8de4 Remove Ceilometer tempest tests
Now Ceilometer and Aodh tempest plugins are implemented and all
tests and supported code is present in plugins.

NOTE- Need to keep config option 'CONF.service_available.ceilometer'
as it is being used in Congress tests in Kilo and Liberty branch and
Ceilometer tempest plugin is available since Mitaka.
If we remove that then, we will be breaking Congress gate for stable branches.

Change-Id: I0775bcc15dc9cbae6e075fe92f44b5f6c9b9d5d2
Depends-On: Ic0e6b72d8767d92cc63968c442c4ff65bb001cda
2016-04-22 16:24:34 +09:00
Ken'ichi Ohmichi d079c8925d Add pep8 check to use data_utils.rand_uuid()
There is a common method for generating random uuid value.
So it is nice to avoid pointing it out on human review to reduce
reviewing workload.
In addition, this patch removes str() around the method calls
because str() is called in the method.

Change-Id: Ic8caea352d27c479fdb765a699d277b59f1c5bb4
2016-04-19 14:23:00 -07:00
Jenkins a912f4c76a Merge "Enable T108 check for network api tests" 2016-04-14 08:25:14 +00:00
Ken'ichi Ohmichi 1eb3fb6a7d Enable T108 check for network api tests
Since the commit I3a8660838c1af2baefe21bbc57b006c9889ac35a ,
API tests of neutron advanced services have been migrated from
Tempest. So we can enable T108 check for remaining network api
tests to avoid -1 against small thing on human reviewing.

Change-Id: Ide15543c84d9b171d62db28274dbaecb4e880a6e
2016-04-10 16:46:24 -07:00
Ken'ichi Ohmichi 12b28e993b Extend T110 check for lib's service clients
Current T110 check is just working for tempest.service modules, but
we have migrated lib's service clients into tempest.lib.service.
So we need to check these modules also on T110.
This patch extends the check for these modules.

Change-Id: Id102ba359f86bb7417e121e70c707aae36a24740
2016-04-07 15:53:57 -07:00
Ken'ichi Ohmichi 0dc9747519 Add pep8 check for tempest.lib import
tempest.lib should not import local tempest code to avoid circular dependency,
so this patch adds pep8 check to block such kind of code.

Change-Id: I392d28b3195040a800d96171ef275c6e73f9fef4
2016-03-29 11:14:05 -07:00
Matthew Treinish 9e26ca8760
Migrate tempest-lib code into new lib dir
This commit migrates all of the code from tempest-lib as of it's
current HEAD, 6ad0ce42c2791a28125d38b40e7dcddf32dbeed7. The only
changes made to the tempest-lib code is to update the imports and
other references to tempest_lib. Since in it's new home it should
be tempest.lib.

Partially implements bp tempest-lib-reintegration

Change-Id: Iadc1b61953a86fa9de34e285a0bb083b1ba06fa8
2016-02-23 14:15:04 -05:00
Matthew Treinish fc11264f50
Remove zaqar tests from tempest
These tests aren't run in any gating job that I can find on either
tempest or zaqar. Zaqar also falls outside of the scope we set for
projects with in-tree tempest tests. A plugin is being worked in
zaqar change I06053d2d0d761c6398708280f3ef38274dec390c. But,
since these tests are unverified and haven't been run in quite some
time lets just remove them.

Change-Id: I047392de3341169992fc5e64aec1e571059840a8
2016-02-18 20:39:44 -05:00
Ken'ichi Ohmichi f878e6e36d Add network wrapper method checks
The pep8 rules T110/T111 check the method names of service clients,
but the rules could not find violations on network clients because
these clients use some wrapper methods.
This patch adds some checks for covering network clients.

Partially implements blueprint consistent-service-method-names

Change-Id: I4aede28261626a27a3a857ba05c1a0fdcc3cac5b
2016-01-14 00:15:17 +00:00
Ken'ichi Ohmichi b8461cb21f Add T111 hacking rule for consistent DELETE method
As the consistent-service-method-names bp, "DELETE /resources/<id>"
methods' names should be "delete_<resource name>".
So this patch adds it as the hacking rule.

Partially implements blueprint consistent-service-method-names

Change-Id: I36592a91001361d816e302745fe8417aee5e7565
2015-11-20 09:06:26 +00:00
Ken'ichi Ohmichi 5334660423 Refactor T110 rule
We need to implement more rules for the other methods(DELETE/POST/PUT)
for consistent method names. Before doing that, this patch separates
common part from T110 rule for sharing it.

Partially implements blueprint consistent-service-method-names

Change-Id: I24c90d331a45a4f6ad50303c4d17e7cd4d2d784c
2015-11-20 08:08:22 +00:00
Ken'ichi Ohmichi c0d96bec35 Add hacking rule for "GET /resources"
This patch is a prototype for "GET /resources" hacking rule.
black_list_T110.txt file contains the service client files which
are against this rule. So we need to fix them with removing them
from this file.

Partially implements blueprint consistent-service-method-names

Change-Id: I150fe2ef21d4d4d246a46d9baf2fb14cc7d79ee5
2015-11-16 05:32:01 +00:00
John Warren 3059a09e24 Add hacking check for testtools.skip
Adding hacking check for testtools.skip decorator as described
in the bug referenced below.

Closes-Bug: #1490590
Change-Id: I9c456bb8b57a54fbcd1cd5249eec2292916d4ff9
2015-09-02 15:22:10 -04:00
Ken'ichi Ohmichi 80369a96a2 Add a rule for blocking "-" from rand_name call
data_utils.rand_name() appends randam charactors with a hypen like:

 def rand_name(name=''):
     randbits = str(random.randint(1, 0x7fffffff))
     if name:
         return name + '-' + randbits

So it is not necessary to specify a hypen in caller side.
This patch adds a hacking rule for blocking "-" at the end of argument
of rand_name() calls.

Change-Id: I9b9f25dbe5a3ef5ac5900113bcc46e0d911becc9
2015-06-18 00:41:22 +00:00
ghanshyam 50f1947484 Fix doc for usage of python clients in scenario tests
Scenario tests has been migrated from official python clients to
Tempest clients.
Documents for the same needs to be fixed.

This patch fix the README & HACKING file for above changes.

This patch also extends hacking rule of not import python clients
for scenario tests too.

Change-Id: Ieb19a2c0b09f00fb3d4f6c7c73541275a4cf24ae
2014-11-26 17:04:37 +09:00
Andrea Frittoli 41fa16d214 Hacking rule to forbid resource unsafe fixtures
Extend the existing T105, which was missing checks for
tearDownClass anyways. Forbid overriding setUpClass
and tearDownClass except for tempest/ where the
test base class is defined.

To be able to enforce the rule, fixing setUpClass
which was added with new tests before this patch could

Partially-implements bp resource-cleanup

Change-Id: Ib4d98c2ff8776ea1379a044b5a30fb02e351ce75
2014-10-07 10:30:27 +01:00
Andrea Frittoli 486ede779b Drop OfficialClientManager and references to it
OfficialClientManager is unused now that scenario tests use
clients.Manager instead. Dropping the class, the hacking
rule that forbade its use in API tests and related unit tests.

Change-Id: Ib359b9ab0a903436d9d9c265e8c4d82bdd80a496
2014-10-01 11:27:02 +01:00
Ghanshyam 2a180b8398 Fix "mutable" object as default value
This patch fix the coding issue where "mutable" objects like list, dict
are being used as default value of param.

Current functions with mutable object as default value can create issues
if they will be called in more random order.
For example - first call with non default value and then with default
value. In this case original default will be overridden by the first
function call (as mutable object keep their state among function calls).

This commit also add the hacking check for the same and its
corresponding test.

Closes-Bug: #1330322

Change-Id: I251b316ef6d37f4b95c5e8d73a20a39005c22870
2014-09-01 11:13:22 +09:00
Malini Kamalambal 8681e92a66 Rename Marconi to Zaqar
Due to copyright issues Marconi is renamed to Zaqar. This patch updates
all Marconi references to Zaqar.

Change-Id: Ib37b95d99bbfc666e1a3a17174d87804ecd0649e
2014-08-18 10:10:45 -04:00
Jenkins ff72f45628 Merge "Add a hacking check to block scenario manager in api tests" 2014-07-02 07:00:57 +00:00
Matthew Treinish b12ad76433 Fix T104 for scenario test subdirs
This commit fixes the T104 hacking check to handle subdirs in the
scenario tests. Previously the check assumed that all the scenario
tests were in the scenario dir. However, this isn't necessarily the
case. For example, the heat scenario tests are all in the path:
tempest/scenario/orchestration which means they were excluded from
the T104 check. This commit fixes the oversight and adds unit tests
to cover this condition.

Change-Id: I42d7f22f6194f624f8488493de29e60ff19d211a
2014-06-19 10:18:05 -04:00
Matthew Treinish 7acaba44c4 Add a hacking check to block scenario manager in api tests
This commit adds a new hacking check to ensure that the manager class
from the scenario tests, OfficialClientManager, isn't used in the API
tests. There is already a hacking rule preventing imports of the
official clients, T102. But, the official clients could still be used
through the scenario tests which wouldn't be caught by T102. This
attempts to close that gap by adding a new hacking check for

Change-Id: I608a1319c136118947e3df74fa88f7124011caa4
2014-06-10 18:11:16 -04:00
Matthew Treinish aaa3595549 Add noqa support to no_setupclass_for_unit_tests
This commit adds support for using noqa with the T105 hacking check.
This is need to unit test the _find_caller() utility with setupClass
in the call stack.

Change-Id: I5c1a07d019b79b082cb895a17364f7e7cb2f08fe
2014-05-02 19:14:28 -04:00
Jenkins 2e22f02f66 Merge "Add check that service tag isn't in path name" 2014-04-25 00:28:41 +00:00
Matthew Treinish d75edef97f Move the test_utils module to scenario test dir
The test_utils module in tempest.common.utils is only useful for
scenario tests. So it's a little confusing to have it in the common
directory. This commit corrects this by moving the file into the
scenario directory. The service tag requirement for scenario tests
hacking check is updated to only check scenario test files, because
a method in the new utils module was being picked up incorrectly.

Change-Id: I1d70745a5ee07923e680328e5b8a29bfb82d8099
2014-04-12 19:25:27 -04:00
Matthew Treinish 662bc3cf77 Add check that service tag isn't in path name
This commit adds a new hacking check to ensure that a service tag
isn't added on a test whose path includes the service tag being added.
Having a service tag added where the path already contains the service
name is redundant.

Partially implements bp add-service-tags

Change-Id: I88b3005d05e17751045d766efb1e965822a3c50e
2014-04-09 13:24:26 -04:00
Matthew Treinish 7481350fca Add sahara to list of clients for T102
Change I6cb693298c75e400e5fe3a4bdd2532bca9d87b9b updated the list of
clients to include the missing official clients. However, at roughly
the same time savanna was renamed to sahara, so it wasn't included in
the list of clients. This commit corrects the oversight by adding
sahara to list of clients to check for T102.

Change-Id: Icfaad37b5f3b03f112ad6eb2fdedde38013287dd
2014-03-24 10:43:22 -04:00
Matthew Treinish 7d710f93d6 Add missing client names to T102 hacking check
This commit updates the list of client names that should be checked
for T102.

Change-Id: I6cb693298c75e400e5fe3a4bdd2532bca9d87b9b
2014-03-15 21:41:00 -04:00
Masayuki Igawa fcacf96204 Introduce T106 rule for vi modelines
We don't need to have the vi modelines in each source file anymore.
We've already fixed them several times.
However, newly some files still have it in its header. So we should
check it automatically with our HACKING rule.
This commit introduces T106 rule for that.

Note: This code is copied from Nova's hacking rule.
   Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6

Change-Id: I5c94ef041a39c2377ba6321ace8934f324287bcf
Closes-Bug: #1229324
2014-02-20 11:42:35 +09:00
Masayuki Igawa e8d31a0b80 Remove vim headers
We don't need to have the vim headers in each source files. We can set
it in our vimrc file instead. Also if set incorrectly, gate (pep8 check)
will catch the mistakes.

More discussions:

Change-Id: I43703e2289212389c7841f44691ae7849ed1f505
Closes-Bug: #1229324
2014-01-20 11:22:45 +09:00
Matthew Treinish ecf212c49f Update tempest hacking regarding unit tests
This commit just updates some of the hacking documentation around unit
tests. It also adds a new hacking rule to ensure that setUpClass isn't
used for the unit tests.

Partially implements bp unit-tests

Change-Id: Ie8b1c9f1312a467265d53bc28ee905fa1b5fbb53
2013-12-06 20:00:13 +00:00
Giulio Fidente 83181a9703 introduces skip_because decorator
this change introduces a skip_because decorator which accepts two args
a bug and a condition; also updates the skip tracker accordingly

Change-Id: If53f2ef81d6bddbce284267216254b467046855f
2013-10-04 15:48:28 +02:00
Matthew Treinish 6ba951a987 Add hacking check for service tags in scenario
This commit adds a hacking check to ensure that all the tests in the
tempest.scenario directory include a @services decorator.

Part of bp add-service-tags

Change-Id: Ie2f3c384aac568ab791f4d7665d29ea9b3fda7f8
2013-09-12 14:07:20 +00:00
Matthew Treinish 32d3570c18 Add unittest framework + tests for wrapper scripts
This commit adds a framework for running unittests on tempest to
verify that tempest works as expect. The first tests added are to
verify the response codes of the test runner wrapper scripts we use
on jenkins jobs.

Change-Id: If8e91238054593999e7b5bc34da499bd0ac02311
2013-08-27 11:55:13 -04:00
Mark McClain f2982e8544 update Quantum to Neutron
Update to use the neutronclient library and replaces Quantum references within
the repository.

implements bug: 1197208

Change-Id: Ib3d52e3d9671fe4b30c4bcf76b5e7c2ee9de4664
2013-07-07 00:15:57 -04:00
Sean Dague 973bc99bf3 actually enable our no tempest/tests check
I'm stupid and forgot to enable this, actually turn it on.

Change-Id: I050334bd0f0e5e282e6485b52aaf1e74eed9c7c9
2013-06-15 07:57:32 -04:00