The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix problems found.
Update local hacking checks for new flake8, remove vi check, it's now in
hacking as H106.
The tempest plugin is used on older branches as well.
We really only need hacking on this repo anyways,
where we no longer support python 2, so here we
make the requirement specific to python 3.
Change-Id: I3f4216f66606fbc450a46c93de306399b7f3cd65
This patchset adds 2 hacking checks for making sure negative
tests have correct conventions applied.
* T117: Check that each negative test has the
``@decorators.attr(type=['negative'])`` applied
This patch set adds both hacking checks, adds unit tests
and updates HACKING.rst documentation with both new checks.
Closes-Bug: 1781044
Change-Id: I46df351187d22090861150c84fa0a0c1054ae3d6
hacking 1.0+ adds a couple of additional checks that are helpful as part
of the transition to Python 3. However, hacking 1.1.0 incorporated
flake8 2.6.x, which transitioned from pep8 to pycodestyle. Tempest
provides some hacking extensions and these use pep8 imports. As things
stand, projects that wish to use these hacking extensions must either
constrain themselves to the same older hacking versions or install pep8
manually. Neither of these approaches is particularly attractive so
instead transition to hacking 1.1.0 ourselves. This has the benefit of
giving us the additional checks for Python 3.
Change-Id: I09e3d2461366bb439bcb817149ce807a232acfbb
* Fix unsupported 'message' Exception attribute in PY3
The 'message' attribute has been deprecated and removed from Python3.
Use six.text_type(e) instead of e.message. For more details, please check [1]:
[1] https://www.python.org/dev/peps/pep-0352/
* Add hacking to prevent this from happening in the future.
Change-Id: Id40000c2c453815b04a7d2fd765e19997291d8e3
Sometimes commiters tried to add tempest tests which require admin
credential under non-admin test path and that caused confusions to
tempest users. This patch adds some coding rule to make test path
clear for the maintenance.
NOTE: This patch adds #noqa to AbsoluteLimitsTests because the test
class needs force_tenant_isolation which requires admin
credential indirectly but the test itself is not admin test.
The history is Id71a705cf9b1dd0c0d41a2fb45ab77c95430a123
Change-Id: Id11eec13f2e431af8bbb83ac4904b2047e7932a7
This commit removes the trove tests from tempest as part of plugin
decomposition of tempest. Trove isn't in scope for testing in tempest
anymore and tempest testing will be done via a plugin now.
Change-Id: I0407e96f628e96bd4d3f7f42cf433b756a5bbaa9
This commit adds a hacking rule to enforce that we never add a config
dependency on tempest/lib. Right now we're completely dependent on
reviewers catching this, it is a strong rule so we should ensure we
can't ever land a change that does this.
Change-Id: I1ab1ba52573c6706a50abcd021759c93dd19aa44
Now Ceilometer and Aodh tempest plugins are implemented and all
tests and supported code is present in plugins.
-https://github.com/openstack/aodh/tree/master/aodh/tests/tempest
-https://github.com/openstack/ceilometer/tree/master/ceilometer/tests/tempest
NOTE- Need to keep config option 'CONF.service_available.ceilometer'
as it is being used in Congress tests in Kilo and Liberty branch and
Ceilometer tempest plugin is available since Mitaka.
If we remove that then, we will be breaking Congress gate for stable branches.
Change-Id: I0775bcc15dc9cbae6e075fe92f44b5f6c9b9d5d2
Depends-On: Ic0e6b72d8767d92cc63968c442c4ff65bb001cda
There is a common method for generating random uuid value.
So it is nice to avoid pointing it out on human review to reduce
reviewing workload.
In addition, this patch removes str() around the method calls
because str() is called in the method.
Change-Id: Ic8caea352d27c479fdb765a699d277b59f1c5bb4
Since the commit I3a8660838c1af2baefe21bbc57b006c9889ac35a ,
API tests of neutron advanced services have been migrated from
Tempest. So we can enable T108 check for remaining network api
tests to avoid -1 against small thing on human reviewing.
Change-Id: Ide15543c84d9b171d62db28274dbaecb4e880a6e
Current T110 check is just working for tempest.service modules, but
we have migrated lib's service clients into tempest.lib.service.
So we need to check these modules also on T110.
This patch extends the check for these modules.
Change-Id: Id102ba359f86bb7417e121e70c707aae36a24740
tempest.lib should not import local tempest code to avoid circular dependency,
so this patch adds pep8 check to block such kind of code.
Change-Id: I392d28b3195040a800d96171ef275c6e73f9fef4
This commit migrates all of the code from tempest-lib as of it's
current HEAD, 6ad0ce42c2791a28125d38b40e7dcddf32dbeed7. The only
changes made to the tempest-lib code is to update the imports and
other references to tempest_lib. Since in it's new home it should
be tempest.lib.
Partially implements bp tempest-lib-reintegration
Change-Id: Iadc1b61953a86fa9de34e285a0bb083b1ba06fa8
These tests aren't run in any gating job that I can find on either
tempest or zaqar. Zaqar also falls outside of the scope we set for
projects with in-tree tempest tests. A plugin is being worked in
zaqar change I06053d2d0d761c6398708280f3ef38274dec390c. But,
since these tests are unverified and haven't been run in quite some
time lets just remove them.
Change-Id: I047392de3341169992fc5e64aec1e571059840a8
The pep8 rules T110/T111 check the method names of service clients,
but the rules could not find violations on network clients because
these clients use some wrapper methods.
This patch adds some checks for covering network clients.
Partially implements blueprint consistent-service-method-names
Change-Id: I4aede28261626a27a3a857ba05c1a0fdcc3cac5b
As the consistent-service-method-names bp, "DELETE /resources/<id>"
methods' names should be "delete_<resource name>".
So this patch adds it as the hacking rule.
Partially implements blueprint consistent-service-method-names
Change-Id: I36592a91001361d816e302745fe8417aee5e7565
We need to implement more rules for the other methods(DELETE/POST/PUT)
for consistent method names. Before doing that, this patch separates
common part from T110 rule for sharing it.
Partially implements blueprint consistent-service-method-names
Change-Id: I24c90d331a45a4f6ad50303c4d17e7cd4d2d784c
This patch is a prototype for "GET /resources" hacking rule.
black_list_T110.txt file contains the service client files which
are against this rule. So we need to fix them with removing them
from this file.
Partially implements blueprint consistent-service-method-names
Change-Id: I150fe2ef21d4d4d246a46d9baf2fb14cc7d79ee5
Adding hacking check for testtools.skip decorator as described
in the bug referenced below.
Closes-Bug: #1490590
Change-Id: I9c456bb8b57a54fbcd1cd5249eec2292916d4ff9
data_utils.rand_name() appends randam charactors with a hypen like:
def rand_name(name=''):
randbits = str(random.randint(1, 0x7fffffff))
if name:
return name + '-' + randbits
So it is not necessary to specify a hypen in caller side.
This patch adds a hacking rule for blocking "-" at the end of argument
of rand_name() calls.
Change-Id: I9b9f25dbe5a3ef5ac5900113bcc46e0d911becc9
Scenario tests has been migrated from official python clients to
Tempest clients.
Documents for the same needs to be fixed.
This patch fix the README & HACKING file for above changes.
This patch also extends hacking rule of not import python clients
for scenario tests too.
Change-Id: Ieb19a2c0b09f00fb3d4f6c7c73541275a4cf24ae
Extend the existing T105, which was missing checks for
tearDownClass anyways. Forbid overriding setUpClass
and tearDownClass except for tempest/test.py where the
test base class is defined.
To be able to enforce the rule, fixing setUpClass
which was added with new tests before this patch could
merge.
Partially-implements bp resource-cleanup
Change-Id: Ib4d98c2ff8776ea1379a044b5a30fb02e351ce75
OfficialClientManager is unused now that scenario tests use
clients.Manager instead. Dropping the class, the hacking
rule that forbade its use in API tests and related unit tests.
Change-Id: Ib359b9ab0a903436d9d9c265e8c4d82bdd80a496
This patch fix the coding issue where "mutable" objects like list, dict
are being used as default value of param.
Current functions with mutable object as default value can create issues
if they will be called in more random order.
For example - first call with non default value and then with default
value. In this case original default will be overridden by the first
function call (as mutable object keep their state among function calls).
This commit also add the hacking check for the same and its
corresponding test.
Closes-Bug: #1330322
Change-Id: I251b316ef6d37f4b95c5e8d73a20a39005c22870
Due to copyright issues Marconi is renamed to Zaqar. This patch updates
all Marconi references to Zaqar.
Change-Id: Ib37b95d99bbfc666e1a3a17174d87804ecd0649e
This commit fixes the T104 hacking check to handle subdirs in the
scenario tests. Previously the check assumed that all the scenario
tests were in the scenario dir. However, this isn't necessarily the
case. For example, the heat scenario tests are all in the path:
tempest/scenario/orchestration which means they were excluded from
the T104 check. This commit fixes the oversight and adds unit tests
to cover this condition.
Change-Id: I42d7f22f6194f624f8488493de29e60ff19d211a
This commit adds a new hacking check to ensure that the manager class
from the scenario tests, OfficialClientManager, isn't used in the API
tests. There is already a hacking rule preventing imports of the
official clients, T102. But, the official clients could still be used
through the scenario tests which wouldn't be caught by T102. This
attempts to close that gap by adding a new hacking check for
OfficialClientManager.
Change-Id: I608a1319c136118947e3df74fa88f7124011caa4
This commit adds support for using noqa with the T105 hacking check.
This is need to unit test the _find_caller() utility with setupClass
in the call stack.
Change-Id: I5c1a07d019b79b082cb895a17364f7e7cb2f08fe
The test_utils module in tempest.common.utils is only useful for
scenario tests. So it's a little confusing to have it in the common
directory. This commit corrects this by moving the file into the
scenario directory. The service tag requirement for scenario tests
hacking check is updated to only check scenario test files, because
a method in the new utils module was being picked up incorrectly.
Change-Id: I1d70745a5ee07923e680328e5b8a29bfb82d8099
This commit adds a new hacking check to ensure that a service tag
isn't added on a test whose path includes the service tag being added.
Having a service tag added where the path already contains the service
name is redundant.
Partially implements bp add-service-tags
Change-Id: I88b3005d05e17751045d766efb1e965822a3c50e
Change I6cb693298c75e400e5fe3a4bdd2532bca9d87b9b updated the list of
clients to include the missing official clients. However, at roughly
the same time savanna was renamed to sahara, so it wasn't included in
the list of clients. This commit corrects the oversight by adding
sahara to list of clients to check for T102.
Change-Id: Icfaad37b5f3b03f112ad6eb2fdedde38013287dd
We don't need to have the vim headers in each source files. We can set
it in our vimrc file instead. Also if set incorrectly, gate (pep8 check)
will catch the mistakes.
More discussions:
http://lists.openstack.org/pipermail/openstack-dev/2013-October/017353.html
Change-Id: I43703e2289212389c7841f44691ae7849ed1f505
Closes-Bug: #1229324
This commit just updates some of the hacking documentation around unit
tests. It also adds a new hacking rule to ensure that setUpClass isn't
used for the unit tests.
Partially implements bp unit-tests
Change-Id: Ie8b1c9f1312a467265d53bc28ee905fa1b5fbb53
this change introduces a skip_because decorator which accepts two args
a bug and a condition; also updates the skip tracker accordingly
Change-Id: If53f2ef81d6bddbce284267216254b467046855f
This commit adds a hacking check to ensure that all the tests in the
tempest.scenario directory include a @services decorator.
Part of bp add-service-tags
Change-Id: Ie2f3c384aac568ab791f4d7665d29ea9b3fda7f8
This commit adds a framework for running unittests on tempest to
verify that tempest works as expect. The first tests added are to
verify the response codes of the test runner wrapper scripts we use
on jenkins jobs.
Change-Id: If8e91238054593999e7b5bc34da499bd0ac02311
Update to use the neutronclient library and replaces Quantum references within
the repository.
implements bug: 1197208
Change-Id: Ib3d52e3d9671fe4b30c4bcf76b5e7c2ee9de4664
Andreas Jaeger