Commit Graph

175 Commits

Author SHA1 Message Date
Zuul
8da64f1d17 Merge "Avoid tox_install.sh for constraints support" 2017-12-11 02:38:31 +00:00
Andreas Jaeger
880d94352b Avoid tox_install.sh for constraints support
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.

This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.

Change-Id: Ifcc108dc4f91fcdf1d9d4c21ed8ad2983c7ff2c1
2017-12-02 17:07:29 +00:00
Matthew Treinish
069d828781
Only use stestr and tempest run in tempest tox jobs
Now that we're starting to unify everything on a single test runner we
should only be using either tempest run or stestr (which tempest run
will use internally in the near future). This commit changes the
coverage job to use stestr and removes the ostestr job as part of that
effort.

Change-Id: I5972346b8dbef4a57c733f85ceeeeca425bf263e
2017-11-30 11:42:14 -05:00
Masayuki Igawa
d0b8ebd84a
Switch to use stestr for unit tests directly
This commit switches to use stestr command to run unit tests directly.
Recently, the latest ostestr command uses stestr instead of testr.
However, we should use stestr directly because it's simple and straight
forward. There is no reason to use ostestr for unit tests anymore.

We still have the other ostestr/testr usage such as tempest run command
and documentations. So, that will be changed with following patches.

Change-Id: Iff4abef50178bdc83b868eed4a906e22d790762b
2017-09-14 16:00:31 -06:00
Jenkins
a977ccd79e Merge "Capture logs when running unit tests" 2017-09-12 21:50:23 +00:00
Andrea Frittoli
8f018cb7c8 Capture logs when running unit tests
The unit test logs are mixed up in the output of subunit-trace.
Adding OS_LOG_CAPTURE=1 by default to avoid that.

Change-Id: Ie322795c8638963b276c4bd54b845331c3567370
2017-09-12 17:11:08 +00:00
junboli
8bd00e3ba0 TrivialFix for flake8 and docs build in tox.ini
1. exclude "build" in flake8 section.
2. to ensure the old files are not left over when running these
   document builds.

Change-Id: Ia5590b7816b36351559c692eebfecb3c8e6fd1a3
2017-08-23 07:04:20 +00:00
Masayuki Igawa
aa516f871f
Add flag to generate tempest plugin list
This commit adds a flag to generate a tempest plugin list. In a local
environment, we don't need to generate a tempest plugin list, always
when we just need to see the other documents. Besides, generating the
tempest plugin list takes a long time. When GENERATE_TEMPEST_PLUGIN_LIST
environment variable is 'true'(default), the plugin list is created.
When the other value such as 'false' is specified, it isn't.

Change-Id: I6fd3cb3045ba8bbfb39c597413e13faa3cd24ae3
2017-07-12 13:56:04 +09:00
Chandan Kumar
b7affe4c03 Added script for doing tempest plugin sanity
It will help to catch tempest plugin config and dependency issues
with tempest.
sh ./tools/tools/tempest-plugin-sanity.sh or
tox -e plugin-sanity-check will invoke the same.

Change-Id: If42561bfc1d6dcffbb6537634d7769ef90fab21d
2017-07-11 08:17:04 +05:30
Masayuki Igawa
c860912ddd
Fix tox coverage section
This commit removes the setenv parameter of testenv:cover section in
tox.ini. Recently, ''tox -e cover doesn't'' work, actually.

Change-Id: I82ac3dace47c187160dc683e0f740b0589773c0b
2017-04-04 16:23:09 +09:00
rajat29
9b66085037 Removing py34 from tox
Now gate job for Python 3.5 exists and not 3.4,
we should remove those references to the 3.4 that is untested

Change-Id: I2bfedb9a7cc48413dee35d519c52cc2fd3bd3f13
2017-03-30 11:54:51 +05:30
Sean Dague
8ea57c2f40 Remove explicit install of setuptools
This is for an old pip workaround which may no longer be valid.

Change-Id: I0b5e392d58828929f3d18569e77337fb79d93c0f
2017-03-22 12:26:51 -04:00
Andrea Frittoli
b838af91f7 Add a new scenario tox env
The scenario environment forces serial execution, and it selects
all scenario tests, including those tagged as slow.

Change-Id: I3ad362da88f770cd03f32534683888ea59cfe5c6
2017-03-06 15:11:13 +00:00
Jenkins
f0fe745108 Merge "Use upper-constraints for tox envs" 2017-03-05 01:22:36 +00:00
Sean Dague
49505df20f make -e full run tempest scenarios serially
The scenario tests end up generating a large load on target
environments. In most cases this means the whole system is running at
an extremely unrealisticly high load, and many of our scenario
failures are probably traceable back to this.

This classifies many more scenario tests as slow so that during normal
full runs they are not run, they are still accessible for running with
the tox -e all flag.

Selection based on https://ethercalc.openstack.org/nu56u2wrfb2b

Depends-on: Ia303773cabffc25d910cd1c0950d6739592bade6
Change-Id: I35cf4a77ff61d04ed8be58acff3edf47b1ae36f7
2017-03-03 20:50:46 +00:00
Jordan Pittier
ec2eefebc8 Use upper-constraints for tox envs
Pin tox environments to upper-constraints to avoid conflicts with
library releases.

Change-Id: I6e564f3035e2fdc5461f57f15168f498c99fc164
Closes-Bug:#1628597
2017-02-22 17:59:28 +01:00
Masayuki Igawa
134d9f748b
Introduce flake8-import-order
This commit introduces flake8-import-order to check import orders
automatically. We shouldn't do it manually.

Depends-On: Ia2140f5566ce169b076ffa5c9ebfbdbfc41e5bed
Change-Id: Iff75ebec42b295870edd1c04001adfb17460a682
2017-02-13 17:16:18 +09:00
Jordan Pittier
525ec718f6 Hacking: enable H904
It's a really cool rule that will avoid suboptimal usage of the
logging library.

Change-Id: I6414624f6b51333d477dd88290158aaeb51b851f
2016-12-19 16:25:24 +01:00
Jenkins
fe5f786001 Merge "Added bindep.txt to the project" 2016-12-08 18:36:06 +00:00
Jenkins
28d142be69 Merge "Tox.ini: removed an outdated code comment." 2016-12-08 14:55:30 +00:00
Jordan Pittier
d0c4c5697b Tox.ini: removed an outdated code comment.
We do support and comply with H405.

Change-Id: I05f1969142ea994d1c74eaa17baddb23375ddff8
2016-12-07 13:51:20 +01:00
Jordan Pittier
e976a2a982 Tox: Hacking: enable extensions H106 and H203
[H106] Don't put vim configuration in source files.
[H203] Use assertIs(Not)None to check for None.

Change-Id: I1837e4d2228740290bb58c92e6e557800843deec
2016-12-07 13:46:24 +01:00
Chandan Kumar
ac5c5537b1 Added bindep.txt to the project
It allows to avoid installing unnecessary packages while preparing enviroment
for testing.

Change-Id: I6b993b2c2fd56c7517e6cc41efc07bf4a83ff945
2016-11-30 18:54:03 +05:30
Jordan Pittier
2e7ae7c6e5 Remove the Stress framework
It's not used anymore. There was general consensus in Feb 2016
to deprecate it (see [1]) and remove it in Newton.

[1]  [qa] deprecating Tempest stress framework

Change-Id: Ib229985ea2a1fee495c9492c9ce1781e6bac1dc6
2016-10-19 14:31:13 +02:00
Matthew Treinish
c3fef9f530 Switch unit test tox jobs to use ostestr
Now that we've deprecated all our bash wrappers for running tests
we shouldn't be using them for running unit tests either. This commit
updates the unit test tox job definitions so that we no longer are
relying on deprecated code in the gate and we can remove the wrappers
when the time is right.

Change-Id: I127786537c0792456d26b13fc3a2044afc4e93a1
2016-10-04 10:34:13 +00:00
Jordan Pittier
cefaa2badd Introduce a new tox target that checks requirements.txt
That new tox target checks that we don't have missing requirements in
requirements.txt AND that we don't have superfluous (unused) requirements
in requirements.txt.

That patch is copy/pasted from I014375a5ecfebc723541c2f2db6c2e0fba636aa2
which tried to do the same for Nova (unmerged yet for an unknown reason).

This change also includes the fixes to requirements.txt needed
to pass the new tests.

Change-Id: Iff37af95269c38e4d435a53a21e5d60e2c5a3fa8
2016-09-26 16:12:34 +00:00
Jenkins
01bff317f9 Merge "Don't include openstack directory in exclude list for flake8" 2016-09-23 18:45:46 +00:00
Jenkins
d1b9789549 Merge "Deprecate method get_ipv6_addr_by_EUI64" 2016-09-21 22:44:41 +00:00
Masayuki Igawa
730f59b2ba Fix missing serial option of tox full-serial target
This commit fixes the missing 'serial' option of tox 'full-serial'
target.

Change-Id: If33d8ad6135752fb31228c02810b36a3b9825018
2016-09-21 11:47:32 +02:00
ChangBo Guo(gcb)
cd0374a2a5 Deprecate method get_ipv6_addr_by_EUI64
Oslo.utils provides same method get_ipv6_addr_by_EUI64,
so deprecate it in Newton and remove it in Ocata.

Co-Authored-By: guo yunxian <yunxian.guo@easystack.cn>

Change-Id: Ia72f866cda69b4f98e88936df32c76e455619d07
2016-09-21 13:44:54 +08:00
ChangBo Guo(gcb)
6ebbc01ebd Don't include openstack directory in exclude list for flake8
Directory openstack doesn't exist, so don't need skip it.

Change-Id: I68c841019a52393bde5852bf390452c7aa127e94
2016-09-21 13:40:42 +08:00
Matthew Treinish
68c787100c
Switch tox to use tempest run and deprecate bash runners
This commit switches the tox commands that run tempest to use tempest
run instead of pretty_tox.sh. Tempest run is the primary method of
running tempest moving forward so we should dogfood it where we can.
This also officially starts the deprecation timer (which was long
overdue as these were unofficially deprecated before) on the
run_tempest.sh and pretty_tox scripts as we move to having a unified
method of running things.

Partially implements bp tempest-run-cmd
Depends-On: I8786b9709584378aaf48393115dda94397e22544
Depends-On: I0529c2922242d8ed9c92ef718328e09f3ea9e05b
Depends-On: If0064b9e8358332972ef4a1eee8f150e66f8c50f
Depends-On: Ibaffa3a8568ea058d964463df4b76196c4d2bc7a
Change-Id: I3684fce66a799579fa68af119652cafef25a9f03
2016-09-16 11:19:51 -04:00
Jenkins
0e8859b2fe Merge "Add python 3.5 classifier and venv" 2016-09-01 08:59:27 +00:00
Ken'ichi Ohmichi
3febde7d62 Remove thirdparty from tox.ini
Since the change Ib5e24e19bcba9808a9f49fe7f328668df77fe4f9 , the
directory thirdparty has been removed. So we don't need to specify
it on tox.ini anymore.

Change-Id: I7eddcb975ebf20a78920acd6f48ba8260f6cb75a
2016-08-23 15:03:43 -07:00
guo yunxian
c78bda0031 Add python 3.5 classifier and venv
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.

Change-Id: Iaec0c9e88cb38ab876b5bc957f53989615cab246
2016-07-14 12:33:06 +08:00
Matthew Treinish
bdef1c77ab
Move oslo config generator config inside package
This commit moves the oslo config generator config file from living as
a data file, which makes it very difficult to rely on, to just living
in the package. By including it in the package we have a guaranteed
relative path we can rely on for building utilities on it, like the
tempest init command.

Change-Id: I5b9deae733f42fe3ed39418103304f71b43dccb3
Partially-implements: bp tempest-run-cmd
2016-06-22 14:39:42 -04:00
Matthew Treinish
0e25a90793
Use a single venv for all tempest jobs
This commit simplifies the venv management via tox for tempest jobs.
All tox targets that actually run tempest will now share a single tox
venv .tox/tempest. This also adds a new target tempest-venv which is
the same as the pre-existing venv target, but uses the common venv.
We can't simply use the common venv with the venv target because
external tooling, like the docs jobs, rely on the .tox/venv path
existing when using the venv target.

Depends-On: I34a69020eee07156e64026781a3c0bffdb5ab415
Change-Id: Iab2e6e04b6c5795a4d0c8214564106525b942308
2016-05-04 13:10:49 -04:00
Matthew Treinish
3a9d3e70d5 Add ostestr tox jobs
This commit adds tox jobs to run tempest with ostestr instead of the
pretty_tox*.sh wrappers. ostestr provides essentially the same
functionality except it exposes slightly more rich selection logic
and additional debugging options. However, since the cli isn't
backwards compatible with pretty_tox we can't simply remove the old
tox jobs.

Change-Id: I347438b1d561af7e05fdce895e2e3e51d2b38b0c
2016-04-21 13:14:50 +09:00
Masayuki Igawa
d81d3ceed7 Align multiple lines in tox
This commit aligns multiple lines in tox for visual consistency.
I fixed them to 4 white spaces because the tox example[1] shows 4 white
spaces indentation.

[1] http://tox.readthedocs.org/en/latest/example/basic.html

Change-Id: I4f3b9c5c4982a0e96bb120ce8546dd48b0fa8a4b
2016-04-20 15:11:35 +09:00
Jenkins
0aeea56f18 Merge "remove largeops test" 2016-03-01 06:11:22 +00:00
Sean Dague
1976da83a5 remove largeops test
This isn't really a scenario test, it's a performance test, however
without any of the infrastructure to do performance testing /
comparisons. It's time has come to say goodbye.

Depends-On: I788cd70dc4e6a9c3f8e6805d5bdcdf7f549fce35

Which removes it's use from the gate.

Change-Id: I93b2fb33e97381f7c1e0cb1ef09ebc5c42c16ecc
2016-02-24 20:24:58 -05:00
Andrea Frittoli (andreaf)
897ecfb602 Add reno to tempest
Tempest is going to be on pypi and expose a stable interface in
the tempest.lib namespace, so adding reno to it.

Partially-implements: bp tempest-lib-reintegration
Change-Id: Ice4c27f7bc6d0154cfced9789f3c82236106a9e3
2016-02-24 11:31:32 -05:00
Ghanshyam
3a59e1ef92 Raise tox min version to avoid maximum recursion issue
There was regression in tox 2.2.0 for env-substitution
with setenv.

https://bitbucket.org/hpk42/tox/issues/99/environment-variable-usage-schema-env-xxx

It is fixed in 2.3.1
https://bitbucket.org/hpk42/tox/issues/285/tox-220-breaks-some-toxini-config-files

So no issue with 2.3.1, but let's raise the min version in tox.ini
to avoid maximum recursion issue.

Change-Id: Ie808e46a90a681b21c30998d2fc867981c2c05fd
Closes-Bug: #1534006
2016-01-14 17:15:29 +09:00
Sergey Reshetnyak
e05246d035 Add support override OS_TEST_TIMEOUT for all environments
Change-Id: Ic615c3958c30aa14f46784923d4cc12d634ab099
2015-12-28 18:06:33 +00:00
Jenkins
02ef4c2956 Merge "Put py34 first in the env order of tox" 2015-12-11 15:01:56 +00:00
Matthew Treinish
c1802bc956
Use the idempotent id and tooling from tempest-lib
The idempotent id decorator and the check-uuid script were migrated to
tempest-lib as part of tempest-lib 0.10.0. However, tempest was never
updated to use the library version of idempotent id or check-uuid.
This commit updates tempest to remove it's local copies of the
idempotent_id decorator and the check-uuid tool and use them from
tempest-lib instead.

Also, instead of updating the import of every single test module in
tempest an alias is just added to tempest.test to let all the old
imports work seamlessly. Updating the imports and removing the alias
may or may not be worth it in the future, but that will be a follow
on patch or discussion.

Change-Id: Iae3f491ada8f7b55c1fd817e033d968c2102d11e
2015-12-03 19:20:44 -05:00
lvdongbing
768c807657 Put py34 first in the env order of tox
To solve the problem of "db type could not be determined" on py34
we have to run first the py34 env to, then, run py27. This patch
puts py34 first in the tox.ini list of envs to avoid this problem
to happen.
Closes-bug: #1489059
Change-Id: Ia11ecf7bd147e35b7c8ad3db0eaad17f893e78ba
2015-12-01 22:46:38 -05:00
Masayuki Igawa
835220498e Fix typo in tox.ini
This commit fixes typos in tox.ini.

Change-Id: I990b5e3d713de5a19300f4ae7ead66d75a423429
2015-12-01 16:51:34 +09:00
Ken'ichi Ohmichi
2e2ee19179 Fix H404/405 violations for remaining
There is a lot of H404/405 violations in Tempest now, and that leads
difficult to migrate the code to tempest-lib or the other projects'
repos. This patch fixes these violations for remaining.

Change-Id: I3abac40ebb33836980c119d59bc97f035b213c0a
2015-11-19 11:22:29 +00:00
David Paterson
0bf52d497b Fix init command
Init command was not working properly as it could not locate the
configuration sample file.
Steps taken:
- Move config-generator.tempest.conf to etc so it will be installed
with pbr.
- Update all files with new path to config-generator-tempest.conf as
needed.
- Refactor init command so if it detects we are not in a virtual
environment, try to find the global config directory /etc/tempest.
If that fails fallback to [sys.prefix]/etc/tempest.

Closes-Bug: #1491058
Closes-Bug: #1490670
Change-Id: I960bc711ff78ac2b0441ef63dff8ec4fb268fd3a
2015-09-23 09:42:07 -04:00