405 Commits

Author SHA1 Message Date
Jenkins
91eec92d67 Merge "Network API: default to ipv4, add ipv6 tests" 2014-02-18 03:09:25 +00:00
Jenkins
1b5e86dea1 Merge "enable volume list tests for cinder v2 - part2" 2014-02-17 09:32:08 +00:00
Julien Leloup
a5ee542bc0 cleanUp() removing all test resources as an admin
Add a parameter in tempest.conf "full_clean_stack" to
allow/prevent the use of cleanup() method at the end
of a stress test run.

This is mandatory to use stress tests in a multi-user
environment since the actual cleanup() is designed to
remove every object of every tenant (and thus will
impact other users on the same openstack)

Change-Id: I42dc9592617aeeca9660f670940ecbf50b6108ab
Closes-Bug: #1273254
2014-02-13 14:29:02 +01:00
Jenkins
f8870ec8f3 Merge "Negative tests: Add result check for resources" 2014-02-12 16:27:06 +00:00
ivan-zhu
af8c4e64cc port flavors and server_password tests into nova v3 part2
this ports the flavors, server_password tests and corresponding client
into v3 version.

Partially implements blueprint nova-v3-api-tests

Change-Id: Iaf2056d56429898d524536e54296bac5e479fd3e
2014-02-12 21:22:59 +08:00
Zhi Kun Liu
8cc3c84737 enable volume list tests for cinder v2 - part2
this ports test_volumes_list into cinder v2 api,
and also ports corresponding client.

1. changes of the response data of list and list details in v2:
(their items are same in v1)
- list view includes items:
id, name, link
- list details view includes items:
id, status, size, availability_zone,created_at, attachments, name,
description, volume_type, snapshot_id, source_volid, metadata,
links, user_id, bootable

2. use 'name' instead of 'display_name'

v2 commit id:
0c507aa6d08a9471bf896961cc99d40f337f1e4d
484fb9e1c748e718349e451e80a0e025ed007b1d

Change-Id: I19cf00270f0ca55770188c6c71c02e13c5bd82f6
Implements: blueprint cinder-v2-api-tests
2014-02-08 14:22:18 +08:00
Henry Gessau
ffda37aac8 Network API: default to ipv4, add ipv6 tests
Change-Id: I34da0ede4d7b92b2e752c2da82172c495e1946b1
2014-02-07 21:54:54 -05:00
Matthew Treinish
db2c59770c Make v2 and v3 identity apis configurable
This commit adds a config option to the identity feature enabled group
to specify whether the v2 and/or v3 apis are enabled in keystone. In
the process of adding this feature the logical split between v2 and v3
tests in the identity api tests is made more clear. Additionally as part,
of the split several bugs are fixed where the v2 api was being called
in the v3 tests.

Change-Id: I0d727f3567c82fe512b956110bb813119ced0b3d
2014-02-06 17:01:36 +00:00
Marc Koderer
424c84fd6e Negative tests: Add result check for resources
This add an result check if a invalid resource is given. Introducing a
new dict that can be enhanced later for other cases.

Change-Id: Ief3e51fafb9437a8e6b9a71aad79459716bac3d4
Partially-implements: bp negative-tests
2014-02-06 17:02:19 +01:00
Andrea Frittoli
77f9da4c59 Multiversion authentication part2
Address last comments from part1:
- replace auth_data[0] with get_token()
- add token property to rest_client
- Remove comment about planned changes from config.py

Partially implements: bp multi-keystone-api-version-tests

Change-Id: Ia54cd3d824e44dd99d9226d4c0c6d6861d8c12ae
2014-02-06 11:46:27 +00:00
Jenkins
bac25745dd Merge "Clean/leave OpenStack after a stress test" 2014-02-06 10:54:21 +00:00
Julien Leloup
04d40f73b7 Clean/leave OpenStack after a stress test
New parameter "leave_dirty_stack" available in tempest.conf
to prevent (or not) the execution of tearDownClass()
after an exception during a stress test

Closes-Bug: #1273245
Change-Id: I89ecac1fb428cf8809a3634869f2dfdf564aea64
2014-02-05 16:53:06 +01:00
Andrea Frittoli
8bbdb1660f Multiversion authentication part1
Moves authentication from rest_client to an external auth_provider,
assigned to the client by the client manager.

The auth provider can decorate a request based on credentials (coming
from the client manager) and filters (region, service, endpoint_type)
given by the client. The auth provider can also return the raw
auth_data, which in the Keystone implementation is a tuple (token,
auth_data). The auth provider allows mixing multiple credentials
when decorating a request, possibly using empty or invalid credentials,
to facilitate negative tests.

The auth provider caches auth data, so that all API requests for a
specific set of credentials only require a single call to obtain a
token, unless the token expires or is forcefully deleted from the
auth provder.

Two implementations of the auth provider are included: Keystonev2 and
Keystonev3.  The Manager object behaves as factory of auth_providers,
building the correct one based on the configured auth_version, and on
the interface type (JSON or XML).

Fixes endpoint selection for v3 auth. Drops unused basic_auth.
Extends TokenClients to provide both token and auth data,
and accept different combinations of credentials for v3.
Removes redundant server_client_v3_auth.

Adapts tempest unit tests to work with modified rest client.

Introduces a configuration parameter for authentication version to be
used. That is used when instantiating the client manager, and it
applies to all clients used by api tests.

Next steps (next patches):
- move to credentials as dict (as opposed to tuple)
- expose a get_client method from the client manager
  and create clients only when requested
- remove redundant CustomizedHeader object storage clients
- supports keystone v3 in tenant isolation
- use Auth Provider in scenario tests
- use Auth Provider in CLI tests
- implement parsing of catalog XML format (?)

Partially implements: bp multi-keystone-api-version-tests

Change-Id: Icfa921e9051c01f339f8d2471b12d6ec950cc456
2014-02-05 11:34:53 +00:00
Marc Koderer
24eb89cd3e Negative test autogeneration framework
Generates and executes negative tests based on a json schema for the API.
It's using testscenarios lib to generate a test case for each variation.
It contains resource handling and result proofing of invalid requests.

Not covered in this patch (will be done later):
- Documentation
- Unit tests
- Actual json schemas for all APIs

Partially-implements: bp negative-tests
Co-author: David Kranz <dkranz@redhat.com>
Change-Id: I828f9b74c31f2e25c91e8149ecd7cd0b189ce99a
2014-02-03 17:30:11 +01:00
Jenkins
557d47065d Merge "Matches one flavor and one image by default" 2014-02-01 07:57:19 +00:00
Matthew Treinish
e2b56b5713 Convert cli tests to use global CONF object
This commit takes all the uses of config in the cli tests and coverts
them to use the global CONF object.

Partially implements bp config-cleanup

Change-Id: I7a39839d7a4ebbf8372489507486ad2cac7f3adc
2014-01-30 18:43:22 -05:00
Andrea Frittoli
99901c043e Matches one flavor and one image by default
Changes default in the input-scenario settings to match only
one image and one flavor, to avoid consuming gate time.

Fixes bug 1274628

Change-Id: Ide865012269d3f60250d24443e4f20af263ee021
2014-01-30 19:43:07 +00:00
Jenkins
cb8e5f8acd Merge "Add a couple log errors to whitelist.yaml" 2014-01-29 22:17:16 +00:00
Darragh O'Reilly
6b636677c2 Increase ping timeout from 60 to 120 seconds
The Cirros image only sends out three DHCPDISCOVERs with 60 second
waits between. The ovs_neutron_agent may not have the port wired in
time for the first one, and with ping_timeout=60 the test will be
terminated before the second one is sent. This patch increases the
timeout to 120 seconds to prevent that.

Change-Id: I03fcec50b91fd89ad64e16fce82f9211c2e086a1
Partial-Bug: 1253896
2014-01-24 12:17:40 +00:00
Jenkins
becee21140 Merge "Add a config option for trusts" 2014-01-19 13:55:13 +00:00
Clark Boylan
1acacd3ad4 Add a couple log errors to whitelist.yaml
These log errors have hit the large ops jobs several times. There is one
in ceilometer acentral and one in nova conductor.

Change-Id: Idc30110085e95c615958fc5f90e86417855e6d7a
2014-01-17 23:44:05 -08:00
Jenkins
6cbd7f5f9c Merge "Add missing whitespace in config description" 2014-01-16 11:08:20 +00:00
David Kranz
36df1b8775 Add to whitelist two errors that slipped through
Change-Id: I0347074735359d33f294d1ededc45149a29157ec
2014-01-15 19:03:21 -05:00
Dirk Mueller
14bd562317 Add missing whitespace in config description
Change-Id: Ie7e66b9de0a1d14de795166671d31fc9bbeae206
2014-01-14 19:33:05 +01:00
Jenkins
9cc56849e4 Merge "Add a discoverable_apis option for swift" 2014-01-14 00:28:04 +00:00
Jenkins
cad05455c1 Merge "Input scenario capability for tempest" 2014-01-13 13:25:07 +00:00
Jenkins
711fc1a275 Merge "Remove Swift container-sync test skipping" 2014-01-13 01:00:30 +00:00
Andrea Frittoli
f5da28bdf5 Input scenario capability for tempest
Adding the ability to generate multiple tests from
a template one based on input scenarios.
The implementation is based on testscenarios (added
to requirements.txt), and the correct behaviour
could be verified using unittest or testr + subunit,
both serial and parallel runs.

The generation of the array / matrix of tests relies
on the load_test protocol. nosetests does not implement
the protocol by default.

Static scenarios can be added to any tests with very
little effort. This commit introduces a couple of
dynamic scenarios:
- all flavours matching a configurable regex
- all images matching a configurable regex

Dynamic scenarios are coded in test_utils, and provided
by InputScenarioUtils. The class includes in the doc
an example on how to use input scenarios.

Additional scenarios could be AZs, or credentials.

This commit adapts existing test_server_basic_ops
to use input scenarios, and include server
ssh verification.

This change implements blueprint input-scenarios-for-scenario.
Implements: blueprint input-scenarios-for-scenario

Change-Id: Ia86f48772ac02d67faa1c7d764cb9dc0938f6452
2014-01-12 09:27:48 +00:00
Fabien Boucher
2178d31830 Remove Swift container-sync test skipping
The skipping reason was bug #1093743 that
is currently marked as invalid in Swift bugtracker.

This patch remove the skip and propose some
refactoring to the test code.

The test is marked as slow as it can take long
to run as it depends on container-sync process run
internal which is 300 seconds by default.

Change-Id: I939a014646b42cb2a461f874a5ad0edfc79c5561
2014-01-10 14:12:53 +01:00
Matthew Treinish
d5021a753b Add a config option for trusts
This commit adds a new config section identity-feature-enabled which
contains the options to select which optional/configurable features in
keystone are enabled. The only option that fits this category at this
time is trusts.

Partially implements bp config-cleanup

Change-Id: I8110156e48616534c45e9eb2c12557b2d6bdfd46
2014-01-09 18:49:38 +00:00
Matthew Treinish
2034538bf5 Add a discoverable_apis option for swift
This commit adds a new list option for the object-storage-feature-enabled
config group, discoverable_apis, which lists out which Swift discoverable
optional apis are expected to be enabled on the swift server. It also
converts the pre-existing single boolean options for these types of
features to use the new list option. This option is used in the same
manner as the extensions list options for the other services. However,
since swift doesn't have the same concept of extensions as some of the
other OpenStack projects optional discoverable apis (which are often
but not always tied with middleware) were used instead. If an optional
api being enabled isn't discoverable from an external api call then
that should have a separate config option in the
object-storage-feature-enabled config group.

Partially implements bp config-cleanup

Change-Id: I846860804050ec44491ce05d6b72e1cc639609b1
2014-01-09 16:39:48 +00:00
Roman Prykhodchenko
62b1ed19de API tests for Ironic
Ironic is a baremetal provisioning service that
is intended to replace nova-baremetal-driver.

Recently it was integrated to devstack so now
it's reasonable to start testing it with tempest.

This patch adds a client for baremetal
provisioning service and some tests for the Ironic API.

Change-Id: Ifd65d6a60179e72dbfa81825f234f0ff76ebb055
2014-01-08 11:13:46 +02:00
Sean Dague
fc691e33f3 sync oslo to current
this is largely to deal with config generator issues, however
we're apparently quite far behind on other items, so lets just
sync it all.

Change-Id: I2f36e41b5d7cce8dd0b64636e45da3f7481b0e39
2014-01-03 09:25:29 -05:00
Yassine Lamgarchal
b158d41949 Add config for Telemetry
This patch add the configuration for Telemetry.

Partially implements: blueprint add-basic-ceilometer-tests

Change-Id: I8859857c5cf469045f66631c3d52edcb765035d7
2013-12-27 23:47:59 +01:00
Zhi Kun Liu
bb363a2f0e cinder v2 api tests - part1
Change volume base class name:
BaseVolumeTest -> BaseVolumeV1Test
BaseVolumeAdminTest -> BaseVolumeV1AdminTest
for the support of cinder V2 test class

Implements: blueprint cinder-v2-api-tests

Change-Id: I712a90d39981564af09393ed1b7f4be12897c808
2013-12-20 15:20:32 +08:00
Jenkins
4eedb44ab0 Merge "Add Savanna-related configs for testing" 2013-12-20 03:41:08 +00:00
Derek Higgins
85cd514b70 Fix the help message for run_ssh
Change-Id: I3101f68386420479172d51bca5ee658099f9a323
2013-12-17 17:16:27 +00:00
Sergey Lukjanov
cec6c3fe10 Add Savanna-related configs for testing
Currently we're only need to expose catalog_type config.

Change-Id: I29bc1d91adc7098cdc162146ccabe6197162faf0
2013-12-16 20:00:38 +04:00
Matt Riedemann
ed13d4a2dc Add whitelist entry for s-proxy 'Timeout talking to memcached'
According to John Dickinson:

"This isn't an error. It's logged at an error level because it may hint
at other problems, and it's something an operator needs to know about,
but it is not an unhandled failure condition. (Think of it similarly how
in swift a server being down is an "error", but it's something that
swift seamlessly works around.)"

Closes-Bug: #1260894

Change-Id: I41e55c5e34ee214727fbbd7b9daa1f6ea9bf8050
2013-12-13 18:31:22 -08:00
Jenkins
9c9453f78d Merge "Add config options for enabled extensions" 2013-12-11 00:08:45 +00:00
Jenkins
7e0684a310 Merge "Remove redundant whitelist for DHCP agent" 2013-12-10 02:41:47 +00:00
Jenkins
30f6c413e7 Merge "Adds paramiko logs to console output" 2013-12-09 22:28:30 +00:00
Matthew Treinish
e3d26143ab Add config options for enabled extensions
This commit adds a new set of config options to the feature_enabled
groups for a list of enabled extensions. These options are used to
specify whether all extensions are enabled or which subset is expected
to be enabled. This just sets up the initial framework for doing
this and converts the FlavorExtraSpecs tests to use it.

paritally implements bp config-cleanup

Change-Id: I6a5a9b16e62eb8a216334a0662c99f0dd0d16873
2013-12-09 15:54:44 +00:00
armando-migliaccio
6adb65fb9a Remove redundant whitelist for DHCP agent
It looks like the whitelist for the DHCP agent is added
twice. This looks like a genuine typo, and if it isn't
there is clearly something wrong! There's no good reason
for duplicating the list of whitelisted errors.

Change-Id: Ib94f60e8aa3ce1b0a6e28d9051da0124d403bf27
2013-12-06 08:17:34 -08:00
Jenkins
16562bb5e3 Merge "Remove unused allow_tenant_reuse flag" 2013-12-05 01:00:31 +00:00
David Kranz
14e00495c7 Remove unused allow_tenant_reuse flag
Change-Id: Ic4884666965dbe22fb55b12e087dd102345686a1
2013-12-03 15:50:26 -05:00
Saranya Pandian
baefcccdbe Adds paramiko logs to console output
* Adds paramiko log handler to default_log_level

Change-Id: I65b3a544e6a6fc0943c733619b20f3e63cf2aa7e
Closes-Bug: #1200621
2013-12-03 22:48:21 +05:30
Jenkins
65841cf0a6 Merge "Adds improvements to the Swift TempURL test" 2013-12-03 10:15:06 +00:00
Jenkins
72c5d1be96 Merge "Enable a uuid flavor" 2013-12-03 01:36:19 +00:00
Joe H. Rahme
3f2d4c63cf Adds improvements to the Swift TempURL test
Adds one extra test, to cover the Temp-URL-Key-2 header. It also adds
the possibility to skip this test by adding an entry in the config
file.

Change-Id: I084e15b9f405bf880ffd4829f57b283cbc065afa
2013-12-02 12:51:43 +01:00