Add a parameter in tempest.conf "full_clean_stack" to
allow/prevent the use of cleanup() method at the end
of a stress test run.
This is mandatory to use stress tests in a multi-user
environment since the actual cleanup() is designed to
remove every object of every tenant (and thus will
impact other users on the same openstack)
Change-Id: I42dc9592617aeeca9660f670940ecbf50b6108ab
Closes-Bug: #1273254
this ports the flavors, server_password tests and corresponding client
into v3 version.
Partially implements blueprint nova-v3-api-tests
Change-Id: Iaf2056d56429898d524536e54296bac5e479fd3e
this ports test_volumes_list into cinder v2 api,
and also ports corresponding client.
1. changes of the response data of list and list details in v2:
(their items are same in v1)
- list view includes items:
id, name, link
- list details view includes items:
id, status, size, availability_zone,created_at, attachments, name,
description, volume_type, snapshot_id, source_volid, metadata,
links, user_id, bootable
2. use 'name' instead of 'display_name'
v2 commit id:
0c507aa6d08a9471bf896961cc99d40f337f1e4d
484fb9e1c748e718349e451e80a0e025ed007b1d
Change-Id: I19cf00270f0ca55770188c6c71c02e13c5bd82f6
Implements: blueprint cinder-v2-api-tests
This commit adds a config option to the identity feature enabled group
to specify whether the v2 and/or v3 apis are enabled in keystone. In
the process of adding this feature the logical split between v2 and v3
tests in the identity api tests is made more clear. Additionally as part,
of the split several bugs are fixed where the v2 api was being called
in the v3 tests.
Change-Id: I0d727f3567c82fe512b956110bb813119ced0b3d
This add an result check if a invalid resource is given. Introducing a
new dict that can be enhanced later for other cases.
Change-Id: Ief3e51fafb9437a8e6b9a71aad79459716bac3d4
Partially-implements: bp negative-tests
Address last comments from part1:
- replace auth_data[0] with get_token()
- add token property to rest_client
- Remove comment about planned changes from config.py
Partially implements: bp multi-keystone-api-version-tests
Change-Id: Ia54cd3d824e44dd99d9226d4c0c6d6861d8c12ae
New parameter "leave_dirty_stack" available in tempest.conf
to prevent (or not) the execution of tearDownClass()
after an exception during a stress test
Closes-Bug: #1273245
Change-Id: I89ecac1fb428cf8809a3634869f2dfdf564aea64
Moves authentication from rest_client to an external auth_provider,
assigned to the client by the client manager.
The auth provider can decorate a request based on credentials (coming
from the client manager) and filters (region, service, endpoint_type)
given by the client. The auth provider can also return the raw
auth_data, which in the Keystone implementation is a tuple (token,
auth_data). The auth provider allows mixing multiple credentials
when decorating a request, possibly using empty or invalid credentials,
to facilitate negative tests.
The auth provider caches auth data, so that all API requests for a
specific set of credentials only require a single call to obtain a
token, unless the token expires or is forcefully deleted from the
auth provder.
Two implementations of the auth provider are included: Keystonev2 and
Keystonev3. The Manager object behaves as factory of auth_providers,
building the correct one based on the configured auth_version, and on
the interface type (JSON or XML).
Fixes endpoint selection for v3 auth. Drops unused basic_auth.
Extends TokenClients to provide both token and auth data,
and accept different combinations of credentials for v3.
Removes redundant server_client_v3_auth.
Adapts tempest unit tests to work with modified rest client.
Introduces a configuration parameter for authentication version to be
used. That is used when instantiating the client manager, and it
applies to all clients used by api tests.
Next steps (next patches):
- move to credentials as dict (as opposed to tuple)
- expose a get_client method from the client manager
and create clients only when requested
- remove redundant CustomizedHeader object storage clients
- supports keystone v3 in tenant isolation
- use Auth Provider in scenario tests
- use Auth Provider in CLI tests
- implement parsing of catalog XML format (?)
Partially implements: bp multi-keystone-api-version-tests
Change-Id: Icfa921e9051c01f339f8d2471b12d6ec950cc456
Generates and executes negative tests based on a json schema for the API.
It's using testscenarios lib to generate a test case for each variation.
It contains resource handling and result proofing of invalid requests.
Not covered in this patch (will be done later):
- Documentation
- Unit tests
- Actual json schemas for all APIs
Partially-implements: bp negative-tests
Co-author: David Kranz <dkranz@redhat.com>
Change-Id: I828f9b74c31f2e25c91e8149ecd7cd0b189ce99a
This commit takes all the uses of config in the cli tests and coverts
them to use the global CONF object.
Partially implements bp config-cleanup
Change-Id: I7a39839d7a4ebbf8372489507486ad2cac7f3adc
Changes default in the input-scenario settings to match only
one image and one flavor, to avoid consuming gate time.
Fixes bug 1274628
Change-Id: Ide865012269d3f60250d24443e4f20af263ee021
The Cirros image only sends out three DHCPDISCOVERs with 60 second
waits between. The ovs_neutron_agent may not have the port wired in
time for the first one, and with ping_timeout=60 the test will be
terminated before the second one is sent. This patch increases the
timeout to 120 seconds to prevent that.
Change-Id: I03fcec50b91fd89ad64e16fce82f9211c2e086a1
Partial-Bug: 1253896
These log errors have hit the large ops jobs several times. There is one
in ceilometer acentral and one in nova conductor.
Change-Id: Idc30110085e95c615958fc5f90e86417855e6d7a
Adding the ability to generate multiple tests from
a template one based on input scenarios.
The implementation is based on testscenarios (added
to requirements.txt), and the correct behaviour
could be verified using unittest or testr + subunit,
both serial and parallel runs.
The generation of the array / matrix of tests relies
on the load_test protocol. nosetests does not implement
the protocol by default.
Static scenarios can be added to any tests with very
little effort. This commit introduces a couple of
dynamic scenarios:
- all flavours matching a configurable regex
- all images matching a configurable regex
Dynamic scenarios are coded in test_utils, and provided
by InputScenarioUtils. The class includes in the doc
an example on how to use input scenarios.
Additional scenarios could be AZs, or credentials.
This commit adapts existing test_server_basic_ops
to use input scenarios, and include server
ssh verification.
This change implements blueprint input-scenarios-for-scenario.
Implements: blueprint input-scenarios-for-scenario
Change-Id: Ia86f48772ac02d67faa1c7d764cb9dc0938f6452
The skipping reason was bug #1093743 that
is currently marked as invalid in Swift bugtracker.
This patch remove the skip and propose some
refactoring to the test code.
The test is marked as slow as it can take long
to run as it depends on container-sync process run
internal which is 300 seconds by default.
Change-Id: I939a014646b42cb2a461f874a5ad0edfc79c5561
This commit adds a new config section identity-feature-enabled which
contains the options to select which optional/configurable features in
keystone are enabled. The only option that fits this category at this
time is trusts.
Partially implements bp config-cleanup
Change-Id: I8110156e48616534c45e9eb2c12557b2d6bdfd46
This commit adds a new list option for the object-storage-feature-enabled
config group, discoverable_apis, which lists out which Swift discoverable
optional apis are expected to be enabled on the swift server. It also
converts the pre-existing single boolean options for these types of
features to use the new list option. This option is used in the same
manner as the extensions list options for the other services. However,
since swift doesn't have the same concept of extensions as some of the
other OpenStack projects optional discoverable apis (which are often
but not always tied with middleware) were used instead. If an optional
api being enabled isn't discoverable from an external api call then
that should have a separate config option in the
object-storage-feature-enabled config group.
Partially implements bp config-cleanup
Change-Id: I846860804050ec44491ce05d6b72e1cc639609b1
Ironic is a baremetal provisioning service that
is intended to replace nova-baremetal-driver.
Recently it was integrated to devstack so now
it's reasonable to start testing it with tempest.
This patch adds a client for baremetal
provisioning service and some tests for the Ironic API.
Change-Id: Ifd65d6a60179e72dbfa81825f234f0ff76ebb055
this is largely to deal with config generator issues, however
we're apparently quite far behind on other items, so lets just
sync it all.
Change-Id: I2f36e41b5d7cce8dd0b64636e45da3f7481b0e39
This patch add the configuration for Telemetry.
Partially implements: blueprint add-basic-ceilometer-tests
Change-Id: I8859857c5cf469045f66631c3d52edcb765035d7
Change volume base class name:
BaseVolumeTest -> BaseVolumeV1Test
BaseVolumeAdminTest -> BaseVolumeV1AdminTest
for the support of cinder V2 test class
Implements: blueprint cinder-v2-api-tests
Change-Id: I712a90d39981564af09393ed1b7f4be12897c808
According to John Dickinson:
"This isn't an error. It's logged at an error level because it may hint
at other problems, and it's something an operator needs to know about,
but it is not an unhandled failure condition. (Think of it similarly how
in swift a server being down is an "error", but it's something that
swift seamlessly works around.)"
Closes-Bug: #1260894
Change-Id: I41e55c5e34ee214727fbbd7b9daa1f6ea9bf8050
This commit adds a new set of config options to the feature_enabled
groups for a list of enabled extensions. These options are used to
specify whether all extensions are enabled or which subset is expected
to be enabled. This just sets up the initial framework for doing
this and converts the FlavorExtraSpecs tests to use it.
paritally implements bp config-cleanup
Change-Id: I6a5a9b16e62eb8a216334a0662c99f0dd0d16873
It looks like the whitelist for the DHCP agent is added
twice. This looks like a genuine typo, and if it isn't
there is clearly something wrong! There's no good reason
for duplicating the list of whitelisted errors.
Change-Id: Ib94f60e8aa3ce1b0a6e28d9051da0124d403bf27
Adds one extra test, to cover the Temp-URL-Key-2 header. It also adds
the possibility to skip this test by adding an entry in the config
file.
Change-Id: I084e15b9f405bf880ffd4829f57b283cbc065afa