Commit Graph

74 Commits (cd147631f8254c2f727ccfc937566a7a200349f9)

Author SHA1 Message Date
Soniya Vyas 0c84f3ee2e Need to have stable tempest scenario manager
This patch addresses following facts:
1. common manager methods among plugins should be
   defined in Tempest.
2. methods should be consistent with names and
3. Add docstring about method usage and parameter.

Implements: blueprint tempest-scenario-manager-stable
Signed-off by: Soniya Vyas<>
Change-Id: I191c8861c9ec29b78c9c8ad12ff2ddf3ffbf4db1
3 years ago
Jenkins 1238eabef7 Merge "Revert "Move dscv and ca_certs to config section service_clients"" 7 years ago
Daniel Mellado cad3f3d834 Revert "Move dscv and ca_certs to config section service_clients"
This was a backwards incompatible change that breaks tempest's stability
guarantee on tempest.config. While the option was marked as deprecated,
oslo.config does not alias deprecations in code. The fact this patch
had to update unit tests was an indication of this. This breaks anyone
who consumes these options via code. We can only land something like
this after we add aliasing to oslo.config for deprecations.

This reverts commit 1afca56b05.

Change-Id: I91600d8fc49db670b0f78ffd0094575fbfa0ca63
7 years ago
Jenkins 68fc8b093a Merge "Move dscv and ca_certs to config section service_clients" 7 years ago
ghanshyam 9d7bac448a Fix manager->get_auth_provider interface
Tempest is defining stable interface of service clients
handling in lib but need to maintain the old interface
tempest.manager.get_auth_provider() as it is used by many
plugins and provide them a grace period to move to
stable interface.

This interface got broken while doing service clients work.
This patch fixing the same and fix gate for plugin using
this unstable interface.

Change-Id: I5733f20f5fba3c1aa1318405f5af976513eebf14
7 years ago
zhufl 1afca56b05 Move dscv and ca_certs to config section service_clients
In a new config
section "service_clients" is added which hosts parameters
common to all service clients. ca_certificates_file and
disable_ssl_certificate_validation are also common to all
service clients, so this is to move them from section identity
to section service_clients

Change-Id: I296f1080ce89f0cdceae1c476866b215393b2605
7 years ago
Andrea Frittoli (andreaf) e07579c603 Migrate service_clients to tempest.lib
Migrate the service_clients module to
Migrate related unit tests as well.

The clients module atm imports from Tempest which is not
allowed via hacking to avoid cirtular dependencies.
If there is no way around this, I will have to remove the self
registration of the service clients from plugins, and ask the
plugins to do the registration themselves - which is a pity. Ideas?

Change-Id: I40e3478f69af62a7cdc14fa65ed21dcfbbe10e72
7 years ago
Andrea Frittoli (andreaf) 2395014351 Prepare the Manager class for tempest.lib
Remove CONF dependencies from the client manager base class,
to make it useful to external consumers (such as plugins).

The ultimate target is to have a manager which can be used with
as little as possible setup, which only instantiates the clients
which are actually needed by the test, and which allows to register
new service clients defined in plugins.

Since plugins already import both manager.Manager and
clients.Manager, we maintain for now both classes with their names.
The plan is to migrate the 6 core service client groups to
manager.Manager, so that those clients are available to all tests
along with plugin clients. That requires a few steps. I'm doing
changes in clients.Manager for now so that it's easier to review.

The result of this first step is:
- manager.Manager is moved to manager_lib.Manager and does not
  depend on CONF anymore, nor on any tempest unstable class.
  It does not provide any client yet.
  Add unit test coverage for this class.
- manager.Manager is still provided with backward compatible
  interface for plugins benefit.

Change-Id: Ic9ccc7037d15cdd4c6f1749eaeda13d4e7ee0114
Partially-implements: bp client-manager-refactor
7 years ago
Andrea Frittoli (andreaf) af4f7cfe90 Cleanup exceptions in tempest
There is a number of exceptions available in tempest.lib which are duplicated
in the exceptions module in tempest. They are all identitical, except for one:
TimeoutException. The tempest.lib version inherits from RestClientException,
while the tempest version is a TempestException. The only class that should
use the former is the RestClient (and derivative), while it is used by ssh
client as well. Changing the type of exception raised by would be a
backward incompatible change, so I think we cannot change this now.

Change-Id: I98ad80266b00f7d0b9d269a516866d81312aa834
7 years ago
Andrea Frittoli (andreaf) 848c4a1488 Stop passing TestResources to Managers
Dynamic credential providers generate TestResource objects, which
behave like Credentials but also include network resources.
We used to pass those directly into client managers, but that has
several issues: the interface of the client manager is confusing,
because it accepts two different type of objects in one parameter.
Having the client manager depending on the dynamic credential provider
is a problem because in turn that depends on the client manager,
which may lead to circular dependencies when trying to make those
stable interfaces for tempest.lib.
Finally TestResources seem to confused developers, since a number of tests
ignored the possibility of getting Credential fields from TestResources
directly, and went through the inner credentials atttribute.

I think the best approach for now is to pass auth.Credentials objects
only into the client manager. The client manager does not use the
network resources anyways.

We plan to refactor the link between credential provider and network
resources anyways, so it's better to have that outside of client

Change-Id: I71d4587bf0b640d05d0f78fb84364ce4d2d021f3
Partially-implements: bp client-manager-refactor
7 years ago
Andrea Frittoli (andreaf) 3e82af7f6c Introduce scope in the auth API
Adding the ability to select the scope of the authentication.
When using identity v3, this makes it possible to use either
project scope or domain scope regardless of whether a project
is included or not in the Credentials object.

The interface to auth for most tests is the AuthProvider.
The scope is defined in the constructor of the AuthProvider,
and it can also be changed at a later time via 'set_scope'.

In most cases a set of credentials will use the same scope.
Test credentials will use project scope. Admin test credentials
may use domain scope on identity API alls, or project scope on
other APIs. Since clients are initialised with an auth provider
by the client manager, we extend the client manager interface to
include the scope. Tests and Tempest parts that require a domain
scoped token will instanciate the relevant client manager with
scope == 'domain', or set the scope to domain on the 'auth_provider'.

The default scope in the v3 auth provider is 'projet;, which me must
do for backward compatibility reasons (besides it's what most tests
expects. We also filter the list of attributes based on scope, so
that tests or service clients may request a different scope.

The original behaviour of the token client is unchanged:
all fields passed to it towards the API server. This
maintains backward compatibility, and leaves full control
for test that want to define what is sent in the token

Closes-bug: #1475359
Change-Id: I6fad6dd48a4d306f69da27c6793de687bbf72add
7 years ago
Andrea Frittoli (andreaf) db9672e347 Use tempest.lib code in tempest
Now that tempest_lib code is copied back into tempest, stop using
tempest_lib in tempest, and start using the copied code.

Remove the dependency to tempest_lib from requirements, and drop
the script to use tempest_lib in tempest. Add os-testr to the

Partially implements bp tempest-lib-reintegration

Change-Id: I21ab5fe6349f72c98ac9f960a29bf62e813f8b1b
7 years ago
Jordan Pittier 5f4964c898 Remove some dead code
This code is not used anymore.

Change-Id: Ia3b341721510b4886f5288e67cfe3935ca204de6
7 years ago
Ken'ichi Ohmichi 2e2ee19179 Fix H404/405 violations for remaining
There is a lot of H404/405 violations in Tempest now, and that leads
difficult to migrate the code to tempest-lib or the other projects'
repos. This patch fixes these violations for remaining.

Change-Id: I3abac40ebb33836980c119d59bc97f035b213c0a
8 years ago
Andrea Frittoli (andreaf) 290b3e1383 Factor up config dependent credential classes
Separate helpers methods to build credentials, credential providers and
client managers that depend on configuration. They are all moved to
the common.credentials module to achieve two objectives:
- prepare credential provider module to move to tempest-lib
- avoid circular dependencies

There are still dependencies to CONF in the credential providers,
removing all of them would make the patch too large; but there is
with this patch a single place where all relevant CONF parameters can
be looked up and passed into the credential providers.

Partially-implements: bp tempest-library
Change-Id: I351f86f0570070105bbf24f2d904e917feea6e57
8 years ago
Andrea Frittoli (andreaf) c625bcfb52 Stop validating pre-provisioned credentials
Stop validating pre-provisioned credentials when requested
within the credential provider. This drops the need for
estabilishing connection to the identity service, and
reduces the amount of configuration that has to be passed
into the credentials provider for it to work.

This work is in preparation to the migration of the
pre-provisioned credentials provider to tempest-lib.

Change-Id: I825a01ff72cb3a937aafeb2104333db7113ef4d0
8 years ago
Andrea Frittoli (andreaf) f9e0126174 Rename accounts to preprovisioned accounts
Rename accounts to preprovisioned accounts and related
unit tests as well, in preparation for migration to

Partially-addresses: bp tempest-library

Change-Id: Iabf4b42e1beadb0540411b7a16cd494b081aabc5
8 years ago
andreaf b8a52287e7 Drop auth and corresponding unit tests
auth has been migrated to tempest-lib.
Fix all auth imports to use tempest-lib. Drop auth and related
unit tests.
Use token client from tempest-lib as well.

Depends-on: Ie6435b4f3a367b0a8cec68f21c0b4f5f61d6b688
Change-Id: I8c2772d9fb42d352f4a1d3e74e20ce6e8f483559
8 years ago
Andrea Frittoli (andreaf) 9540dfd4d7 Embed network resoruces and credentials in TestResources
Define a new class TestResources which embeds credentials and
their network resources.
This makes it easier to obtain network resources for a set
of credentials in case of preprovisioned accounts.

Partially-implements: bp test-accounts-continued

Change-Id: Ie3d5e2471db7371e1109fd24a716e7c449843904
8 years ago
Andrea Frittoli 90012355b4 Prepare token clients for migration to tempest-lib
Prepare token clients for migration to tempest-lib together
with Removing all the dependencies from CONF, and thus
introducing new parameters in auth providers as well.

Change-Id: I7c03b07ec1b9268fea68125b5bbfa8fe0ff75b71
8 years ago
ghanshyam 5ff763f5ac Remove CONF values from Token clients
To move Token clients to tempest-lib, this patch moves
CONF values from Token clients to the client setting and classes
instantiate Token clients.

Also making necessary changes in tests files.

Change-Id: Ie405ba6a71d29258e99d2f0b68afb013a9619e9b
8 years ago
Andrea Frittoli c097835388 Drop the legacy and un-used _interface
We used to run JSON and XML tests. XML has been dropped now,
but still in the code there are remains of the logic to switch
between JSON and XML. Cleaning them up.

Partially-implements: bp/resource-cleanup

Change-Id: I39ecfbd99861136a6a2adc3ce56bfdbba96d5c9b
8 years ago
Andrea Frittoli 9efbe95571 Move default creds to cred_provider
Credential types available in configuration, and how to parse
them and validate them is currently embedded in the Credentials
class hierarchy.

Move the entire logic to cred_provider, in preparation for the
migration of the auth module to tempest-lib.

The pre-provisioned account credential provider relied on
the list of configured field to find the hash for a set of
credentials. Changed that to use the list of attributes with
which the credentials were initially setup.

Change-Id: Ic3ff15799f016277e6eb59c53cf8f24f6a7be9c9
8 years ago
Andrea Frittoli 455e84498e Drop client_type for auth module
Client type was designed to allow the auth module to consume
the official keystone client for tests based on the official clients.
Scenario tests have been migrated to tempest client, and CLI tests
are being moved out of tempest, and they don't need an external
auth provider anyways.
There's no need for client_type anymore, removing it.

Change-Id: I6eebd80ea2259b72e8013e65e9a83615f86a3f84
Partially-implemnts: bp/tempest-client-scenarios
9 years ago
Andrea Frittoli 1781188e1a Select AuthProvider type from credentials type
auth_version is used to get defaul credentials as well as to
select the AuthProvider class. If credentials of an API version
different from the configured one are used, a mismatch happens.

Instantiating the AuthProvider class that corresponds to the
version of API represented by the credentials object.

Closes-Bug: #1369557

Change-Id: I1ed6da90ce0abed805cf39190cb1ac3baa3f63b6
9 years ago
Andrea Frittoli 422fbdf0a0 Enforces the use of Credentials (part2)
Multiversion auth part5

Refactor mangers, utils and test base classes to use Credentials instead
of username, password and tenant_name.
Makes changes to tests where needed - some of the tests create their own

Partially implements: bp multi-keystone-api-version-tests

Change-Id: If05f5704d90390362cebf45e2664f2bfbc72268d
9 years ago
Andrea Frittoli 3099ffb85d Object storage tests to use default auth_provider
Object storage test uses get_auth_provider(), it should
just take the existing auth_provider instead.

Partially implements: bp multi-keystone-api-version-tests

Change-Id: I790c22becb28ef8c2946efcc4b1c47f8e295fb74
9 years ago
Andrea Frittoli f9cde7e942 Refactor Managers to a common base class
Multiversion auth part3

Refactor client managers to inherit from a common manager.Manager
class. Moves scenario base manager to

Partially implements: bp multi-keystone-api-version-tests

Change-Id: Iddacbaa4593b7cb4d32538a5cade814751c180e0
9 years ago
Matthew Treinish bc0e03ef6a Remove last uses of config without global CONF object
This commit removes that last uses of config without using the global
CONF object.

Partially implements bp config-cleanup

Change-Id: Idc649993b7cd5a45282fde827b2b08f6fb19ab4d
9 years ago
Masayuki Igawa e8d31a0b80 Remove vim headers
We don't need to have the vim headers in each source files. We can set
it in our vimrc file instead. Also if set incorrectly, gate (pep8 check)
will catch the mistakes.

More discussions:

Change-Id: I43703e2289212389c7841f44691ae7849ed1f505
Closes-Bug: #1229324
10 years ago
Sean Dague 86bd8427e3 moving to global lazy loaded config
one of the problems we've got with tempest is the fact that config
loading is tied into the class hierarchy. However there is no
reason why it should be. If we instead create a config proxy
object we can lazy load the actual config when we are executing,
and not do it at import time.

This could use future iteration, but it does a huge job in
removing config from the object inheritance tree which massively
simplifies our ability to use config variables throughout the code.

Change-Id: I9b1bbfe231c85c01938bd68be4e5974bd24130d6
10 years ago
ZhiQiang Fan 39f9722500 Replace OpenStack LLC with OpenStack Foundation
Some files still use trademark OpenStack LLC in header, which
should be changed to OpenStack Foundation.

NOTE: tools/ is not synced from oslo-incubator, so
it is modified too.

Change-Id: I0084684ff5225076291fea857eb010d638291bd0
Fixes-Bug: #1214176
10 years ago
Attila Fazekas d6d8629107 Simplify whitebox/manager
* Allow the generic resource deletion (thing.delete) to
operate on resources allocated in the setUpClass-es too.

* is using the python
  client libraries as the scenario tests.

* Remove the not used part of the tempest/

Change-Id: I980a70d22dc7a3a65a26a8197c7888b125eb3b05
10 years ago
Sean Dague 43cd905141 fix use of locals() in strings
hacking 0.6 brings in H501, don't use locals() in strings, as it
has the ability to hide errors. This fixes it in preparation of
landing hacking 0.6.

Change-Id: I7972befbb945a27ed8aa601975205736038eefa2
10 years ago
Li Ma 216550f81a Remove basic_auth strategy
The auth strategy selection is removed
since it is confusing and other authentication
methods are not supported.

Re-commit the files, because the previous update is not correct.

Implements: remove basic_auth strategy
Fixes: bug #1180972
Change-Id: I498be0b1e3eeea397e1f164caffe7cc88be0ceda
10 years ago
Tony Yang 3d5f16331c Tests for os-hypervisors API extension of Nova
Tests against following paths are added:

Implements: blueprint nova-os-hypervisors-api-extension-test
Change-Id: Ia209b113d0193720b445385e0fe399b81f48bb4b
10 years ago
Mitsuhiko Yamazaki 46818aaad8 Configure logging format flexibly
Now we can get tempest log file with options in
This adds and enables to configure log format more flexibly
using configuration file.
This adds a LoggerAdaptor and a Formatter to output each test name
to log lines.

Implements: blueprint add-logging-configuration

Change-Id: I88cf18bb8bbc152e62ac83a9c7dc26067b7a11bd
10 years ago
Sean Dague 6dbc6da416 add scenario directory
this adds the scenario directory as part of the tempest
restructure, and moves scenario tests over to the new directory.

To ensure we got everything that used the old DefaultClient and
DefaultManager classes these are refactored out into the scenario
directory as OfficialClient and OfficialClientManager.

The Nework Smoke test base class becomes NetworkScenarioTest, and
the two test files that used that come over to scenario directory
in the process.

These are all done with a single tempest/scenario/ file
for now. As the scenario tests grow we'll figure out better
refactorings there.

This gives us 4 scenario test files as a starting point for future
scenario work.

Fixed remaining refactor issues with network tests

Clean up the documentation on the extracted classes

Part of bp:tempest-repo-restructure

Change-Id: I143b282c58cfac3bf979ba5ac68226155beff343
10 years ago
Attila Fazekas 36b1fcf417 Add a volume from snapshot test case
Fixes bug #1034513

Change-Id: Ie37a0ae59c2dc2d805113c73a824951acef13663
10 years ago
Attila Fazekas e4cb04c438 Merge console_output_client to server_client
Change-Id: Iae11ef91c426dc4c12996b1b815ce09a1f6c0d18
10 years ago
rajalakshmi-ganesan 1982c3cac9 Addition of XML support to
Added logic to file so as to support XML calls.
Hence added exclusive client files for XML. Also modified JSON client
file, and according to new addition of
XML support.

Change-Id: Ibbdd87c2d1bb4e26ff0f77441dc91d71abc00d7f
Implements: blueprint add-xml-support
10 years ago
Attila Fazekas cadcb1f948 Credentials Configuration changes
* Credentials are moved to the identity section
* If the compute admin user defined in the compute-admin section it takes precedence
* The region just defined in the identity section

Change-Id: I1241116ce9312c90656a78235a4a91dd31460761
11 years ago
Attila Fazekas 407b6db5b7 Refactor identity
* Remove identity-admin section
* Remove network-admin section
* Remove image credentials
* "admin" word changed to "identity" where it makes more sense

The credentials will be moved to the identity section, in a future

Change-Id: Id443ea12d32f1de78487084e7364774efa838aec
11 years ago
Jay Pipes 7c88eb2b3e Allows identity endpoint to be specified as URI
Deprecates the piecemeal identity URL buildup and
replaces with a simple uri configuration setting.
Gets rid of the /tokens path stuff that was not
necessary to have in configuration file.

Change-Id: I5e80177073f756781f813438d054fede2d6334a3
11 years ago
Jay Pipes cd8eaec4fe Adds setting to disable SSL cert validation
* Breakout of prior large patchset. This patch only adds the
  configurability of SSL cert validation in all the clients.

Change-Id: I48385eabd617d19705f3f2bff4820566547be56d
11 years ago
rajalakshmi-ganesan d793d69e2b Addition of XML support to
Added logic to file so as to support XML calls.
Hence added exclusive client file for XML. Also modified JSON client
file and according to new addition of XML support.

Change-Id: Ib1fab3e3dc97d3b61ceb60e0d846c4b36f472932
Implements: blueprint add-xml-support
11 years ago
Maru Newby 81f07a09ba Add smoke tests for quantum.
* Added test_network_basic_ops, a port of the devstack exercise
 * Tenant network connectivity can be tested by setting
   the 'tenant_networks_reachable' key in tempest.conf to 'true'.
 * Public (floating ip) connectivity can be tested by setting the
   'public_network_id' key in tempest.conf.
 * Addresses bug 1043980

Change-Id: I506518c431a8da0b91e3044f2a6aabce48081d93
11 years ago
Maru Newby b72f37cfb4 Add admin credential config for network client.
* Supports bug 1043980

Change-Id: I5ffc2d57a19e9f3a8112308998f0b957b7d93b1f
11 years ago
Matthew Treinish a83a16ead5 Fix import order to comply with import ordering rules.
Fixes pep8 failures TEMPEST N306 and N301.

Change-Id: I87d6ef4058170da73ff64d4e4a4caf43f8a18046
11 years ago
Rohit Karajgi 07599c524b Adds a Quotas client for Nova
* Adds a client for the 'os-quota-sets' extension
* Adds basic Admin and non-admin tests for GET and PUT operations for the
Quotas API
* Adds some tests to check Quota enforcement for create server (bug 1034453)

Fixes LP Bug #1040760
Fixes LP Bug #1034453
Change-Id: I7eb0041dbc80d8733bb2df54e4fc4755cfe9ae9c
11 years ago