This commit adds a check to subunit-trace to fail if the test run
doesn't actually run anything. Despite technically passing without
running anything, if you're passing a regex to testr that runs 0 tests
that is probably an error, so treat it as such. This also uncovered a
bug in the test_wrappers unit test where the test was using a bad test
regex but still passsing. Since we start enforcing it here the unit
test is also updated to use the proper regex.
Change-Id: I109a8cd6eb731e5b8daf5cc2d6a76498ddd48c6d
The check logs script was not properly detecting grenade and would fail
grenade jobs if the logs were not clean. We don't want to do this
because the upgrade process is not log clean yet. Correct this by
setting grenade flag to true only if the DEVSTACK_GATE_GRENADE variable
has content.
Change-Id: Id8a339d78d981376b9af3a80aae8e4e18a68a85f
After bumping the hacking version to the 0.9.x series ignores were
added for several rules. This commit fixes the violations for a subset
of these rules and re-enables the checks.
E123 is ignored because this rule is ignored in the default pep8 by
default, due to a lack of consensus around it.
Change-Id: I8f14cd2ca6afc38d3fe8ee758272071111022896
This commit removes the generate_sample wrapper script and adds an
rc file for the config generator to specify the alternate path.
This makes console messages from the generator tooling point
consistent with what is actually being used. Before because a wrapper
was used error messages would recommend running the incorrect tool.
Change-Id: I0ac792454777af6e3f1d718836031f9a14850229
This commit cleans up how tempest failure logs are displayed. We'll
no longer dump the failure information to the console as a test fails.
Instead all the failure logs will be printed after the tests are run.
Change-Id: I7ecdc349d913b43f4fb0505d5c17c66f811774b4
This script isn't being used by anyone to my knowledge nor has it
really been maintained since it was first added. We will be working
on a plan for automating tempest configuration soon and this
doesn't really fit the initial set requirements of requirements. It
also does not have any testing, so I expect that it has bit rotted
and probably doesn't work anymore.
Change-Id: Id896d9877312dfb6b12b54ea1c598a1940cdd9ce
a summary report that includes counts of the test results so far
as well as a worker report to let us understand how our workers
are balanced at the moment, as well as if we are missing worker
0 in our data (which seems to happen quite often).
Change-Id: Ide0b63b86e87f8a18c2f0e45264692ac3797c791
A live filter for subunit stream that will let us display the
concurrency of the streams as we go.
* This includes the worker id as {#} at the beginning of the lines
* Dumps out stdout/stderr if they are found inline (makes for easier
debug
* Dumps out pythonlogging on failures if found inline
* Prints skip reasons
Based on I1b529546e005f47aba56b451e1c0d8b0da09fca3, but because
that started as Robert's I couldn't un Abandon it.
Change-Id: Icc99b652e4e8ae85b73bb905a3b704447a63195f
Co-Authored-By: Robert Collins <rbtcollins@hp.com>
This commit moves the verify_tempest_config script from the tools dir
to tempest.cmd and adds a pbr entry point for the script. This means
that the script will in effect be a packaged binary for tempest.
Partially implements bp config-verification
Change-Id: I75b9743a8117d55c6c445db05eb39404b5708624
This commit adds a new flag to the verify_tempest_config.py script,
'-u' that will update the config file from values found during the
verification process.
Partially implements bp config-verification
Change-Id: I20b8bca7d3368b56da0a0496ed7aa3c1a6eacc9e
This commit adds a check for the cinder api versions that are
supported and checks that with what is set in the config file.
Partially implements bp config-verification
Change-Id: Ifce002a6f42c1565992299965bcc490f20aa5887
This commit updates the service list for verify_tempest_config. The
list was not updated to include new options for marconi and trove. It
also wasn't updated for the sahara rename.
Partially implements bp config-verification
Change-Id: I00500e104dda34098cf445f4bbd522ea843fcabf
This commit fixes a couple of oversights in the common api version
check function. The previous version of the method would never work
because the url wasn't passed to urlparse, and the resulting endpoint
url would be missing a ':' even if it were.
Change-Id: I4b6bb47f0601063aa8164ded386ca71adc6ccc80
we've seen no n-sch errors in passing gate runs in the last week,
I consider this sufficient proof that we're clean on n-sch for now.
Part of bp:clean-logs
Change-Id: Ife0e0602767e155fc645e07546bed36c2c060a20
This commit adds a check for the keystone api versions that are
supported and checks that with the config file. In the process of
adding the additional check the version verification code is reworked
to use a common get versions method.
Partially implements bp config-verification
Change-Id: Ibb15c53e2b2a1e2a2ad95351692d06389f2ebb06
need to add this to the dirty list to make forward progress.
In order to make the results more readable we also need a line
break before the log file stanza.
Change-Id: I863a8a30384edf27a4ba761c707b5cf673c1f86e
instead of marking the logs that we think should be clean, mark
the ones we think should be dirty. This means no new services can
come in with unclean logs.
remove the whitelist data, as that's currently obsolete, we'll
remove the code for it later.
Change-Id: I4b15c932b78f54ec371aa67c7b4e8248b3f9c0eb
This commit syncs the latest version of the config generator and all
it's dependency modules from oslo-incubator. The justification was
change Iee6cc05cad70963f2a151f595ac376c9338f5774 which adds the oslo
synced version of the check_uptodate script. Which among other things
does proper cleanup of the tempdirs
Change-Id: I56bdd239b328cc38a83f80d14e53cc62d4f97954
This commit adds a check to verify_tempest_config for which services
are available. This is used to do 2 things:
1. Verify that the entries in the service_availble group are set correctly
2. Use for filtering which checks inside the script are valid to perform
Partially implements bp config-verification
Change-Id: Ib0e55f21b83d0746994e7e7e1f9666524e1bb3b7
This commit adds support to the verify_tempest_config.py script
for detecting which swift apis are enabled.
Partially implements bp config-verification
Change-Id: I7f4a1fdc41246b3584aa5d3faca5920ca5c2f6e5
This commit fixes the HTTP GET call to return the list of nova api
versions. The enpoint that the tempest clients use for servers
includes the api version. This is because nova registers each version
api as separate endpoints in keystone's catalog. So this commit fixes
the url to use the unversioned endpoint so the list of versions will
be returned.
Change-Id: Iaf4671d4d1dd285161aa5679648956c086a9119c
Moves authentication from rest_client to an external auth_provider,
assigned to the client by the client manager.
The auth provider can decorate a request based on credentials (coming
from the client manager) and filters (region, service, endpoint_type)
given by the client. The auth provider can also return the raw
auth_data, which in the Keystone implementation is a tuple (token,
auth_data). The auth provider allows mixing multiple credentials
when decorating a request, possibly using empty or invalid credentials,
to facilitate negative tests.
The auth provider caches auth data, so that all API requests for a
specific set of credentials only require a single call to obtain a
token, unless the token expires or is forcefully deleted from the
auth provder.
Two implementations of the auth provider are included: Keystonev2 and
Keystonev3. The Manager object behaves as factory of auth_providers,
building the correct one based on the configured auth_version, and on
the interface type (JSON or XML).
Fixes endpoint selection for v3 auth. Drops unused basic_auth.
Extends TokenClients to provide both token and auth data,
and accept different combinations of credentials for v3.
Removes redundant server_client_v3_auth.
Adapts tempest unit tests to work with modified rest client.
Introduces a configuration parameter for authentication version to be
used. That is used when instantiating the client manager, and it
applies to all clients used by api tests.
Next steps (next patches):
- move to credentials as dict (as opposed to tuple)
- expose a get_client method from the client manager
and create clients only when requested
- remove redundant CustomizedHeader object storage clients
- supports keystone v3 in tenant isolation
- use Auth Provider in scenario tests
- use Auth Provider in CLI tests
- implement parsing of catalog XML format (?)
Partially implements: bp multi-keystone-api-version-tests
Change-Id: Icfa921e9051c01f339f8d2471b12d6ec950cc456
This commit adds a check to see if the service is available before
running the extension list verification.
Partially implements bp config-verification
Change-Id: I722440cf84a44d9c38536a3421972c1cb837969e
We don't need to have the vim headers in each source files. We can set
it in our vimrc file instead. Also if set incorrectly, gate (pep8 check)
will catch the mistakes.
More discussions:
http://lists.openstack.org/pipermail/openstack-dev/2013-October/017353.html
Change-Id: I43703e2289212389c7841f44691ae7849ed1f505
Closes-Bug: #1229324
This commit adds another verification method to the
verify_tempest_config script that adds a check to see if the compute
v3 api is enabled.
Partially implements bp config-verification
Change-Id: Idae7db73c9eba94b7f74ec2f71e20d4267aa582c
This commit adds support for the neutron extension detection to the
verify_tempest_config script. The neutron extensions api behaves
slightly differently from the other services in that several of the
extensions' name field is really a title that includes spaces. This
can't be used for the list option and the tempest code already
assumes that the alias field is used instead of name.
Partially implements bp config-verification
Change-Id: Ib60f191fec1cb6b9d60055a848749b07b8393cf5
Due to several changes being made to the config nomenclature around
features and extensions in tempest the verify_tempest_config script
was actually not valid anymore. This commit fixes the script to use
the updated config structure. Additionally it adds support for all
3 api's in tempest that currently have extensions api clients:
nova, nova_v3, and cinder.
Partially implements bp config-verification
Change-Id: I35c325c5f57803b36906bde13f132b69509f5e85
