6320cf53f6
These files are the minimum to start with when configuring tempest to run with refstack-client. Including these as a reference point. Change-Id: I19e1898f2927e0073fe8fd4a9e2e0226435cfe54 |
||
---|---|---|
doc | ||
docker | ||
etc | ||
refstack_client | ||
specs | ||
.dockerignore | ||
.gitignore | ||
.gitreview | ||
.testr.conf | ||
LICENSE | ||
README.rst | ||
refstack-client | ||
requirements.txt | ||
setup_env | ||
setup.cfg | ||
setup.py | ||
tempest-additional-requirements.txt | ||
test-requirements.txt | ||
tox.ini |
RefStack Client
RefStack-client team and repository tags
Overview
refstack-client is a command line utility that allows you to execute Tempest test runs based on configurations you specify. When finished running Tempest it can send the passed test data to a RefStack API server.
Environment setup
We've created an "easy button" for Ubuntu, Centos, RHEL and openSuSe.
Make sure you have
git
installedGet the refstack client:
git clone https://github.com/openstack/refstack-client
Go into the refstack-client directory:
cd refstack-client
Run the "easy button" setup:
./setup_env
Options:
a. -c option allows to specify SHA of commit or branch in Tempest repository which will be installed.
b. -t option allows to specify tag in Tempest repository which will be installed. For example: execute
./setup_env -t tags/3
to install Tempest tag-3.c. By default, Tempest will be installed from commit cc255bbbf431e114a4fc0ef587cd3d72333f750a (October, 2017).
Usage
Prepare a tempest configuration file that is customized to your cloud environment. Samples of minimal Tempest configurations are provided in the
etc
directory intempest.conf.sample
andaccounts.yaml.sample
. Note that these samples will likely need changes or additional information to work with your cloud.Note: Use Tempest Pre-Provisioned credentials to provide user test accounts. :
Go into the refstack-client directory:
cd ~/refstack-client
Source to use the correct Python environment:
source .venv/bin/activate
Validate your setup by running a short test:
refstack-client test -c <Path of the tempest configuration file to use> -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
Run tests.
To run the entire API test set:
refstack-client test -c <Path of the tempest configuration file to use> -v
To run only those tests specified in an OpenStack Powered (TM) Guideline:
refstack-client test -c <Path of the tempest configuration file to use> -v -–test-list <Absolute path of test list>
For example:
refstack-client test -c ~/tempest.conf -v --test-list "https://refstack.openstack.org/api/v1/guidelines/2017.09/tests?target=platform&type=required&alias=true&flag=false"
This will run only the test cases required by the 2017.09 guidelines that have not been flagged.
Note:
- Adding the
-v
option will show the Tempest test result output. - Adding the
--upload
option will have your test results be uploaded to the default RefStack API server or the server specified by--url
. - Adding the
--test-list
option will allow you to specify the file path or URL of a test list text file. This test list should contain specific test cases that should be tested. Tests lists passed in using this argument will be normalized with the current Tempest environment to eliminate any attribute mismatches. - Adding the
--url
option will allow you to change where test results should be uploaded. - Adding the
-r
option with a string will prefix the JSON result file with the given string (e.g.-r my-test
will yield a result file like 'my-test-0.json'). - Adding
--
enables you to pass arbitrary arguments to tempest run. After the first--
, all other subsequent arguments will be passed to tempest run as is. This is mainly used for quick verification of the target test cases. (e.g.-- --regex tempest.api.identity.v2.test_token
) - If you have provisioned multiple user/project accounts you can run
parallel test execution by enabling the
--parallel
flag.
Use
refstack-client test --help
for the full list of arguments.- Adding the
Upload your results.
If you previously ran a test with refstack-client without the
--upload
option, you can later upload your results to a RefStack API server anonymously or with your digital signature. By default, results uploaded with signature are private and you can decide to share or delete the results later. Uploading results with signature is the recommended option.Following is the command to upload your result anonymously:
refstack-client upload <Path of results file>
The results file is a JSON file generated by refstack-client when a test has completed. This is saved in .tempest/.testrepository. When you use the
upload
command, you can also override the RefStack API server uploaded to with the--url
option.Alternatively, you can use the
upload-subunit
command to upload results using an existing subunit file. This requires that you pass in the Keystone endpoint URL for the cloud that was tested to generate the subunit data:refstack-client upload-subunit --keystone-endpoint http://some.url:5000/v3 <Path of subunit file>
Result can also be uploaded with signature by adding
-i <path-to-private-key>
option. Intructions for uploading data with signature can be found at https://github.com/openstack/refstack/blob/master/doc/source/uploading_private_results.rstCreate a JSON web token to use for authentication to your privately uploaded data
In order to authenticate to the refstack-server to which you have uploaded your data, you will need to generate a JSON webtoken. To generate a valid token, use the command:
jwt --key="$( cat %path to private key% )" --alg=RS256 user_openid=%openstackid% exp=+100500
To test authentication in the API, use the command:
curl -k --header "Authorization: Bearer %token%" https://localhost.org/v1/profile
List uploaded test set.
You can list previously uploaded data from a RefStack API server by using the following command:
refstack-client list --url <URL of the RefStack API server>
Tempest hacking
By default, refstack-client installs Tempest into the
.tempest
directory. If you're interested in working with
Tempest directly for debugging or configuration, you can activate a
working Tempest environment by switching to that directory and using the
installed dependencies.
cd .tempest
source ./.venv/bin/activate
and run tests manually withtempest run
.
This will make the entire Tempest environment available for you to
run, including tempest run
.