refstack-client

refstack-client is a command line utility that allows you to execute Tempest test runs based on configurations you specify. When finished running Tempest it sends the passed test data back to the Refstack API server.

Environment setup

We've created an "easy button" for Ubuntu, Centos, RHEL and openSuSe.

1. Make sure you have git installed

2. Get the refstack client: git clone https://github.com/stackforge/refstack-client

3. Go into the refstack-client directory: cd refstack-client

4. Run the "easy button" setup: ./setup_env

   Options:

   a. -c option allows to specify SHA of commit or branch in Tempest repository which will be installed.

   b. -t option allows to specify tag in Tempest repository which will be installed. For example: execute ./setup_env -t tags/3 to install Tempest tag-3.

   c. By default, Tempest will be installed from commit 98f62ee80673532782bd77158512d8f94d67c128 (15 June 2015).

Usage

1. Prepare a tempest configuration file that is customized to your cloud environment.

2. Go into the refstack-client directory.

   cd ~/refstack-client

3. Source to use the correct Python environment.

   source .venv/bin/activate

4. Validate your setup by running a short test.

   ./refstack-client test -c <Path of the tempest configuration file to use> -vv -- tempest.api.identity.admin.v2.test_roles

   or

   ./refstack-client test -c <Path of the tempest configuration file to use> -vv -- tempest.api.identity.v2.test_token

5. Run tests.

   To run the entire API test set:

   ./refstack-client test -c <Path of the tempest configuration file to use> -vv

   To run only those tests specified in a DefCore defined test file:

   `./refstack-client test -c <Path of the tempest configuration file to use> -vv -- --load-list /path/to/test-list.txt

   Note:

   1. Adding -v option will show the summary output.

   2. Adding -vv option will show the Tempest test result output.

   3. Adding --upload option will have your test results be uploaded to the default Refstack API server or the server specified by --url.

   4. Adding --url option will allow you to change where test results should be uploaded.

   5. Adding -r option with a string will prefix the JSON result file with the given string (e.g. '-r my-test' will yield a result file like 'my-test-0.json').

   6. Adding '--' enables you to pass arbitary arguments to the Tempest runner. After the first '--', all other subsequent arguments will be passed to the Tempest runner as is. This can be used for quick verification of the target test cases. For example:

      -- tempest.api.identity.v2.test_token

      -- --load-list /tmp/test-list.txt

6. Upload test set.

   If you previously ran a test with refstack-client without the --upload option, you can upload your results to a Refstack API server by using the following command:

   ./refstack-client upload <Path of results file>

   The results file is a JSON file generated by refstack-client when a test has completed. This is saved in .tempest/.testrepository. When you use the 'upload' command, you can also override the Refstack API server uploaded to with the --url option.

   Note:

   1. Adding -i <path-to-private-key> option will upload test result with digital signature. For signing refstack-client uses private RSA key. OpenSSH format of rsa keys supported, so you can just use your ssh key '~/.ssh/id-rsa' or generate a new one with 'ssh-keygen -b 4096'. For now, signed test results can be considereded as private.

7. List uploaded test set.

   You can list previously uploaded data from a Refstack API server by using the following command:

   ./refstack-client list --url <URL of the Refstack API server>

Tempest Hacking

By default, refstack-client installs Tempest into the .tempest directory. If you're interested in working with Tempest directly for debugging or configuration, you can activate a working Tempest environment by switching to that directory and using the installed dependencies.

1. cd .tempest
2. run tempest with ./run_tempest.sh -V or source ./.venv/bin/activate and run tests manually with testr.

This will make the entire Tempest environment available for you to run, including the run_tempest script and testr.