Because test attributes can constantly change in test lists, refstack-client
should have a native argument for passing in test lists (instead of '-- --load-list')
Refstack-client will normalize the tests in the given list by matching it to
test IDs in the current Tempest environment.
Closes-Bug: #1475407
Change-Id: I1e3f026f5cd706cf73b6febfce98cb96b742d4d6
To better support passing in arbitrary arguments to the Tempest
runner, we now allow the '--' delimeter as an argument. Anything after
'--' will be passed to the Tempest runner as is. This better matches how
run_tempest.sh does things, and will make including test lists clearer.
Closes-Bug: #1467737
Change-Id: I8e091ef7c126cbeb4c388bb19e8343efb1c4c8ab
Add key which alllows to drop ssl verification.
Without this patch it is not possible to post test results on
https endpoint with selfsigned sertificate.
Change-Id: Idd6b0e2f9ee1057f9ae1d3078f70a2cdcee4e360
Defaults for --url param has been changed to env[REFSTACK_URL] if
it is set and 'http://api.refstack.net' if env[REFSTACK_URL] is not set.
It will be helpull during development and for upload to local refstack
server.
Change-Id: I8de8dfba1f6741133585815ace5cfd2ca2780fa2
User gets url to test results page after successfull upload.
Added 'list' command. It allows user to retrieve links on last uploaded
test results.
Change-Id: Id3a33179ad551cf95640d4c9fde147048592a1a4
Added feature to upload test results in to Refstack API signed with
your personal RSA key. Just do upload with key '-i PATH_TO_PRIVATE_KEY'.
You can use your general OpenSSH key ~/.ssh/id-rsa or
generate a new one with 'ssh-keygen -b 4096'.
Change-Id: Icce6d7146f0fa9de892d1f4785ef24f17fc9b286
A user can use the -r/--result-file-tag argument to change the name of the output
json file. The name will always include the subunit stream ID, for example if a user
input '--result-file-tag my-test' and the next-stream id was "2", the output filename
would be my-test-2.json.
Change-Id: Ie7302470fe1b63a21f61451ef62687634b6e0932
The --offline argument was removed and an --upload argument was
added. Adding this argument will have your test results be uploaded to
the default Refstack API server or the server specified by --url.
Story: https://storyboard.openstack.org/#!/story/2000136
Change-Id: I93302e1cacdd9d0f996ea5545fbe21e1fb6aaa23
Previously, after the run_tempest script, was run, all post-processing
was done only if the returncode was 0. If it was non-zero, an error message
was given. This was an oversight, as the run_tempest script will return a code
of 1 when any test case fails. This leaves the unintended side effect of refstack
not parsing the results or producing the output json when Tempest successfully
runs, but a test case fails.
Checking for the existence of the expected subunit file will at least validate
that the Tempest test was at least started successfully, and didn't fail to run
due to some misconfiguration in the environment.
Change-Id: I541b798db41b713525efeb8dedfb9347be780a3f
setup_env script should be able to install Tempest from specific tag
release. Key -t added to setup_env script. It specifies Tempest tag
release which will be installed as a test runner. By default last tag
from Tempest github repository will be used. Installed by setup script,
Tempest test runner is only single available option for now. Because of
it, option --tempest-dir option is removed from refstack-client
https://storyboard.openstack.org/#!/story/309https://storyboard.openstack.org/#!/story/324
Change-Id: I182c3870a7ce4bbcb188836d189538dfd54b1c26
Split refstack-client into two subcommands: upload and test.
The 'test' command will run Tempest like before, while 'upload'
takes in a result file argument and will upload it to the Refstack
API url. Testing with refstack-client now saves a JSON result file
containing the content of what will or would be posted to the
Refstack API.
Change-Id: I28bcb75b51b77872f39e144ae8ffa7e64b26b233
Ubuntu setup script removed.
New setup script was tested on latest vesions of ubuntu, debian, centos
and fedora instalations.
Change-Id: I94cff68de15d44f420e56c7793ed18476e07b442
Normally, catching a subprocess.CalledProcessError would suffice
since that exception is thrown when the subprocess exits with a
non-zero code. However this exception is only thrown when using
specific methods like check_output which we no longer use.
This commit now has us explicitly checking the process return code.
Change-Id: I799f92836fba41e718570e2f7affb27f11c26156
Refactoring was done to make more of the methods easier to test.
Existing unit tests were adjusted/simplified and more tests were
added as a result.
Change-Id: I57c334c6a11fa8e12c88bdd0788d63a35cdfd7bc
Added an --offline argument which will prevent Tempest results
from being uploaded to the Refstack server if used.
Change-Id: I40df179288354a6eb7fac1d0ed634d722a780817
Story: https://storyboard.openstack.org/#!/story/190
