Previously, after the run_tempest script, was run, all post-processing
was done only if the returncode was 0. If it was non-zero, an error message
was given. This was an oversight, as the run_tempest script will return a code
of 1 when any test case fails. This leaves the unintended side effect of refstack
not parsing the results or producing the output json when Tempest successfully
runs, but a test case fails.
Checking for the existence of the expected subunit file will at least validate
that the Tempest test was at least started successfully, and didn't fail to run
due to some misconfiguration in the environment.
Change-Id: I541b798db41b713525efeb8dedfb9347be780a3f
setup_env script should be able to install Tempest from specific tag
release. Key -t added to setup_env script. It specifies Tempest tag
release which will be installed as a test runner. By default last tag
from Tempest github repository will be used. Installed by setup script,
Tempest test runner is only single available option for now. Because of
it, option --tempest-dir option is removed from refstack-client
https://storyboard.openstack.org/#!/story/309https://storyboard.openstack.org/#!/story/324
Change-Id: I182c3870a7ce4bbcb188836d189538dfd54b1c26
Split refstack-client into two subcommands: upload and test.
The 'test' command will run Tempest like before, while 'upload'
takes in a result file argument and will upload it to the Refstack
API url. Testing with refstack-client now saves a JSON result file
containing the content of what will or would be posted to the
Refstack API.
Change-Id: I28bcb75b51b77872f39e144ae8ffa7e64b26b233
Ubuntu setup script removed.
New setup script was tested on latest vesions of ubuntu, debian, centos
and fedora instalations.
Change-Id: I94cff68de15d44f420e56c7793ed18476e07b442
Normally, catching a subprocess.CalledProcessError would suffice
since that exception is thrown when the subprocess exits with a
non-zero code. However this exception is only thrown when using
specific methods like check_output which we no longer use.
This commit now has us explicitly checking the process return code.
Change-Id: I799f92836fba41e718570e2f7affb27f11c26156
Refactoring was done to make more of the methods easier to test.
Existing unit tests were adjusted/simplified and more tests were
added as a result.
Change-Id: I57c334c6a11fa8e12c88bdd0788d63a35cdfd7bc
Added an --offline argument which will prevent Tempest results
from being uploaded to the Refstack server if used.
Change-Id: I40df179288354a6eb7fac1d0ed634d722a780817
Story: https://storyboard.openstack.org/#!/story/190
