QA community default test runner is ostestr, and using run_tempest.sh
script is no longer recommended. BP replaces run_tempest.sh script
with direct ostestr call passing the appropriate parameters.
Implements blueprint: implement-ostestr-refstackclient
Change-Id: If4ffed70c8320abb4b4c7bfd1b84ca7892272665
This patch simply changes 'username' to 'tenant_name' under
the _generate_keystone_data function when the value of for
'name' under 'project' is being called.
Closes-Bug: #1552025
Change-Id: I3fac7a3014368334f69f7ff1732cc3587f9db99d
Credentials should be specified in the accounts file
as described in the Tempest configuration guide
(http://docs.openstack.org/developer/tempest/configuration.html).
Specifying credentials in the identity section of the tempest
config file is deprecated.
If the user provides credentials in the identity section of tempest
config file and does not provide accounts file, show a warning
that this is deprecated and will be not supported in the future.
If the user does not provide credentials at all, show an error and
exit.
Show information to the user how to configure the credentials.
Update the refstack-client unit tests that still use the identity
section to use accounts file instead.
Change-Id: I6eab3b587292a35404c93da35914ad1f53ef49c7
Closes-Bug: #1552377
Use case: I'm running refstack-client in a tox job, which always appears
to succeed, even when the test fails.
Currently, unless it throws a backtrace on it's own, the refstack-client
script always returns successfully, even when the tempest job it's
running fails.
This change simply passes non-zero return codes back through as the
refstack-client's return code.
Only the test() method was affected here, because it's the only
externally callable function that uses subprocess. Other functions can
now return non-zero return codes as well.
Change-Id: I0ce786babcbee47da1c038a3478bcd4de0dff16e
RefStack should use Tempest approach for retrieving Keystone service
Id. For RefStack purpose, raw API call to keystone should be enough
to get necessary information.
Closes-Bug: #1528357
Change-Id: I555308aabb59b638e22249ead882ca5b09043b7b
This patch allows subunit files to be uploaded to a RefStack
server using the refstack-client. This adds the new command
'upload-subunit'.
Change-Id: Ie8079c66a732da34d08c46664e441723cf904c3a
If the client can't get the identity service ID from the cloud,
the client will use a hash of the endpoint's hostname as the CPID.
Change-Id: I0d61ccb05caf40131a2590bd89b6fce7c62cedd2
Implements-Spec: https://review.openstack.org/#/c/255607/
With the test-list option, refstack-client will first normalize the
user input test list to the test list from the currently installed
Tempest. There seems to be some non RefStack updates in the recent
releases that "testr list-tests" would now require a valid
tempest.conf for test listing. The path of the tempest.conf file
is determined by the .tempest/tempest/config.py module. If a
tempest.conf file is not found, then the failsafe file is used,
namely "/etc/tempest/tempest.conf" which does not exist in
refstack-client.
This patch initialzes the tempest conf related environment variables
with the user passes in information before starting the test run.
Closes-Bug: #1516458
Change-Id: Ic2cdbdcdf575e23cb0fc59c5478ceda5807c06fb
Use a try/except block instead of checking if keys exists
for cleaner code. Now if the identity key is not found in the
catalog, the user will get an error instead of the function
returning None as the cpid.
Closes-Bug: #1502949
Closes-Bug: #1515880
Co-Authored-By: Paul Van Eck <pvaneck@us.ibm.com>
Change-Id: I63942fbf28eff5da58de754ef870506fb246d2d6
refstack-client currently uses the user information
provided in the identity section of the Tempest configuration
when determining the cpid. The account information from the
identity section has now been made optional and a secondary
option of using a yaml file with test account information
was added.
refstack-client currently is not aware of this accounts file
and will fail to run if the user is using only the accounts file
and has not filled in the user information in the identity
section. This change checks to see if the user has specified
an accounts file to use and reads the user data from there
when applicable.
Change-Id: Id5a5a1800febc23a1006b29e58fec05a3aa40af1
Closes-Bug: 1501903
Corrects a bug that caused retrieval of the keystone ID to be used as
the Cloud Provider ID to fail when using keystone v2.
Change-Id: Id378e8ff31472fd2eb194fd59ecd8042297229d8
Closes-Bug: #1497067
This patch changes the way refstack-client gets keystone service id.
Now non-admin creds should be enought for that.
This patch adds support for Keystone API V3.
Keystone API version selected from tempest.conf
Section: 'identity-feature-enabled', key: 'api_v3'
Closes-Bug: #1495671
Change-Id: I1f9edffba88cf99c39b2c8b99f792088d35dbd2e
Sometimes a keystone endpoint might have ssl issues, and you might
get an error like "certificate verify failed". This patch extends the
'-k/--insecure' arg to allow skipping ssl checks when connecting to
keystone.
Change-Id: Icd09aa9906cabc1f4f5ab635fb30fbcebbd37ced
The refstack.net site was updated to use a single domain in order for
the new patches to work. Thus, the default api endpoint should reflect
the new api endpoint which is refstack.net/api.
Change-Id: Icc7b501a65792a79635c4a7263eb9a3e0767d33e
Uploading public key in to Refstack requires pub key self-signature
This signature can be generated with refstac-client using "sign"
command. For example:
./refstack-client sign ~/.ssh/id_rsa
Will print public key ~/.ssh/id_rsa.pub and self signature for it.
Change-Id: I0ee3e89c5e8bcae85eae63500a5fd98cea12c273
Because test attributes can constantly change in test lists, refstack-client
should have a native argument for passing in test lists (instead of '-- --load-list')
Refstack-client will normalize the tests in the given list by matching it to
test IDs in the current Tempest environment.
Closes-Bug: #1475407
Change-Id: I1e3f026f5cd706cf73b6febfce98cb96b742d4d6
To better support passing in arbitrary arguments to the Tempest
runner, we now allow the '--' delimeter as an argument. Anything after
'--' will be passed to the Tempest runner as is. This better matches how
run_tempest.sh does things, and will make including test lists clearer.
Closes-Bug: #1467737
Change-Id: I8e091ef7c126cbeb4c388bb19e8343efb1c4c8ab
Add key which alllows to drop ssl verification.
Without this patch it is not possible to post test results on
https endpoint with selfsigned sertificate.
Change-Id: Idd6b0e2f9ee1057f9ae1d3078f70a2cdcee4e360
Defaults for --url param has been changed to env[REFSTACK_URL] if
it is set and 'http://api.refstack.net' if env[REFSTACK_URL] is not set.
It will be helpull during development and for upload to local refstack
server.
Change-Id: I8de8dfba1f6741133585815ace5cfd2ca2780fa2
User gets url to test results page after successfull upload.
Added 'list' command. It allows user to retrieve links on last uploaded
test results.
Change-Id: Id3a33179ad551cf95640d4c9fde147048592a1a4
Added feature to upload test results in to Refstack API signed with
your personal RSA key. Just do upload with key '-i PATH_TO_PRIVATE_KEY'.
You can use your general OpenSSH key ~/.ssh/id-rsa or
generate a new one with 'ssh-keygen -b 4096'.
Change-Id: Icce6d7146f0fa9de892d1f4785ef24f17fc9b286
A user can use the -r/--result-file-tag argument to change the name of the output
json file. The name will always include the subunit stream ID, for example if a user
input '--result-file-tag my-test' and the next-stream id was "2", the output filename
would be my-test-2.json.
Change-Id: Ie7302470fe1b63a21f61451ef62687634b6e0932
The --offline argument was removed and an --upload argument was
added. Adding this argument will have your test results be uploaded to
the default Refstack API server or the server specified by --url.
Story: https://storyboard.openstack.org/#!/story/2000136
Change-Id: I93302e1cacdd9d0f996ea5545fbe21e1fb6aaa23
Previously, after the run_tempest script, was run, all post-processing
was done only if the returncode was 0. If it was non-zero, an error message
was given. This was an oversight, as the run_tempest script will return a code
of 1 when any test case fails. This leaves the unintended side effect of refstack
not parsing the results or producing the output json when Tempest successfully
runs, but a test case fails.
Checking for the existence of the expected subunit file will at least validate
that the Tempest test was at least started successfully, and didn't fail to run
due to some misconfiguration in the environment.
Change-Id: I541b798db41b713525efeb8dedfb9347be780a3f
setup_env script should be able to install Tempest from specific tag
release. Key -t added to setup_env script. It specifies Tempest tag
release which will be installed as a test runner. By default last tag
from Tempest github repository will be used. Installed by setup script,
Tempest test runner is only single available option for now. Because of
it, option --tempest-dir option is removed from refstack-client
https://storyboard.openstack.org/#!/story/309https://storyboard.openstack.org/#!/story/324
Change-Id: I182c3870a7ce4bbcb188836d189538dfd54b1c26
Split refstack-client into two subcommands: upload and test.
The 'test' command will run Tempest like before, while 'upload'
takes in a result file argument and will upload it to the Refstack
API url. Testing with refstack-client now saves a JSON result file
containing the content of what will or would be posted to the
Refstack API.
Change-Id: I28bcb75b51b77872f39e144ae8ffa7e64b26b233
Ubuntu setup script removed.
New setup script was tested on latest vesions of ubuntu, debian, centos
and fedora instalations.
Change-Id: I94cff68de15d44f420e56c7793ed18476e07b442
Normally, catching a subprocess.CalledProcessError would suffice
since that exception is thrown when the subprocess exits with a
non-zero code. However this exception is only thrown when using
specific methods like check_output which we no longer use.
This commit now has us explicitly checking the process return code.
Change-Id: I799f92836fba41e718570e2f7affb27f11c26156
Refactoring was done to make more of the methods easier to test.
Existing unit tests were adjusted/simplified and more tests were
added as a result.
Change-Id: I57c334c6a11fa8e12c88bdd0788d63a35cdfd7bc
Added an --offline argument which will prevent Tempest results
from being uploaded to the Refstack server if used.
Change-Id: I40df179288354a6eb7fac1d0ed634d722a780817
Story: https://storyboard.openstack.org/#!/story/190
