diff --git a/specs/queens/approved/subunit-data-api.rst b/specs/queens/approved/subunit-data-api.rst new file mode 100644 index 00000000..b88e395b --- /dev/null +++ b/specs/queens/approved/subunit-data-api.rst @@ -0,0 +1,498 @@ +============================================= +Subunit Data Management API +============================================= + +Launchpad blueprint: + +* https://blueprints.launchpad.net/refstack/+spec/subunit-data-api + +This specification describes an expansion of the refstack api which, when +complete, will allow for the upload and management of subunit data files +to a RefStack server. + +Problem description +=================== + +The current RefStack API allows for the upload, management, and verification +of test results by server administrators. These capabilities, though +sufficient for the current scope of RefStack, will need an API expansion in +order to allow for similar data management of subunit data results. This +expansion will enable those organizations looking to achieve a greater degree +of interoperability to securely share the details of test runs with the +Foundation so as to get assistance with getting their OpenStack instance to +successfully meet interop standards. + + +Proposed change +=============== + +**Add new API functionality to the RefStack v1 API** + + * Upload new subunit data- nonadmin capability + + * Link new subunit data to a corresponding existing test result- + nonadmin capability + + * Delete subunit data- admin/owner capability + + * Show subunit data for a given test result- admin/owner capability + + + +Note that, amongst the additions to the table that stores test results, +there is no added field intended for the storage of a subunit result id. +This is because, as per the spec defining the changes needed to upload and +utilize subunit data, the current plan is to link the two entries via the +metadata table. + +Alternatives +------------ + +* If subunit2sql takes too long to perform the aforementioned operations, + using asynchronous processing and upload may prove to be a better option. + For now though, it appears as though synchronous operations will be possible +* Possibly require subunit data to be converted into json before being passed + in for upload + +Data model impact +------------------ + +This API will interface with subunit2sql, which will add several tables into +the RefStack database. Though these have been laid out already in the general +subunit data import spec, for the sake of thoroughness, here they +are again::: + + -------------------------------------- + | tests | + -------------------------------------- + | id | String(256) | + | test_id | String(256) | + | run_count | Integer | + | failure | Integer | + | run_time | Float | + -------------------------------------- + + ---------------------------------------- + | runs | + ---------------------------------------- + | id | BigInteger | + | skips | Integer | + | fails | Integer | + | passes | Integer | + | run_time | Float | + | artifacts | Text | + | run_at | DateTime | + ---------------------------------------- + + --------------------------------------------------- + | test_runs | + --------------------------------------------------- + | id | BigInteger | + | test_id | BigInteger | + | run_id | BigInteger | + | status | String(256) | + | start_time | DateTime | + | start_time_microseconds | Integer | + | stop_time | DateTime | + | stop_time_microseconds | Integer | + | test | Test | + | run | Run | + --------------------------------------------------- + + ------------------------------------------- + | run_metadata | + ------------------------------------------- + | id | BigInteger | + | key | String(255) | + | value | String(255) | + | run_id | BigInteger | + | run | Run | + ------------------------------------------- + + ------------------------------------------- + | test_run_metadata | + ------------------------------------------- + | id | BigInteger | + | key | String(255) | + | value | String(255) | + | test_run_id | BigInteger | + | test_run | TestRun | + ------------------------------------------- + + ------------------------------------------- + | test_metadata | + ------------------------------------------- + | id | BigInteger | + | key | String(255) | + | value | String(255) | + | test_id | BigInteger | + | test | Test | + ------------------------------------------- + + ------------------------------------------- + | attachments | + ------------------------------------------- + | id | BigInteger | + | test_run_id | BigInteger | + | label | String(255) | + | attachment | LargeBinary | + | test_run | TestRun | + ------------------------------------------- + + +REST API impact +--------------- + +The current plan, as briefly outlined above, is to make the following +additions to the current API: + +**Upload subunit data** + +* Description: + + This capability will be used to upload the subunit data of a test result + that is not already in the database. It will do so in a few steps. First, + it will take the subunit file open it, and convert it to v2 stream format + (refstack-client outputs a subunit v1 file). Then, it will check to make + sure the data is not already stored in the database, and if there is no + record matching the data stored in the passed-in file, the api should then + use subunit2sql to insert the subunit data into the appropriate fields, as + well as inserting using the parsed data to insert a new entry into the + refstack "runs" table using the existing refstack api utilities. This may + seem a bit complicated for an upload function, but the goal in doing this + all in one fell swoop is to ensure that no subunit data is ever uploaded + that is not connected to some test result. Uploading subunit data will not + require admin privileges. + +* Method Type: POST + +* URI: v1/subunit/ + +* Normal Response Codes: + + * Created (201) + +* Error Response Codes: + + * Bad Request (400) + * Not found (404) + +* Request parameters: N/A + +* JSON schema definition for the body data: + + .. parsed-literal:: + { + { + 'subunit_data': + } + } + +* JSON schema definition for the response data: + + .. parsed-literal:: + { + 'subunit-uuid': 'subunit2sql-defined run id', + 'result-id': 'result id' + } + + +**Link subunit data to a corresponding existing test result** + +* Description: + + This will allow for the linking of a new, unadded set of subunit data + to data a test result already existing in the database. It will do + so by converting the contents of the given file to a subunit v2 stream, + then using the stream to generate a corresponding test result, + and then comparing that to the passed in test result. If the + generated result and the stored result correspond to one another, + it should insert the subunit data into the database and link the two + entries via a key value pair in RefStack's meta table. The two keys I + plan to use are the subunit data's uuid and the test result's id. + Because the validity of the link is easily verifiable, this action will + not be one that requires admin privileges. + +* Method Type: PUT + +* URI: v1/subunit + +* Normal Response Codes: + + * OK (200) + +* Error Response Codes: + + * Bad Request (400) + * Unauthorized (401) + * Not Found (404) + +* Request parameters: + + +---------------+-------+--------------+-----------------------------------+ + | Parameter | Style | Type | Description | + +===============+=======+==============+===================================+ + | result_id | URI | csapi:UUID | test result ID to link to | + +---------------+-------+--------------+-----------------------------------+ + +* JSON schema definition for the body data: + + .. parsed-literal:: + { + 'subunit data': + } + +* JSON schema definition for the response data: + + .. parsed-literal:: + { + 'uuid': 'subunit2sql-defined run id', + 'id': 'refstack test result id' + } + +**Delete subunit data entry** + +* Description + + This utility will be used to delete subunit data from the RefStack + database. Foundation and vendor admins, along with entry owners will + be able to delete subunit data entry. + +* Method type: DELETE + +* URI: v1/subunit/{id} + +* Normal Response Codes: + + * No content (204) + +* Error Response Codes: + + * Bad Request (400) + * Unauthorized (401) + * Forbidden (403) + * Not found (404) + +* Request parameters: + + +---------------+-------+--------------+-----------------------------------+ + | Parameter | Style | Type | Description | + +===============+=======+==============+===================================+ + | id | URI | csapi:UUID | ID to be removed. | + +---------------+-------+--------------+-----------------------------------+ + +* JSON schema definition for the body data: N/A + +* JSON schema definition for the response data: N/A + +**Show subunit data** + +* Description + + This utility will be used to list the subunit data that has been + uploaded into the RefStack database. This action will be available + to vendor and Foundation admins only. A specific subunit data entry + can be selected and viewed using the result_id parameter. It will do + so in two steps. First, it will take the given test result id, and + reference refstack's meta table to find the corresponding subunit + uuid. Then, it will use that uuid to GET the subunit data from the + v1/subunit/{uuid} endpoint. + +* Method type: GET + +* URI: v1/subunit/{uuid} + +* Normal Response Codes: + + * OK (200) + +* Error Response Codes: + + * Bad Request (400) + * Unauthorized (401) + * Forbidden (403) + +* Request parameters: + + +---------------+-------+--------------+-----------------------------------+ + | Parameter | Style | Type | Description | + +===============+=======+==============+===================================+ + | id | URI | csapi:UUID | test result id to search for. | + +---------------+-------+--------------+-----------------------------------+ + + +* JSON schema definition for the body data: N/A + +* JSON schema definition for the response data: + + .. parsed-literal:: + { + 'subunit-data:': { + 'run_at': 2017-08-16 18:34:58.367221Z + 'uuid': '4d7950cb-586e-407e-9acf-5b169825af98', + 'skips': 0, + 'fails': 1, + 'passes': 1, + 'run_time': 2060.7 + 'artifacts': 'http://example-logs.log', + } + 'tests': [ + { + 'id': '1 + 'test_id': 'tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_icmp_type_code' + 'run_count': 1 + 'success': 1 + 'failure': 1 + 'run_time': 5.60538 + }, + { + 'test_id': ' tempest.api.compute.keypairs.test_keypairs_negative.KeyPairsNegativeTestJSON.test_create_keypair_with_empty_public_key', + 'run_count': 1, + 'success': 0, + 'failure': 1, + 'run_time': 0.10919, + }, + ] + 'test_runs': [ + { + 'test_id': 1, + 'run_id': 1, + 'status': 'success', + 'start_time': 2017-08-16 07:21:56, + 'stop_time': 2017-08-16 07:22:02, + 'start_time_microsecond': 929341, + 'stop_time_microsecond': 534721, + }, + { + 'test_id': 2, + 'run_id': 2, + 'status': 'fail', + 'start_time': 2017-08-16 07:13:34, + 'stop_time': 2017-08-16 07:13:35, + 'start_time_microsecond': 693353, + 'stop_time_microsecond': 726471, + }, + ] + 'attachments': [ + { + 'test_run_id': 1, + 'label': '' + 'attachment': '' + } + ] + } + + +**Delete Test Result** + +* Description + + This modification to the v1/results/ endpoint's delete function will + ensure that, when a test result is deleted, the corresponding subunit + data is too. This is neccessary largely because, in our data model, + subunit data should always be linked to an associated test result. + +* Method type: DELETE + +* URI: v1/result/{id} + +* Normal Response Codes: + + * No content (204) + +* Error Response Codes: + + * Bad Request (400) + * Unauthorized (401) + * Forbidden (403) + * Not found (404) + +* Request parameters: + + +---------------+-------+--------------+-----------------------------------+ + | Parameter | Style | Type | Description | + +===============+=======+==============+===================================+ + | id | URI | csapi:UUID | ID to be removed. | + +---------------+-------+--------------+-----------------------------------+ + +* JSON schema definition for the body data: N/A + +* JSON schema definition for the response data: N/A + +Security impact +--------------- + +There has been some concern over the sharing of subunit data via the RefStack +API, and though they are largely based on a misinformation, this is part of +why so few of the API additions are nonadmin. For more details about this +discussion, please refer to the generalized spec for the upload and usage of +subunit tests. + +Notifications impact +-------------------- + +None. + +Other end user impact +--------------------- + +None. + +Performance impact +------------------ + +None. + +Other deployer impact +--------------------- + +None. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + Megan Guiney + +Other contributors: + TBD + +Work Items +---------- + +* Discuss, amend, and merge this spec +* Run subunit2sql performance tests +* add field to "test" table +* add subunit api functionity +* add subunit-adjacent test result api functionality + + +Dependencies +============ + +* subunit2sql and its dependencies will need to be installed + during refstack server setup. As a result, puppet-refstack may + need some adjustments. + + +Testing +======= + +* Add unit tests to verify the proper functionality of the new API + additions. + + +Documentation Impact +==================== + +* Add documentation to detail the usage and functionality of the + new API additions. + + +References +========== +[1] https://github.com/openstack/refstack/blob/master/specs/pike/approved + /upload-subunit-tests.rst