Merge "Specification to associate test results to vendor products."

specs/newton/approved/associate-test-result-to-product.rst

@@ -0,0 +1,353 @@
+=============================================
+Test Results to Vendor Products Association
+=============================================
+
+Launchpad blueprint:
+
+* https://blueprints.launchpad.net/refstack/+spec/result-listing-page
+* https://blueprints.launchpad.net/refstack/+spec/marketplace-product-model
+
+This specification defines the processes and mechanisms to associate test
+results to vendor products.
+
+Problem description
+===================
+
+So far, community test results are shared to the public anonymously, with no
+linkage to users, vendors or products.  DefCore, OpenStack Foundation (OSF)
+and community user feedback revealed strong interest in enabling RefStack to
+show the results with linkage to vendors, products and test statuses. This is
+especially important for those test results that were used for official
+OpenStack Powered Logo applications.
+
+With the recent implementation of vendor and product registration process, it
+is now possible to associate a particular test result record to a vendor's
+product of a particular version.
+
+**Note**
+
+Test results are not associated directly to vendors. Test results are
+associated to products which belong to vendors.  A product may have many
+versions.
+
+RefStack interprets OSF/DefCore requirements as following:
+
+* Vocabulary:
+
+  * product version: a product version is a version of a product.
+  * product vendor: a product vendor is a vendor who owns the product.
+
+* Model:
+
+  * A product version can be used to deploy many clouds.
+  * A product version can have many tests.
+  * Test results will be associated to product versions (not products).
+
+* Use cases:
+
+  * Only the user who uploads the result and is also an admin of the product
+    vendor or an interop admin can associate a test result to a product
+    version.
+  * Once a test is associated to a product version, only the admins of the
+    product vendor or interop admins can perform CRUD operations on the test
+    result.
+
+
+Proposed change
+===============
+
+* Add a new columns to the "test" table in the database:
+
+  * **product_version_id**:  this field stores the product version ID that the
+    test run is associated to.
+
+* Associate a test result to a product
+
+  * The association must be initiated by a user who creates the test result
+    record and is an admin of the vendor which owns the product.  The
+    association is done by updating the product_version_id field of the test
+    result with the ID of the product version from the "version" table.
+
+  * Once a test result is associated to a product version, the test result can
+    not be deleted until it is disassociate from a product.
+
+  * Once a test result is associated to a product version, only interop
+    (i.e. RefStack site) admins or vendor admins can manage the test result
+    (e.g. making decision of sharing the test result record).
+
+  * A product version can't be deleted if there are tests associated to it.
+
+  **Note**
+
+  Currently, RefStack has not implemented "user role" to differentiate the
+  roles of users in a "user group".  As such, at this time, all users in a
+  "user group" are admin users.
+
+Alternatives
+------------
+
+An alternative method to associate a test record to a product is by matching
+values of the "cpid" field (in the "version" table) and the "cpid" field (in
+the "test" table).  The major concern and shortcomings of this option are as
+follows:
+
+* The "cpid" field is a user input parameter, therefore it is not guarantee to
+  be unique.
+
+* The cpid fields may be identical for test results run by different users
+  against the same public cloud instance.
+
+* A product may have many cloud instances which are identified by different
+  cpids.
+
+
+Data model impact
+-----------------
+
+Add a new column named "product_version_id" to the existing "test" table.
+This field can be null.
+
++------------------------+-------------+----------+
+| Column                 |   Type      |          |
++========================+=============+==========+
+| product_version_id     | varchar(36) |   FK     |
++------------------------+-------------+----------+
+
+Note: The user input product_version_id must exist in the "version" table.
+
+REST API impact
+---------------
+
+The following REST APIs will be modified.
+
+**Update result**
+
+* Description:
+
+  This API will be used to make update to a test entry of the "test" table.
+  To begin with, only the owner user who uploaded the test result, can make
+  update to the product_version_id filed.  Once a test is associated to a
+  product version, only interop or vendor admins can make updates to a test
+  result.
+
+* Method type: PUT
+
+* URI: v1/results/{result_id}/
+
+* Normal Response Codes:
+
+  * OK (200)
+
+* Error Response Codes:
+
+  * Bad Request (400)
+  * Unauthorized (401)
+  * Forbidden (403)
+  * Not found (404)
+
+* Request parameters:
+
+  +---------------+-------+--------------+-----------------------------------+
+  | Parameter     | Style | Type         | Description                       |
+  +===============+=======+==============+===================================+
+  | result_id     | URI   | csapi:UUID   | Test result ID for marking.       |
+  +---------------+-------+--------------+-----------------------------------+
+
+
+* JSON schema definition for the body data:
+
+  .. parsed-literal::
+    {
+       {
+          "verification_status" : 1,
+          "product_version_id": "85346866-307f-4052-ba31-ff6270635e14",
+          "required": []
+       }
+    }
+
+  **Note**
+
+  * Although the verification_status column is listed here for completeness of
+    the API body schema, this field can only be updated by interop admin as
+    described in spec https://review.openstack.org/#/c/343954/ .
+  * Update request including the "verification_status" field will return
+    "Forbidden(403)" if the requester is not an interop admin.
+
+* JSON schema definition for the response data: N/A
+
+**List results**
+
+* Description: (No update)
+
+* Method type: GET (No update)
+
+* URI: v1/results/ (No update)
+
+* Normal Response Codes: (No update)
+
+* Error Response Codes: (No update)
+
+* Request parameters: (No update)
+
+  Add the following parameter to the existing ones:
+
+  +---------------------+-------+-------------+---------------------------------+
+  | Parameter           | Style | Type        | Description                     |
+  +=====================+=======+=============+=================================+
+  | product_version_id  | query | xsd:string  | Only return the test records    |
+  | (optional)          |       |             | belonging to this               |
+  |                     |       |             | product_version_id.             |
+  +---------------------+-------+-------------+---------------------------------+
+
+* JSON schema definition for the body data: N/A
+
+* JSON schema definition for the response data:
+
+  Update to add product_version_id to the response body.
+
+  .. parsed-literal::
+    {
+       pagination: {
+          current_page: 6,
+          total_pages: 37
+       },
+       results: [
+          {
+             url: "https://refstack.openstack.org/#/results/7943e04a-2b95-453c-b627-8a24b2c6faa0",
+             created_at: "2016-07-25 02:24:34",
+             meta: { },
+             id: "7943e04a-2b95-453c-b627-8a24b2c6faa0",
+             duration_seconds: 0,
+             verification_status : 0,
+             product_version_id: ""
+          },
+          {
+             url: "https://refstack.openstack.org/#/results/91ae10c5-ecf5-4823-81d4-09836dc212cf",
+             created_at: "2016-07-13 18:37:53",
+             meta: {
+                shared: ""true"",
+                target: "compute",
+                guideline: "2016.01.json"
+             },
+             id: "91ae10c5-ecf5-4823-81d4-09836dc212cf",
+             duration_seconds: 6037,
+             verification_status : 1,
+             product_version_id: "68668534-307f-4052-ba31-ff6270635e14"
+          },
+          ........
+       ]
+    }
+
+**Show result details**
+
+* Description: (No update)
+
+* Method type: GET (No update)
+
+* URI: v1/results/{result_id} (No update)
+
+* Normal Response Codes: (No update)
+
+* Error Response Codes: (No update)
+
+* Request parameters: (No update)
+
+* JSON schema definition for the body data: N/A
+
+* JSON schema definition for the response data:
+
+  Update to add product_version_id to the response body.
+
+  .. parsed-literal::
+     {
+       user_role: "user",
+       created_at: "2016-07-13 18:37:53",
+       meta: {
+          shared: ""true"",
+          target: "compute",
+          guideline: "2016.01.json"
+       },
+       id: "91ae10c5-ecf5-4823-81d4-09836dc212cf",
+       duration_seconds: 6037,
+       verification_status : 1,
+       product_version_id; "68668534-307f-4052-ba31-ff6270635e14",
+       results: [
+          "tempest.api.compute.certificates.test_certificates.CertificatesV2TestJSON.test_create_root_certificate",
+          "tempest.api.compute.certificates.test_certificates.CertificatesV2TestJSON.test_get_root_certificate",
+          ......
+       ]
+    }
+
+
+Security impact
+---------------
+
+None.
+
+Notifications impact
+--------------------
+
+None.
+
+Other end user impact
+---------------------
+
+None
+
+Performance Impact
+------------------
+
+None
+
+Other deployer impact
+---------------------
+
+None
+
+Developer impact
+----------------
+
+None
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+
+Primary assignee:
+  Paul Van Eck
+  Andrey Pavlov
+
+Other contributors:
+  TBD
+
+Work Items
+----------
+
+* Add the defined additional field to the "test" table.
+* Develop business and UI code to enable association of a test result to a
+  product.
+
+
+Dependencies
+============
+
+None
+
+Testing
+=======
+
+* Add unit tests to test the newly added code.
+
+
+Documentation Impact
+====================
+
+None
+
+
+References
+==========
+
+None