Update Readme

* remove old links
* mention new options of setup_env script
* mention OpenStack Marketing Programs and interop repo
* mention ansible-role-refstack-client

Change-Id: Ia62b400d38eeeebd22e79b33616a04877b9ea33b

README.rst

@@ -2,18 +2,10 @@
 RefStack Client
 ===============
 
-RefStack-client team and repository tags
-########################################
-
-
-.. image:: https://governance.openstack.org/tc/badges/refstack-client.svg
-    :target: https://governance.openstack.org/tc/reference/tags/index.html
-
-
 Overview
 ########
 
-refstack-client is a command line utility that allows you to execute Tempest
+``refstack-client`` is a command line utility that allows you to execute Tempest
 test runs based on configurations you specify.  When finished running Tempest
 it can send the passed test data to a RefStack API server.
 
@@ -24,7 +16,7 @@ We've created an "easy button" for Ubuntu, Centos, RHEL and openSUSE.
 
 1. Make sure you have ``git`` installed
 2. Get the refstack client: ``git clone https://opendev.org/osf/refstack-client.git``
-3. Go into the refstack-client directory: ``cd refstack-client``
+3. Go into the ``refstack-client`` directory: ``cd refstack-client``
 4. Run the "easy button" setup: ``./setup_env``
 
    **Options:**
@@ -34,24 +26,34 @@ We've created an "easy button" for Ubuntu, Centos, RHEL and openSUSE.
 
    b. -t option allows to specify tag in Tempest repository which will be installed.
    For example: execute ``./setup_env -t tags/3`` to install Tempest tag-3.
-
+   By default, Tempest will be installed from commit
-   c. By default, Tempest will be installed from commit
    e64c78dcf720202a0542bb1e1184f5229a11524f (Oct, 2019).
 
+   c. -p option allows to specify python version - python 2.7 (-p 2), 3.6 (-p 3)
+   or any specific one by -p X.X.X. Default to python 3.6.
+
+   d. -q option makes ``refstack-client`` run quitely - if ``.tempest``
+   directory exists ``refstack-client`` is considered as installed.
+
+   e. -s option makes ``refstack-client`` use ``python-tempestconf`` from the
+   given source (path) - used when running f.e. in Zuul.
+
 Usage
 #####
 
-1. Prepare a tempest configuration file that is customized to your cloud
+1. Prepare a tempest configuration file (or let ``refstack-client`` generate it
-   environment. Samples of minimal Tempest configurations are provided in
+   for you, see step #4) that is customized to your cloud environment.
-   the ``etc`` directory in ``tempest.conf.sample`` and ``accounts.yaml.sample``.
+   Samples of minimal Tempest configurations are provided in the ``etc``
+   directory in ``tempest.conf.sample`` and ``accounts.yaml.sample``.
    Note that these samples will likely need changes or additional information
    to work with your cloud.
 
-   Note: Use Tempest Pre-Provisioned credentials_ to provide user test accounts. ::
+   **Note**: Use Tempest Pre-Provisioned credentials_ to provide user test
+   accounts.
 
 .. _credentials: https://docs.openstack.org/tempest/latest/configuration.html#pre-provisioned-credentials
 
-2. Go into the refstack-client directory::
+2. Go into the ``refstack-client`` directory::
 
        cd ~/refstack-client
 
@@ -59,7 +61,7 @@ Usage
 
        source .venv/bin/activate
 
-4. Generate tempest.conf using refstack-client::
+4. (optional) Generate tempest.conf using ``refstack-client``::
 
        refstack-client config --use-test-accounts <path to account file>
 
@@ -74,7 +76,9 @@ Usage
 
 5. Validate your setup by running a short test::
 
-       refstack-client test -c <Path of the tempest configuration file to use> -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
+       refstack-client test \
+         -c <Path of the tempest configuration file to use> -v -- \
+         --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
 
 6. Run tests.
 
@@ -88,10 +92,20 @@ Usage
 
    For example::
 
-       refstack-client test -c ~/tempest.conf -v --test-list "https://refstack.openstack.org/api/v1/guidelines/2018.02/tests?target=platform&type=required&alias=true&flag=false"
+       refstack-client test \
+         -c ~/tempest.conf -v \
+         --test-list "https://refstack.openstack.org/api/v1/guidelines/2020.11/tests?target=platform&type=required&alias=true&flag=false"
 
-   This will run only the test cases required by the 2018.02 guidelines
+   This will run only the test cases required by the 2020.11 guidelines under
-   that have not been flagged.
+   Platform OpenStack Marketing Program that have not been flagged. More about
+   the marketing programs at `Interop and OpenStack Marketing Programs`_.
+
+   For example tests under the compute program are available:
+   https://refstack.openstack.org/api/v1/guidelines/2020.11/tests?target=compute&type=required&alias=true&flag=false
+   Tests of add-on programs can be found similarly, f.e. tests under dns program:
+   https://refstack.openstack.org/api/v1/guidelines/dns.2020.11/tests?target=dns&type=required&alias=true&flag=false
+   or tests under orchestration program:
+   https://refstack.openstack.org/api/v1/guidelines/orchestration.2020.11/tests?target=orchestration&type=required&alias=true&flag=false
 
    **Note:**
 
@@ -118,7 +132,7 @@ Usage
 
 6. Upload your results.
 
-   If you previously ran a test with refstack-client without the ``--upload``
+   If you previously ran a test with ``refstack-client`` without the ``--upload``
    option, you can later upload your results to a RefStack API server
    with your digital signature. By default, the results are private and you can
    decide to share or delete the results later.
@@ -127,7 +141,7 @@ Usage
 
        refstack-client upload <Path of results file> -i <path-to-private-key>
 
-   The results file is a JSON file generated by refstack-client when a test has
+   The results file is a JSON file generated by ``refstack-client`` when a test has
    completed. This is saved in .tempest/.stestr. When you use the
    ``upload`` command, you can also override the RefStack API server uploaded to
    with the ``--url`` option.
@@ -143,31 +157,23 @@ Usage
    Intructions for uploading data with signature can be found at
    https://opendev.org/osf/refstack/src/branch/master/doc/source/uploading_private_results.rst
 
-7. Create a JSON web token to use for authentication to your privately
+7. View uploaded test set.
-   uploaded data
-
-   In order to authenticate to the refstack-server to which you have uploaded
-   your data, you will need to generate a JSON webtoken. To generate a valid
-   token, use the command::
-
-       jwt --key="$( cat %path to private key% )" --alg=RS256 user_openid=%openstackid% exp=+100500
-
-   To test authentication in the API, use the command::
-
-       curl -k --header "Authorization: Bearer %token%" https://localhost.org/v1/profile
-
-8. List uploaded test set.
 
    You can list previously uploaded data from a RefStack API server by using
    the following command::
 
-       refstack-client list --url <URL of the RefStack API server>
+       refstack-client list --url <URL of the RefStack API server> -i <path to private key>
+
+   Alternatively, if you uploaded the results to the official RefStack_ server
+   you can view them by using RefStack_ page where all uploaded results
+   associated with the particular account (the account private key used to
+   upload the results belongs to) will be shown and may be further managed.
 
 
 Tempest hacking
 ###############
 
-By default, refstack-client installs Tempest into the ``.tempest`` directory.
+By default, ``refstack-client`` installs Tempest into the ``.tempest`` directory.
 If you're interested in working with Tempest directly for debugging or
 configuration, you can activate a working Tempest environment by
 switching to that directory and using the installed dependencies.
@@ -177,4 +183,32 @@ switching to that directory and using the installed dependencies.
    and run tests manually with ``tempest run``.
 
 This will make the entire Tempest environment available for you to run,
-including ``tempest run``.
+including ``tempest run``. More about Tempest can be found at its documentation_.
+
+.. _documentation: https://docs.openstack.org/tempest/latest/
+
+
+Interop and OpenStack Marketing Programs
+########################################
+
+The tests ``refstack-client`` runs are defined within interop_ repository
+and divided into several OpenStack Marketing Programs, the list of the programs
+can be found at RefStack_ page.
+
+.. _interop: https://opendev.org/osf/interop
+.. _RefStack: https://refstack.openstack.org/#/
+
+
+ansible-role-refstack-client
+############################
+
+We have created an ansible role called ansible-role-refstack-client_ in order
+to simplify and automate running of ``refstack-client``. The role can be easily
+integrated to an automation machinery - f.e. we use the role for running
+``refstack-client`` on a devstack_ environment in Zuul where we run tests of
+every OpenStack Marketing Program of the current guideline. The latest builds
+can be found here__.
+
+.. _ansible-role-refstack-client: https://opendev.org/x/ansible-role-refstack-client
+.. _devstack: https://opendev.org/openstack/devstack/
+.. __builds: https://zuul.openstack.org/builds?project=x%2Fansible-role-refstack-client