Browse Source

Update documentation, add playbook for developers

The patch updates the user-faced documentation and adds a playbook
together with a doc meant mainly for RefStack developers which
helps with running the server instance locally.

Change-Id: I268fdf717c8f0e2cd0c130c560306bf54e48f348
changes/56/798056/1
Martin Kopec 3 months ago
parent
commit
5790634ff4
  1. 1
      doc/source/index.rst
  2. 106
      doc/source/run_in_docker.rst
  3. 72
      doc/source/uploading_private_results.rst
  4. 2
      doc/source/vendor_product.rst
  5. 12
      playbooks/README.rst
  6. 52
      playbooks/run-refstack-in-container.yml

1
doc/source/index.rst

@ -42,4 +42,3 @@ Indices and tables
------------------
* :ref:`search`

106
doc/source/run_in_docker.rst

@ -1,6 +1,111 @@
Run-in-docker manual
====================
Option 1 - Using an ansible playbook
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
These steps are meant for RefStack developers to help them with setting up
a local refstack instance.
In production RefStack server is managed by a set of playbooks and ansible roles
defined in `system-config <https://opendev.org/opendev/system-config.git>`__
repository. This option takes advantage of those.
The RefStack server is running on Ubuntu 20.04 LTS in the production.
You can find an ansible playbook in ``playbooks`` directory which semi-automates
the process of running refstack server in a container.
Execute the playbook by::
$ ansible-playbook playbooks/run-refstack-in-container.yaml
In order to avoid setting certificates and https protocol (it's simpler and more
than enough for a testing instance), edit
``/etc/apache2/sites-enabled/000-default.conf`` like following:
* remove VirtualHost section for the port 80 and change the port of VirtualHost from 443 to 80
* Turn off the SSLEngine (`SSLEngine on -> SSLEngine off`)
* Remove SSLCertificate lines
and then restart the apache service so that it loads the new configuration::
$ systemctl restart apache2
How to edit refstack files within the container
-----------------------------------------------
List the running container by::
$ docker container list
You can enter the container by::
$ sudo docker exec -it <container name> /bin/bash
If you wanna install new packages like f.e. vim, do the following::
$ apt update
$ apt install vim
Edit what's needed, backend is installed under
``/usr/local/lib/python3.7/site-packages/refstack/`` and frontend source files
can be found at ``/refstack-ui``
After you made the changes, make pecan to reload the files served::
$ apt install procps # to install pkill command
$ pkill pecan
Killing pecan will kick you out of the container, however, pecan serves the
edited files now and you may re-enter the container.
Installing refstack with changes put for a review
-------------------------------------------------
In order to do this, you will need to rebuild the refstack image built by the
playbook.
Go to the location where the playbook downloaded system-config, default in
``/tmp/refstack-docker`` and edit the refstack's Dockerfile::
$ cd /tmp/refstack-docker
$ vim ./refstack-docker-files/Dockerfile
Replace::
$ RUN git clone https://opendev.org/osf/refstack /tmp/src
by::
$ RUN git clone https://opendev.org/osf/refstack.git /tmp/src \
&& cd /tmp/src && git fetch "https://review.opendev.org/osf/refstack" \
refs/changes/37/<change id/<patchset number> && git checkout -b \
change-<change id>-<patchset number> FETCH_HEAD
Then rebuild the image::
$ docker image build -f Dockerfile -t <name:tag> .
Edit the ``docker-compose.yaml`` stored (by default) in
``/etc/refstack-docker/docker-compose.yaml`` and change the the image
(under `refstack-api`) to your image name and tag you set in the previous step.
After then spin a new container using the new image::
$ cd /etc/refstack-docker
$ docker-compose down # if refstack container is already running
$ docker-compose up -d
To see the server's logs use the following command::
$ docker container logs -f <container name>
Option 2 - Using a script
^^^^^^^^^^^^^^^^^^^^^^^^^
NOTE: This is currently outdated, follow the Option 1 for now.
The main purpose of the ``run-in-docker`` script is to provide a
convenient way to create a local setup of RefStack inside a Docker
container. It should be helpful for new developers and also for testing
@ -70,4 +175,3 @@ Useful in-container commands/aliases:
files on the host
- ``activate`` - activate the python virtual env
- ``mysql`` - open the MySQL console

72
doc/source/uploading_private_results.rst

@ -5,7 +5,7 @@ RefStack allows test results contributors to submit test results and
have them displayed either anonymously, or identified with a vendor. As
such, test results should be uploaded with validated users. Users will
first log into RefStack with their OpenStack ID to upload their public
keys. RefStack test results can then be uploaded to RefStack using the
keys. RefStack test results need to be uploaded to RefStack using the
corresponding private key. By default, the uploaded data isn't shared,
but authorized users can decide to share the results with the community
anonymously.
@ -19,10 +19,11 @@ Register an OpenStack ID
The RefStack server uses OpenStack OpenID for user authentication.
Therefore, the RefStack server requires that anyone who wants to upload
test data to have an OpenStack ID. As you click on the Sign In/Sign Up
link on the RefStack pages, you will be redirected to the official
OpenStack user log in page where you can either log in with your
OpenStack ID or register for one. The registration page can also be
found directly through: https://www.openstack.org/join/register.
link on the `RefStack pages <https://refstack.openstack.org/#/>`__, you
will be redirected to the official OpenStack user log in page where you
can either log in with your OpenStack ID or register for one.
The registration page can also be found directly through:
https://www.openstack.org/join/register.
Generate ssh keys locally
^^^^^^^^^^^^^^^^^^^^^^^^^
@ -30,18 +31,18 @@ Generate ssh keys locally
You will need to generate ssh keys locally. If your operating system is
a Linux distro, then you can use the following instructions.
First check for existing keys with command:
First check for existing keys with command::
- ``ls -al ~/.ssh``
$ ls -al ~/.ssh
If you see you already have existing public and private keys that you
want to use, you can skip this step; otherwise:
want to use, you can skip this step; otherwise::
- ``ssh-keygen -m PEM -t rsa -b 4096 -C "youropenstackid"``
$ ssh-keygen -m PEM -t rsa -b 4096 -C "youropenstackid"
The 'youropenstackid' string is the username you chose when you
The `youropenstackid` string is the username you chose when you
registered for your OpenStack ID account. Enter the file name in which
to save the key (/home/you/.ssh/id\_rsa), then press enter. You will be
to save the key (``/home/you/.ssh/id\_rsa``), then press enter. You will be
asked to enter a passphrase. Just press enter again as passphrase
protected keys currently aren't supported. Your ssh keys will then be
generated.
@ -49,15 +50,17 @@ generated.
Sign Key with RefStack Client
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
\*\* IMPORTANT \*\* You must have the RefStack client on you computer to
**IMPORTANT** You must have the RefStack client on you computer to
complete this step.
Follow `its README <https://opendev.org/osf/refstack-client>`__ on how to
install it.
Generate a signature for your public key using your private key with
`refstack-client <https://opendev.org/osf/refstack-client>`__
`refstack-client <https://opendev.org/osf/refstack-client>`__::
- ``./refstack-client sign /path-of-sshkey-folder/key-file-name``
$ ./refstack-client sign /path-of-sshkey-folder/key-file-name
The '/path-of-sshkey-folder' string is the path of the folder where the
The ``/path-of-sshkey-folder`` string is the path of the folder where the
generated ssh keys are stored locally. The 'key-file-name' portion
refers to the private key file name. If the command runs correctly,
there will be output like below:
@ -74,37 +77,37 @@ Upload the ssh public key and the signature
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sign into https://refstack.openstack.org with your OpenStack ID. Click
the "Profile" link in the upper right corner. Now click the "Import
public key" button on your profile page. A popup window with two entry
the `Profile` link in the upper right corner. Now click the `Import
public key` button on your profile page. A popup window with two entry
fields will appear. Just copy and paste the key and signature generated
in the previous step into the corresponding textboxes.
Note that the literal strings 'Public key:' and 'Self signature:' from
the refstack-client "sign" command output should not be copied/pasted
into the text boxes. Otherwise you will get an error like:
Note that the literal strings `Public key:` and `Self signature:` from
the ``refstack-client sign`` command output **should not** be copied/pasted
into the text boxes. Otherwise you will get an error like::
- ``Bad Request Request doesnt correspond to schema``
Bad Request Request doesn't correspond to schema
Once complete, click the 'Import public key' button.
Once complete, click the `Import public key` button.
Upload the test result with refstack-client
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``./refstack-client upload /path_to_testresult_json_file --url https://refstack.openstack.org/api -i ~/.ssh/id_rsa``
The results can be uploaded using the ``refstack-client`` by::
\*\* NOTE \*\* Users may need to add the '--insecure' optional argument
to the command string if certificate validation issues occur when
uploading test result. To use with insecure:
- ``./refstack-client upload --insecure /path_to_testresult_json_file --url https://refstack.openstack.org/api -i ~/.ssh/id_rsa``
- The ``path_to_testresult_json_file`` here is the json file of your test result.
$ ./refstack-client upload /path_to_testresult_json_file \
--url https://refstack.openstack.org/api -i ~/.ssh/id_rsa
- By default, it's in ``.tempest/.stestr/<test-run-number>.json`` where refstack-client runs from.
**NOTE** Users may need to add the `--insecure` optional argument
to the command string if certificate validation issues occur when
uploading test result. Usage with the insecure argument::
- Here '' is a serial number that matches its corresponding subunit file name.
$ ./refstack-client upload --insecure /path_to_testresult_json_file \
--url https://refstack.openstack.org/api -i ~/.ssh/id_rsa``
- If the command runs correctly, there will be output like below:
The ``path_to_testresult_json_file`` there is the json file of the test result.
By default, it's in ``.tempest/.stestr/<test-run-number>.json``. If the command
runs correctly, there will be output like below:
::
@ -112,6 +115,5 @@ uploading test result. To use with insecure:
Test results uploaded!
URL: https://refstack.openstack.org/#/results/88a1e6f4-707d-4627-b658-b14b7e6ba70d.
You can find your uploaded test results by clicking the 'My Results'
You can find your uploaded test results by clicking the `My Results`
link on the RefStack website.

2
doc/source/vendor_product.rst

@ -88,4 +88,4 @@ Product version
A default version is created whenever a product is created. The name of the
default version is blank. The default version is used for products that have
no version. Users can add new product versions to the product as needed.
no version. Users can add new product versions to the product as needed.

12
playbooks/README.rst

@ -0,0 +1,12 @@
Playbook for running refstack locally
######################################
The playbook is meant for developers to help them with debugging and
reviewing new changes in the refstack project.
The playbook semi-automates running the refstack server on the localhost.
It downloads refstack role and templates from
`system-config <https://opendev.org/opendev/system-config.git>`__ repository
which is used for deploying and maintaining upstream servers, one of which is
refstack. Then it builds the refstack image and spins a container using the
refstack role.

52
playbooks/run-refstack-in-container.yml

@ -0,0 +1,52 @@
---
- hosts: localhost
become: true
gather_facts: true
vars:
# dir where refstack files for buidling an image, running a container
# will be stored
refstack_dir: /tmp/refstack-docker
refstack_openid_endpoint: ''
# ip address of the machine you're running this on
refstack_url: # 'http://<IP address of your server>'
# the default credentials for the refstack database
refstack_db_username: refstack
refstack_db_password: Jz4ooq9TL7nc3hX3
refstack_root_db_password: KbgY3r9HYnEYpgRP
tasks:
- name: Clone system-config repository
git:
repo: https://opendev.org/opendev/system-config.git
dest: "{{ refstack_dir }}/system-config"
- name: Extract docker files
copy:
src: "{{ refstack_dir }}/system-config/docker/refstack/"
dest: "{{ refstack_dir }}/refstack-docker-files"
remote_src: yes
- name: Extract refstack role
copy:
src: "{{ refstack_dir }}/system-config/playbooks/roles/refstack/"
dest: "{{ refstack_dir }}/refstack-role"
remote_src: yes
- name: Delete the rest of system-config content
file:
state: absent
path: "{{ refstack_dir }}/system-config"
- name: Install Docker
apt:
name:
- docker
- docker-compose
state: present
- name: Build refstack image
command: docker image build -f Dockerfile -t refstack:1 .
args:
chdir: "{{ refstack_dir }}/refstack-docker-files"
- include_role:
name: "{{ refstack_dir }}/refstack-role"
Loading…
Cancel
Save