diff --git a/README.rst b/README.rst index 3eba61f78..1b1e265a3 100644 --- a/README.rst +++ b/README.rst @@ -2,23 +2,35 @@ Tobiko ====== -Tobiko is an OpenStack testing framework focusing on areas -complementary to Tempest: -- Minor updates and major upgrades. Tobiko tests populate the - cloud with workloads such as instances, allows the CI workflow - to perform an operation such as an update or upgrade, and then runs - tests that validate that the cloud workloads are still functional. -- Fault injection, like restarting nodes, OpenStack services and - dependencies such as OVS, RabbitMQ or the DB. -- White box testing; Specifically the ability to run commands on - nodes. +Test Big Cloud Operations +------------------------- -Links -~~~~~ +Tobiko is an OpenStack testing framework focusing on areas mostly +complementary to `Tempest `__. +While tempest main focus has been testing OpenStack rest APIs, the main Tobiko +focus would be to test OpenStack system operations while "simulating" +the use of the cloud as the final user would. -* Documentation: `Tobiko Documentation `__ -* Usage: `Tobiko User Guide `__ -* Bugs: `Tobiko StoryBoard `__ -* Source Code: https://opendev.org/x/tobiko.git -* License: `Apache License v2.0 `__ +Tobiko's test cases populate the cloud with workloads such as instances, allows +the CI workflow to perform an operation such as an update or upgrade, and then +runs test cases to validate that the cloud workloads are still functional. + + +Main Project Goals +~~~~~~~~~~~~~~~~~~ + +- Provide a Python framework to write system scenario test cases; +- Provide CLI tools to implement a workflow designed to test potentially + destructive operations (like cloud nodes reboot, update, upgrade...). +- Provide tools to monitor and recollect the healthy status of the cloud + + +References +---------- + +* Free software: Apache License, Version 2.0 +* Documentation: https://docs.openstack.org/tobiko/latest/ +* Release notes: https://docs.openstack.org/releasenotes/tobiko/ +* Source: https://opendev.org/x/tobiko +* Bugs: https://storyboard.openstack.org/#!/project/x/tobiko diff --git a/doc/source/contents.rst b/doc/source/contents.rst new file mode 100644 index 000000000..adc1ca018 --- /dev/null +++ b/doc/source/contents.rst @@ -0,0 +1,13 @@ +Tobiko Documentation Contents +----------------------------- + +.. toctree:: + :maxdepth: 2 + + user/index + contributor/index + +.. toctree:: + :maxdepth: 1 + + reference/index diff --git a/doc/source/index.rst b/doc/source/index.rst index 79b02afd8..7943b7f88 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -1,6 +1,3 @@ -Tobiko -====== - .. include:: ../../README.rst Document Overview @@ -9,24 +6,14 @@ Document Overview This document describes the tools for final user and contributors of the project, and assumes that you are already familiar with OpenStack from an end-user perspective. If not, hop over to the -`OpenStack doc site `__ +`OpenStack doc site `__. You can look for additional documentation also in the -`OpenStack wiki `__ +`OpenStack wiki `__. -This documentation is generated by the Sphinx toolkit and lives in the source -tree. +This documentation is generated by the Sphinx toolkit and lives in the `source +tree `__. Enjoy! -.. toctree:: - :maxdepth: 2 - - install/index - user/index - contributor/index - -.. toctree:: - :maxdepth: 1 - - reference/index +.. include:: contents.rst diff --git a/doc/source/install/index.rst b/doc/source/install/index.rst deleted file mode 100644 index 12c9c15d3..000000000 --- a/doc/source/install/index.rst +++ /dev/null @@ -1,50 +0,0 @@ -.. _tobiko-install-guide: - -========================= -Tobiko Installation Guide -========================= - - -Document Overview ------------------ - -This document describes how to install Tobiko inside a Python virtualenv. For -a quick and simpler start you can jump to the :ref:`quick-start-guide`. - - -Install Tobiko Using Virtualenv -------------------------------- - -Make sure Gcc, Git and base Python packages are installed on your system. -For instance on RHEL Linux you could type:: - - sudo yum install -y gcc git python python-devel - -Make sure pip and setuptools are installed and up-to date:: - - wget https://bootstrap.pypa.io/get-pip.py - sudo python get-pip.py - sudo pip install --upgrade setuptools - -Make sure virtualenv and wheel are installed and up to date:: - - sudo pip install --upgrade tox virtualenv wheel - -Get Tobiko source code using Git:: - - git clone https://opendev.org/x/tobiko.git - cd tobiko - -To install Tobiko and its dependencies is safer to create a clean Virtualenv -where to install it. Create a Virtualenv and activate it:: - - virtualenv .tobiko-env - source .tobiko-env/bin/activate - -Install Tobiko and its requirements:: - - pip install \ - -c https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt \ - -r requirements.txt \ - -r extra-requirements.txt - pip install . diff --git a/doc/source/user/config.rst b/doc/source/user/config.rst new file mode 100644 index 000000000..1e72fdde8 --- /dev/null +++ b/doc/source/user/config.rst @@ -0,0 +1,273 @@ +.. _tobiko-configuration-guide: + +========================== +Tobiko Configuration Guide +========================== + + +Document Overview +----------------- + +This document describes how to configure Tobiko. + +.. sidebar:: See also + + For a quick and simpler start you can jump to the + :ref:`tobiko-quick-start-guide`. + + To install Tobiko inside a virutalenv please read + :ref:`tobiko-installation-guide`. + + To run Tobiko scenario test cases please look at + :ref:`tobiko-test-case-execution-guide`. + + +Configure Tobiko Framework +-------------------------- + +In order to make sure Tobiko tools can connect to OpenStack services via Rest +API configuration parameters can be passed either via environment variables or +via a ini configuration file (referred here as :ref:`tobiko-conf`). Please look +at :ref:`authentication-methods` for more details. + +To be able to execute scenario test cases there some OpenStack resources that +has to be created before running test cases. Please look at +:ref:`setup-required-resources` for more details. + + +.. _tobiko-conf: + +tobiko.conf +~~~~~~~~~~~ + +Tobiko tries to load :ref:`tobiko-conf` file from one of below locations: + +* current directory:: + + ./tobiko.conf + +* user home directory:: + + ~/.tobiko/tobiko.conf + +* system directory:: + + /etc/tobiko/tobiko.conf + + +Configure Logging +~~~~~~~~~~~~~~~~~ + +Tobiko can configure logging system to write messages to a log file. You can +edit below options in :ref:`tobiko-conf` to enable it as below:: + + [DEFAULT] + # Whenever to allow debugging messages to be written out or not + debug = true + + # Name of the file where log messages will be appended. + log_file = tobiko.log + + # The base directory used for relative log_file paths. + log_dir = . + + +.. _authentication-methods: + + +Authentication Methods +~~~~~~~~~~~~~~~~~~~~~~ + +Tobiko uses +`OpenStack client `__ +to connect to OpenStack services. + + +.. _authentication-environment-variables: + +Authentication Environment Variables +++++++++++++++++++++++++++++++++++++ + +To configure how Tobiko can connect to +services you can use the same +`environment variables `__ +you would use for OpenStack Python client CLI. + +Currently supported variables are:: + + # Identity API version + export OS_IDENTITY_API_VERSION=3 + + # URL to be used to connect to OpenStack Irentity Rest API service + export OS_AUTH_URL=http://10.0.0.109:5000/v3 + + # Authentication username (name or ID) + export OS_USERNAME=admin + export OS_USER_ID=... + + # Authentication password + export OS_PASSWORD=... + + # Project-level authentication scope (name or ID) + export OS_PROJECT_NAME=admin + export OS_TENANT_NAME=admin + export OS_PROJECT_ID=... + export OS_TENANT_ID=... + + # Domain-level authorization scope (name or ID) + export OS_DOMAIN_NAME=default + export OS_DOMAIN_ID=... + + # Domain name or ID containing user + export OS_USER_DOMAIN_NAME=default + export OS_USER_DOMAIN_ID=... + + # Domain name or ID containing project + export OS_PROJECT_DOMAIN_NAME=default + export OS_PROJECT_DOMAIN_ID=... + + # ID of the trust to use as a trustee user + export OS_TRUST_ID=... + + +.. _authentication-configuration: + +Autentication Configuration ++++++++++++++++++++++++++++ + +You can also configure the same authentication parameters by editing 'keystone' +section in :ref:`tobiko-conf` file. For example:: + + [keystone] + # Identity API version + api_version = 3 + + # URL to be used to connect to OpenStack Irentity Rest API service + auth_url=http://10.0.0.109:5000/v3 + + # Authentication username (name or ID) + username = admin + + # Authentication password + password = ... + + # Project-level authentication scope (name or ID) + project_name = admin + + # Domain-level authorization scope (name or ID) + domain = default + + # Domain name or ID containing user + user_domain_name = default + + # Domain name or ID containing prject + project_domain_name = default + + # ID of the trust to use as a trustee user + trust_id = ... + + +.. _proxy-server-configuration: + +Proxy Server Configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The first thing to make sure is Tobiko can reach OpenStack services. In case +OpenStack is not directly accessible from where test cases or Tobiko CLI are +executed it is possible to use an HTTP proxy server running on a network that +is able to reach all OpenStack Rest API service. This can be performed +by using below standard environment variables:: + + export http_proxy=http://:/ + export https_proxy=http://:/ + export no_proxy=127.0.0.1,... + +For convenience it is also possible to specify the same parameters via +:ref:`tobiko-conf`:: + + [http] + http_proxy = http://:/ + https_proxy = http://:/ + no_proxy = 127.0.0.1,... + +Because Tobiko test cases could execute local commands (like for example ping) +to reach network services we have to specify in tobiko.conf file a shell +(like OpenSSH client) to be used instead of the default local one +('/bin/sh'):: + + [shell] + command = /usr/bin/ssh + +Please make sure it is possible to execute commands on local system without +having to pass a password:: + + /usr/bin/ssh echo 'Yes it works!' + +To archive it please follow one of the +`many guides available on Internet +`__ +. + + +.. _setup-required-resources: + +Setup Required Resources +~~~~~~~~~~~~~~~~~~~~~~~~ + +To be able to execute Tobiko scenario test cases there some OpenStack +resources that has to be created before running test cases. + +Install required Python OpenStack clients:: + + pip install --upgrade \ + -c https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt \ + python-openstackclient \ + python-glanceclient \ + python-novaclient \ + python-neutronclient + +You need to make sure ref:`authentication-environment-variables` are properly +set:: + + source openstackrc + openstack image list + openstack flavor list + openstack network list + +`Get an image `__ +for Nova instances created by Tobiko:: + + wget http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img + openstack image create cirros \ + --file cirros-0.4.0-x86_64-disk.img \ + --disk-format qcow2 \ + --container-format bare \ + --public + +Create a flavor to be used with above image:: + + openstack flavor create --vcpus 1 --ram 64 --disk 1 m1.tiny + +Create an SSH key file to be used to ssh to Nova server instances:: + + ssh-keygen -f ~/.ssh/id_rsa -P '' + +Add reference to above resources into your :ref:`tobiko-conf` file:: + + [nova] + image = cirros + flavor = m1.tiny + key_file=~/.ssh/id_rsa + +Add reference to the network where Tobiko should create floating IP instances +in :ref:`tobiko-conf` file:: + + [neutron] + floating_network = public + + +What's Next +----------- + +To know how to run Tobiko scenario test cases you can look at +:ref:`tobiko-test-case-execution-guide` diff --git a/doc/source/user/index.rst b/doc/source/user/index.rst index c93e9cf88..f701079ec 100644 --- a/doc/source/user/index.rst +++ b/doc/source/user/index.rst @@ -1,3 +1,5 @@ +.. _tobiko-user-guide: + ================= Tobiko User Guide ================= @@ -6,3 +8,6 @@ Tobiko User Guide :maxdepth: 2 quick-start + install + config + run-test-cases diff --git a/doc/source/user/install.rst b/doc/source/user/install.rst new file mode 100644 index 000000000..b1c771b35 --- /dev/null +++ b/doc/source/user/install.rst @@ -0,0 +1,69 @@ +.. _tobiko-installation-guide: + +========================= +Tobiko Installation Guide +========================= + + +Document Overview +----------------- + +This document describes how to install Tobiko inside a +`Python Virtualenv `__. + +.. sidebar:: See also + + For a quick and simpler start you can jump to the + :ref:`tobiko-quick-start-guide`. + + To configure Tobiko please read :ref:`tobiko-configuration-guide`. + + To run Tobiko scenario test cases please look at + :ref:`tobiko-test-case-execution-guide`. + +Install Tobiko Using Virtualenv +------------------------------- + +Make sure Gcc, Git and base Python packages are installed on your system. + +For instance on RHEL Linux 7.6 or CentOS 7 you could type:: + + sudo yum install -y gcc git python python-devel wget + +For instance on RHEL Linux 8 or CentOS 8 you could type:: + + sudo dnf install -y gcc git python3 python3-devel wget + sudo alternatives --set python /usr/bin/python3 + +Make sure pip is installed and up-to date:: + + wget https://bootstrap.pypa.io/get-pip.py + sudo python get-pip.py + PIP=$(which pip) + +Make sure setuptools, virtualenv and wheel are installed and up to date:: + + sudo $PIP install --upgrade setuptools virtualenv wheel + +Get Tobiko source code using Git and enter into Tobiko soruce folder:: + + git clone https://opendev.org/x/tobiko.git + cd tobiko + +To install Tobiko and its dependencies is safer to create a clean Virtualenv +where to install it. Create a Virtualenv and activate it:: + + virtualenv .tobiko-env + source .tobiko-env/bin/activate + +Install Tobiko and its requirements:: + + pip install \ + -c https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt \ + . + + +What's Next +----------- + +To know how to configure Tobiko please read :ref:`tobiko-configuration-guide`. diff --git a/doc/source/user/quick-start.rst b/doc/source/user/quick-start.rst index 0e43cae57..b7328b899 100644 --- a/doc/source/user/quick-start.rst +++ b/doc/source/user/quick-start.rst @@ -1,35 +1,50 @@ -.. _quick-start-guide: +.. _tobiko-quick-start-guide: -================= -Quick Start guide -================= +======================== +Tobiko Quick Start Guide +======================== Document Overview ----------------- This document describes how to install execute Tobiko scenarios test cases -using Tox. To install Tobiko inside a virtualenv directory please refers to -:ref:`tobiko-install-guide`. +using `Tox `__. + +.. sidebar:: See also + + To install Tobiko inside a virutalenv please read + :ref:`tobiko-installation-guide`. + + To configure Tobiko please read :ref:`tobiko-configuration-guide`. + + To run Tobiko scenario test cases please look at + :ref:`tobiko-test-case-execution-guide`. Install Required Packages ------------------------- Make sure Gcc, Git and base Python packages are installed on your system. + For instance on RHEL Linux you could type:: sudo yum install -y gcc git python python-devel +For instance on RHEL Linux 8 or CentOS 8 you could type:: + + sudo dnf install -y gcc git python3 python3-devel wget + sudo alternatives --set python /usr/bin/python3 + Make sure pip and setuptools are installed and up-to date:: wget https://bootstrap.pypa.io/get-pip.py sudo python get-pip.py - sudo pip install --upgrade setuptools + PIP=$(which pip) -Make sure tox, virtualenv and wheel are installed and up to date:: +Make sure setuptools, wheel, virtualenv, and tox are installed and up to date:: - sudo pip install --upgrade tox virtualenv wheel + sudo $PIP install --upgrade setuptools wheel virtualenv tox Get Tobiko @@ -56,73 +71,94 @@ credentials. You can do it in one of below ways: Set Tobiko Credentials Via Environment Variables ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can use an existing shell rc file that is valid for Python OpenStack -client:: +.. sidebar:: See also + + For more details about supported environment variables please read + :ref:`authentication-environment-variables` section. + +You can use an existing shell RC file that is valid for +`Python OpenStack client `__ +:: source openstackrc An example of 'openstackrc' file could looks like below:: - export API_VERSION=2 + export OS_IDENTITY_API_VERSION=3 + export OS_AUTH_URL=https://my_cloud:13000/v3 export OS_USERNAME=admin export OS_PASSWORD=secret - export PROJECT_NAME=admin + export OS_PROJECT_NAME=admin export OS_USER_DOMAIN_NAME=Default - export OS_PROJECT_DOMAIN_NAME=admin - export OS_AUTH_URL=https://my_cloud:13000/v3 + export OS_PROJECT_DOMAIN_NAME=Default .. _credentials-from-config: -Set Tobiko Credentials Via 'tobiko.conf' File -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Set Tobiko Credentials Via :ref:`tobiko-conf` File +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Create a file at ~/.tobiko/tobiko.conf adding a section like below:: +.. sidebar:: See also + + For more details about supported configuration options please read + :ref:`authentication-configuration` section. + +Create a file at `~/.tobiko/tobiko.conf` adding a section like below:: [keystone] - api_version = 2 + api_version = 3 + auth_url = http://my_cloud:13000/v3 username = admin password = secret project_name = admin - user_domain_name = admin - project_domain_name = admin - auth_url = http://my_cloud:13000/v3 + user_domain_name = Default + project_domain_name = Default -Create Required Resources -~~~~~~~~~~~~~~~~~~~~~~~~~ +Setup Required Resources +~~~~~~~~~~~~~~~~~~~~~~~~ -Install Python OpenStack client:: +To be able to execute Tobiko scenario test cases there some OpenStack +resources that has to be created before running test cases. - source overcloudrc - sudo pip install python-openstackclient \ - python-novaclient \ - python-glanceclient +To execute commands from a virtualenv created by Tox you can type as below:: -Create an image for Nova instances created by Tobiko:: + tox -e venv -- - wget http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img - openstack image create "cirros" \ - --file cirros-0.3.5-x86_64-disk.img \ +You need to make sure ref:`authentication-environment-variables` are properly +set:: + + tox -e venv -- openstack image list + tox -e venv -- openstack flavor list + tox -e venv -- openstack network list + +`Get an image `__ +for Nova instances created by Tobiko:: + + wget -c http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img + tox -e venv -- openstack image create cirros \ + --file cirros-0.4.0-x86_64-disk.img \ --disk-format qcow2 \ --container-format bare \ --public Create a flavor to be used with above image:: - openstack flavor create --id 0 --vcpus 1 --ram 64 --disk 1 m1.tiny + tox -e venv -- openstack flavor create --vcpus 1 --ram 64 --disk 1 m1.tiny -Add reference to above resources into your `tobiko.conf` file:: +Create an SSH key file to be used to ssh to Nova server instances:: + + ssh-keygen -f ~/.ssh/id_rsa -P '' + +Add reference to above resources into your :ref:`tobiko-conf` file:: [nova] image = cirros flavor = m1.tiny + key_file=~/.ssh/id_rsa - -Configure Public Network Name -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Add reference to the network where Tobiko should create floating IP instances:: +Add reference to the network where Tobiko should create floating IP instances +in :ref:`tobiko-conf` file:: [neutron] floating_network = public @@ -131,6 +167,10 @@ Add reference to the network where Tobiko should create floating IP instances:: Run Test Cases -------------- -Run Tobiko scenario test cases using Tox:: +Finally run Tobiko scenario test cases using Tox:: tox -e scenario + +List resources stacks created by test cases:: + + tox -e venv -- openstack stack list diff --git a/doc/source/user/run-test-cases.rst b/doc/source/user/run-test-cases.rst new file mode 100644 index 000000000..8df12730d --- /dev/null +++ b/doc/source/user/run-test-cases.rst @@ -0,0 +1,39 @@ +.. _tobiko-test-case-execution-guide: + +================================= +Tobiko Test Cases Execution Guide +================================= + +This document describes how to execute Tobiko scenario test cases. + +.. sidebar:: See also + + For a quick and simpler start you can jump to the + :ref:`tobiko-quick-start-guide`. + + To install Tobiko inside a virutalenv please read + :ref:`tobiko-installation-guide`. + + To configure Tobiko please read :ref:`tobiko-configuration-guide`. + + +Prepare Your System +~~~~~~~~~~~~~~~~~~~ + +Before running Tobiko test cases you need to be sure you are doing it from +Tobiko source files folder and that you have actived a Virtualenv where Tobiko +and its requirements are installed. Please refers to +:ref:`tobiko-installation-guide` and :ref:`tobiko-configuration-guide` to know +how to setup your system before running test cases. + + +Run Scenario Test Cases +~~~~~~~~~~~~~~~~~~~~~~~ + +To run test cases you need a test runner able to execute Python test cases. +Test cases delivered with Tobiko has been tested using +`stestr `__ + +From Tobiko source folder you can run scenario test cases using below command:: + + stestr run --test-path tobiko/tests/scenario/