x/tobiko
Code Issues Proposed changes

Merge "Modify quick-start doc"

Browse Source
This commit is contained in:
Zuul 2019-04-17 07:26:58 +00:00 committed by Gerrit Code Review
commit 361e3f471e
1 changed files with 28 additions and 228 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

256
docs/quick-start.md
View File

@ -1,241 +1,41 @@
# Quick start guide # Quick start guide
## Before reading
This is work in progress and what documented here is still under development.
Some of below sections should be shorter here and more detailed in dedicated
documents. Because this documentation is a Draft for features still to be
implemented and discussed, its format and organization is secondary at this
stage and it is probably going to change soon.
## Overview
Tobiko is a tool used to prepare cloud test resources for system testing and
make them accessible to test cases written in Python with (or without) tempest.
The main purpose of Tobiko is taking out test resources management from test
cases to allow them to have a longer life than test case execution.
This a fundamental requirement that motivated this project with the purpose of
testing global cloud operations like whole cloud upgrades, reboot, migration.
It could also be used to test other smaller scale operations like for
example single node evacuation.
To avoid re-inventing the wheel Tobiko integrates some pre-existing tools
with the purpose of setting up and managing test resources. Example of such
tools integrated or to be integrated are:
- OpenStack Python clients
- OpenStack Heat Orchestration Service to manage test resources. The test
developer is expected to use Heat as core tool for writing part of its test
cases, in special cases test fixtures to reach initial test case state
and to prepare test resources to be used inside test case. Tobiko should
provide a Python interface to create Heat stacks starting from user provided
"Heat Orchestartion Templates" and configuration parameters taken from
user provided tobiko.conf file.
- (TODO) Ansible Configuration Management tool user to prepare testing
environment and test fixtures. Some fixtures are better handled by this
non-Openstack generic tool, even if it offers a completely different approach
to achieve the same purpose of Heat. Test developer should decide when to
prefer this tool or others according his experience and his goals.
This is another peculiar aspect of Tobiko that distinguish it from other tools
like Tempest and it is being chosen mostly to take advantage of a good mature
production tool-set before introducing a new one from scratch.
It has been discussed about using tempest as fundamental toolset for setting
up resources and it has been forecast that to avoid some limitations and
complications Tempest has, it would be better to choose Orchestration
tools that has specifically designed for managing cloud resources from its
fundamentals. This has also another implicit minor advantage: Tobiko is also
a tool for testing Heat and regular OpenStack Python clients, while tempest
choose to re-implement clients and resources management from scratch (work
mostly incomplete and under very slow maintenance).
## Install Tobiko ## Install Tobiko
Tobiko is preferably used from a dedicated virtual environment: virtualenv path/to/yuor/venv
source path/to/yuor/venv/bin/activate
```
virtualenv path/to/virtualenv
source path/to/virtualenv/bin/activate
```
You can download Tobiko source code and install it in a single step inside
your environment by typing:
```
pip install git+https://git.openstack.org/openstack/tobiko
```
or simply
```
pip install tobiko pip install tobiko
```
Tobiko doesn't need to be used from its source code directory, so, by instance, ## Set up credentials
after you installed it, you could use it as any Python tool from the directory
where your own test files are.
Tobiko is also distributed together with some example test cases that could In order to run the tests successfully you'll need to set up OpenStack credentials.
be used as a reference and this option grants you the ability to see if Tobiko You can do it in one of two ways:
is properly installed on your system.
1. Using environment variables.
Either by downloading the credentials file directly from your OpenStack project or
setting them up manually like this:
export API_VERSION = 2
export OS_USERNAME = admin
export OS_PASSWORD = admin
export PROJECT_NAME = admin
export OS_USER_DOMAIN_NAME="Default"
export OS_PROJECT_DOMAIN_NAME = admin
export OS_AUTH_URL=https://my_cloud:13000/v3
## Tobiko Workflow 2. Setting up tobiko.conf file with the following format:
Here we are going to describe the typical Tobiko workflow in a few simple api_version = 2
steps username = admin
password = admin
project_name = admin
user_domain_name = admin
project_domain_name = admin
auth_url = http://my_cloud:13000/v3
## Run Tests
### Step 1) Set up your cloud To run neutron tests, use the following command:
Tobiko requires a pre-installed OpenStack cloud. As it relies on tox -e neutron
python-openstackclient, you also have to assign environment variable with
the client credentials.
```
source <your-stackrc-file>
```
for example:
```
source overcloudrc
```
In order to verify Tobiko's connectivity, exectute following command:
```
openstack stack list
```
#### Configure Tobiko
Create your tobiko.conf file
TODO: document here minimal configuration file options required for executing
test cases.
### Step 2) Populate test resources
Tobiko creates a set of persisting resources (eventually using Heat) to be used
in test cases.
```
tobiko create
```
Tobiko is going to look into the current folder, searching into all registered
Python test packages for all test modules (Python files matching wildcard
expression ```test_*.py```). Every test case would eventually have a reference
to some Tobiko fixture. Some of these fixtures could be used as an example of a
stack of resources orchestracted from OpenStack Heat tool. By importing those
test case modules Tobiko will be able to find these fixtures and set them all
up.
The list of these fixtures can be narrowed down by using expressions to select
test cases to be considered. A test case could also be identified from its Python
source code files as below:
```
tobiko create [-f|--file] <test-case-files|test-case-dirs>...
```
Or it could be restricted using test case packages or file names as below:
```
tobiko create (-m|--module) <test-case-names|test-case-packages>...
```
To have a summary of all test cases which resources could be set up using
Tobiko you could type:
```
tobiko list
```
To have a summary of all test cases which resources has been actually set up
using Tobiko you can type:
```
tobiko list (-e|--existing)
```
### 3) Run test cases
Once test cloud resources (also called here Tobiko test fixtures) are ready to
be used, test cases can finally be executed to check if the cloud is working by
executing test cases with such preconfigured test resources.
To run test cases you can type:
```
tobiko run <test-case-files|test-case-dirs>...
```
or similarly to Tobiko create command:
```
tobiko run [-m|--module] <test-case-modules|test-case-packates>...
```
You can also use your favorite Python test runner as there are very good ones
around.
```
stestr ...
```
Please note Tobiko is only a wrapper around your favorite test runner and
you could change it by configuring Tobiko.
The first time that a test case requires test resources managed by Tobiko then
Tobiko is going to discover test case resources from your given stack name.
To achieve it, test case developer is expected to use Tobiko Python API.
### 4) Execute your cloud operation
In this step you should execute a sequence of cloud operations that you want to
test. For example you could upgrade your OpenStack installation in order to see
if the test resources managed by Tobiko are still working as expected.
### 5) Run test cases again
After executing your operations on top of your OpenStack instance, you should
expect test resources set up by Tobiko to still be there and they should pass
the same test cases that has been passed at step 3. So here you should execute
the same command line you executed at step 3:
```
tobiko run [<test-cases>]
```
### 6) Cleanup all populated resources
Finally once all your tests has been executed you can ask Tobiko to cleanup
all resources by typing below command:
```
tobiko delete
```
The previous command could also be more selective by deleting only test resources
for specific test cases:
```
tobiko delete <test-case-files|test-case-dirs>
```
```
tobiko delete (-m|--module) <test-case-modules|test-case-modules>
```