58 lines
3.0 KiB
ReStructuredText
58 lines
3.0 KiB
ReStructuredText
..
|
|
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
not use this file except in compliance with the License. You may obtain
|
|
a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
License for the specific language governing permissions and limitations
|
|
under the License.
|
|
|
|
Historical background
|
|
---------------------
|
|
|
|
Tempest, OpenStack's official test suite, is a powerful tool for running a set
|
|
of functional tests against an OpenStack cluster. Tempest automatically runs
|
|
against every patch in every project of OpenStack, which lets us avoid merging
|
|
changes that break functionality.
|
|
|
|
Unfortunately, it has limited opportunities to be used, to process its results,
|
|
etc. That is why we started Verification Component initiative a long time ago
|
|
(see `a blog post
|
|
<https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler/>`_
|
|
for more details, but be careful as all user interface is changed completely
|
|
since that time).
|
|
|
|
What is Verification Component and why do you need it?
|
|
------------------------------------------------------
|
|
|
|
The primary goal of Rally Product is to provide a simple way to do complex
|
|
things. As for functional testing, Verification Component includes interfaces
|
|
for:
|
|
|
|
* **Managing things**. Create an isolated virtual environment and install
|
|
verification tool there? Yes, we can do it! Clone tool from Git repositories?
|
|
Sure! Store several versions of one tool (you know, sometimes they are
|
|
incompatible, with different required packages and so on)? Of course!
|
|
In general, Verification Component allows to install, upgrade, reinstall,
|
|
configure your tool. You should not care about zillion options anymore Rally
|
|
will discover them via cloud UX and make the configuration file for you
|
|
automatically.
|
|
* **Launching verifiers**. Launchers of specific tools don't always contain all
|
|
required features, Rally team tries to fix this omission. Verification
|
|
Component supports some of them like expected failures, a list of tests to
|
|
skip, a list of tests to launch, re-running previous verification or just
|
|
failed tests from it and so on. Btw, all verification runs arguments are
|
|
stored in the database.
|
|
* **Processing results**. Rally DataBase stores all `verifications
|
|
<link-to-glossary>`_ and you can obtain unified (across different verifiers)
|
|
results at any time. You can find a verification run summary there, run
|
|
arguments which were used, error messages and etc. Comparison mechanism for
|
|
several verifications is available too. Verification reports can be generated
|
|
in several formats: HTML, JSON, JUnit-XML (see :ref:`verification-reports`
|
|
for more details). Also, reports mechanism is expendable and you can write
|
|
your own plugin for whatever system you want.
|