openstack/tripleo-quickstart
Code Issues Proposed changes

Add ability to run tripleo-validations tests

Browse Source 
This patch allows us to run tripleo-validations tests through the
Mistral framework and running negative tests from Ansible directly.

Depends-On: Iec07a18322252af2d484bc3f06a59dbd1b08f537

Change-Id: I2e0831fc8b7bdf6acca1d0f99bf41adcad94f688
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
This commit is contained in:
Gael Chamoulaud 2016-11-28 15:26:47 +01:00
parent 18566f5dfd
commit d565afccf9
10 changed files with 185 additions and 57 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
config/general_config/featureset004.yml
View File

@ -2,6 +2,14 @@ undercloud_generate_service_certificate: false
ssl_overcloud: false
test_ping: true
# This enables the run of several tripleo-validations tests through Mistral
run_tripleo_validations: True
# This enables the run of tripleo-validations negative tests through shell
# scripts
run_tripleo_validations_negative_tests: True
# Exit tripleo-quickstart on validations failure
exit_on_validations_failure: False
deploy_composable_scenario: false
composable_services: true
composable_scenario: multinode.yaml

8
config/general_config/featureset005.yml
View File

@ -2,6 +2,14 @@ undercloud_generate_service_certificate: false
enable_opstools_repo: true
test_ping: true
# This enables the run of several tripleo-validations tests through Mistral
run_tripleo_validations: True
# This enables the run of tripleo-validations negative tests through shell
# scripts
run_tripleo_validations_negative_tests: True
# Exit tripleo-quickstart on validations failure
exit_on_validations_failure: False
deploy_composable_scenario: true
composable_scenario: scenario001-multinode.yaml
validate_template: scenario001-multinode.yaml

8
config/general_config/featureset006.yml
View File

@ -1,6 +1,14 @@
undercloud_generate_service_certificate: false
test_ping: true
# This enables the run of several tripleo-validations tests through Mistral
run_tripleo_validations: True
# This enables the run of tripleo-validations negative tests through shell
# scripts
run_tripleo_validations_negative_tests: True
# Exit tripleo-quickstart on validations failure
exit_on_validations_failure: False
deploy_composable_scenario: true
composable_scenario: scenario002-multinode.yaml
validate_template: scenario002-multinode.yaml

8
config/general_config/featureset007.yml
View File

@ -1,6 +1,14 @@
undercloud_generate_service_certificate: false
test_ping: true
# This enables the run of several tripleo-validations tests through Mistral
run_tripleo_validations: True
# This enables the run of tripleo-validations negative tests through shell
# scripts
run_tripleo_validations_negative_tests: True
# Exit tripleo-quickstart on validations failure
exit_on_validations_failure: False
deploy_composable_scenario: true
composable_scenario: scenario003-multinode.yaml
validate_template: scenario003-multinode.yaml

8
config/general_config/featureset008.yml
View File

@ -1,6 +1,14 @@
undercloud_generate_service_certificate: false
test_ping: true
# This enables the run of several tripleo-validations tests through Mistral
run_tripleo_validations: True
# This enables the run of tripleo-validations negative tests through shell
# scripts
run_tripleo_validations_negative_tests: True
# Exit tripleo-quickstart on validations failure
exit_on_validations_failure: False
deploy_composable_scenario: true
composable_scenario: scenario004-multinode.yaml
validate_template: scenario004-multinode.yaml

8
config/general_config/minimal.yml
View File

@ -31,6 +31,14 @@ test_ping: true
run_tempest: false
tempest_workers: 4
# This enables the run of several tripleo-validations tests through Mistral
run_tripleo_validations: True
# This enables the run of tripleo-validations negative tests through shell
# scripts
run_tripleo_validations_negative_tests: True
# Exit tripleo-quickstart on validations failure
exit_on_validations_failure: False
# options below direct automatic doc generation by tripleo-collect-logs
artcl_gen_docs: true
artcl_create_docs_payload:

114
doc/source/feature-configuration.rst
View File

@ -18,63 +18,63 @@ Below is a table with various features listed in each row and the features enabl
in each feature set configuration file in each column. When adding new configurations
please consult the following `etherpad <https://etherpad.openstack.org/p/quickstart-featuresets>`_
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| FS | ipv6 | SSL UC | SSL OC | NI SN | NI MN | NI BOND | introsp | UC idem | cont UC | cont OC | cont upg | ping | tempest | sc001 | sc002 | sc003 | sc004 | sc005 | sc006 | sc007 |nonha | ceph | HC |
+=====+======+========+========+=======+=======+=========+=========+=========+=========+=========+==========+======+=========+=======+=======+=======+=======+=======+=======+=======+========+======+====+
| 001 | | X | X | | X | | X | | | | | X | | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 002 | | X | X | | X | | X | | | | | X | | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 003 | | | | | | | | X | | | | | | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 004 | | | | | | | | | | | | | | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 005 | | | | | | | | | | | | X | | X | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 006 | | | | | | | | | | | | X | | | X | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 007 | | | | | | | | | | | | X | | | | X | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 008 | | | | | | | | | | | | X | | | | | X | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 009 | | | | | | | | | | | | X | | | | | | X | X | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 010 | | | | | | | | | | X | | | | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 011 | | | | | | | | | | | X | | | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 012 | | | | | | | | | | | X | | | X | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 013 | | | | | | | | | | | X | | | | X | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 014 | | | | | | | | | | | X | | | | | X | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 015 | | | | | | | | | | | X | | | | | | X | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 016 | | | | | | | | | | X | | | | X | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 017 | | | | | | | | | | X | | | | | X | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 018 | | | | | | | | | | X | | | | | | X | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 019 | | | | | | | | | | X | | | | | | | X | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 020 | | | | | | | | | | | | | X | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 021 | | X | X | | X | | X | | | | | | X | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 022 | | X | | | | | X | | | X | | X | | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 024 | X | X | | | X | | X | | | | | X | | | | | | | | | | X | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 025 | | X | X | | X | | X | | | | | X | | | | | | | | | | | X |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 026 | | | | | | | | | | X | | | | | | | | | X | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 027 | | | | | | | | | X | | | | | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 028 | | | | | | | | | | | | X | | | | | | | | X | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| FS | ipv6 | SSL UC | SSL OC | NI SN | NI MN | NI BOND | introsp | UC idem | cont UC | cont OC | cont upg | validations | ping | tempest | sc001 | sc002 | sc003 | sc004 | sc005 | sc006 | sc007 |nonha | ceph | HC |
+=====+======+========+========+=======+=======+=========+=========+=========+=========+=========+==========+=============+======+=========+=======+=======+=======+=======+=======+=======+=======+========+======+====+
| 001 | | X | X | | X | | X | | | | | | X | | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 002 | | X | X | | X | | X | | | | | | X | | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 003 | | | | | | | | X | | | | | | | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 004 | | | | | | | | | | | | X | | | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 005 | | | | | | | | | | | | X | X | | X | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 006 | | | | | | | | | | | | X | X | | | X | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 007 | | | | | | | | | | | | X | X | | | | X | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 008 | | | | | | | | | | | | X | X | | | | | X | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 009 | | | | | | | | | | | | | X | | | | | | X | X | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 010 | | | | | | | | | | X | | | | | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 011 | | | | | | | | | | | X | | | | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 012 | | | | | | | | | | | X | | | | X | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 013 | | | | | | | | | | | X | | | | | X | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 014 | | | | | | | | | | | X | | | | | | X | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 015 | | | | | | | | | | | X | | | | | | | X | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 016 | | | | | | | | | | X | | | | | X | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 017 | | | | | | | | | | X | | | | | | X | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 018 | | | | | | | | | | X | | | | | | | X | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 019 | | | | | | | | | | X | | | | | | | | X | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 020 | | | | | | | | | | | | | | X | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 021 | | X | X | | X | | X | | | | | | | X | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 022 | | X | | | | | X | | | X | | | X | | | | | | | | | X | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 024 | X | X | | | X | | X | | | | | | X | | | | | | | | | | X | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 025 | | X | X | | X | | X | | | | | | X | | | | | | | | | | | X |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 026 | | | | | | | | | | X | | | | | | | | | | X | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 027 | | | | | | | | | X | | | | | | | | | | | | | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
| 028 | | | | | | | | | | | | | X | | | | | | | | X | | | |
+-----+------+--------+--------+-------+-------+---------+---------+---------+---------+---------+----------+-------------+------+---------+-------+-------+-------+-------+-------+-------+-------+--------+------+----+
Definition of Terms:

1
doc/source/index.rst
View File

@ -19,6 +19,7 @@ Contents:
devmode
devmode-ovb
design
validations
contributing
working-with-extras
rhosp

70
doc/source/validations.rst Normal file
View File

@ -0,0 +1,70 @@
TripleO-Validations
===================
Introduction to TripleO-Validations
-----------------------------------
This is a collection of Ansible playbooks to detect and report potential issues
during TripleO deployments.
The validations will help to detect issues early in the deployment process and
prevent field engineers from wasting time on misconfiguration or hardware issues
in their environments.
All validations are written in Ansible and are written in a way that's consumable
by the `Mistral Validation framework` or by Ansible directly. They are available
independently from the UI or the command line client.
* Free software: Apache license
* Source: http://git.openstack.org/cgit/openstack/tripleo-validations
* Bugs: https://bugs.launchpad.net/tripleo/+bugs?field.tag=validations
Running TripleO-Validations using TripleO-Quickstart
----------------------------------------------------
TripleO-Quickstart allows you to run TripleO-Validations through the two ways of
execution (according to the introduction above). The first one is using the
`Mistral framework` and will run all the validations tests by group. The second
one is using `Ansible` directly and the goal is to run negative tests. Both are
launched through shell scripts and these scripts will be available in the
undercloud in the home directory of the unprivileged account created by
TripleO-Quickstart (by default the ``stack`` user).
Running Validations tests through Mistral
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To enable TripleO-Validations run using `Mistral`, you need to set the
``run_tripleo_validations`` variable to ``True``. By default this variable
is set to ``False``.
By Default, Tripleo-Quickstart won't fail when a validations test will fail.
If you want to disable this behaviour, you just need to set the
``exit_on_validations_failure`` to ``True``.
.. WARNING::
Note that for most of these validations, a failure does not mean that
you’ll be unable to deploy or run OpenStack. But it can indicate potential
issues with long-term or production setups. If you’re running an environment
for developing or testing TripleO, it’s okay that some validations fail.
In a production setup they should not.
To run them manually, you can run the ``run-tripleo-validations.sh``. This script
takes the name of validation group as an argument::
$ bash ./run-tripleo-validations.sh [pre-introspection|pre-deployment|post-deployment]
For more informations about each validations tests owning to these groups, you
can read:
- `pre-introspection group <https://docs.openstack.org/tripleo-validations/latest/validations-pre-introspection-details.html>`__
- `pre-deployment group <https://docs.openstack.org/tripleo-validations/latest/validations-pre-deployment-details.html>`__
- `post-deployment group <https://docs.openstack.org/tripleo-validations/latest/validations-post-deployment-details.html>`__
.. NOTE::
If you want to know more about running a single or a group of validations, please
take a look at the `<https://docs.openstack.org/tripleo-docs/latest/install/validations/validations.html>`__
Running Negative tests using Ansible
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To enable negative tests run using `Ansible`, you need to set the
``run_tripleo_validations_negative_tests`` variable to ``True``. By default
this variable is set to ``False``.

9
roles/common/defaults/main.yml
View File

@ -209,3 +209,12 @@ non_root_chown: false
# It is safe to mark this as default true as it only runs on a virthost
# This variable is set to true in config/environments/default_libvirt.yml
enable_port_forward_for_tripleo_ui: false
# This enables the run of several tripleo-validations tests through Mistral
run_tripleo_validations: False
# This enables the run of tripleo-validations negative tests through shell
# scripts
run_tripleo_validations_negative_tests: False
# Exit tripleo-quickstart on validations failures
exit_on_validations_failure: False