2018-01-02 15:14:37 +00:00
|
|
|
Whitebox Tempest plugin
|
|
|
|
=======================
|
2016-07-19 13:08:36 +02:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
This is a Tempest plugin for whitebox testing. While Tempest's scope is limited
|
|
|
|
to only the REST APIs, whitebox allows tests to peak behind the curtain,
|
|
|
|
similar to how a cloud admin might. Examining things on the compute host(s)
|
|
|
|
and/or the controller(s) is not only allowed, it's required for a test to be in
|
|
|
|
whitebox's scope. Whitebox tests must still be REST API-driven, however their
|
|
|
|
assertions can involve things like the instance XML (if the Nova libvirt driver
|
|
|
|
is in use) or the database.
|
2016-07-21 14:17:04 +02:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
* Bugs: https://storyboard.openstack.org/#!/project/1162
|
2021-05-31 18:44:25 +00:00
|
|
|
* IRC: #openstack-qa on OFTC
|
2018-01-11 15:05:40 +00:00
|
|
|
|
2018-01-02 15:14:37 +00:00
|
|
|
Requirements
|
|
|
|
------------
|
2016-07-21 14:17:04 +02:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
While Tempest is cloud-agnostic because all clouds expose the same OpenStack
|
|
|
|
APIs (with some caveats around extensions), whitebox peaks behind the curtain,
|
|
|
|
and thus is coupled to the way the cloud was deployed. Currently, devstack and
|
|
|
|
TripleO (with undercloud and overcloud) are supported, with only devstack being
|
|
|
|
tested in CI.
|
2016-07-25 10:15:06 -04:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
Some tests have specific hardware requirements. These should be documented as
|
|
|
|
config options, and tests are expected to skip if their hardware requirements
|
2022-05-12 16:29:14 +02:00
|
|
|
are not declared in the configuration.
|
2016-09-27 12:33:04 +02:00
|
|
|
|
2018-01-02 15:14:37 +00:00
|
|
|
Install, configure and run
|
|
|
|
--------------------------
|
2016-09-27 12:33:04 +02:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
0. Tempest needs to be installed and configured.
|
|
|
|
|
2018-01-02 15:14:37 +00:00
|
|
|
1. Install the plugin.
|
2016-07-21 14:17:04 +02:00
|
|
|
|
2018-01-02 15:14:37 +00:00
|
|
|
This should be done from source. ::
|
2016-09-27 12:33:04 +02:00
|
|
|
|
2024-05-10 09:18:56 -04:00
|
|
|
WORKSPACE=/some/directory
|
|
|
|
cd $WORKSPACE
|
|
|
|
git clone https://opendev.org/openstack/whitebox-tempest-plugin
|
2024-04-19 17:48:59 +01:00
|
|
|
|
2024-05-10 09:18:56 -04:00
|
|
|
sudo pip install whitebox-tempest-plugin
|
2016-09-27 12:33:04 +02:00
|
|
|
|
2018-01-02 15:14:37 +00:00
|
|
|
2. Configure Tempest.
|
2016-07-21 14:17:04 +02:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
The exact configuration will depend on the deployment. There is no
|
|
|
|
configuration reference yet, have a look at
|
|
|
|
``whitebox_tempest_plugin/config.py`` instead. As an example, here is a
|
2024-05-10 09:18:56 -04:00
|
|
|
configuration for a multinode TripleO deployment. ::
|
2024-04-19 17:48:59 +01:00
|
|
|
|
|
|
|
[whitebox]
|
|
|
|
ctlplane_addresses = compute-0.localdomain:192.168.24.6,compute-1.localdomain:192.168.24.12
|
|
|
|
ctlplane_ssh_username = heat-admin
|
|
|
|
ctlplane_ssh_private_key_path = /home/stack/.ssh/id_rsa
|
|
|
|
containers = true
|
|
|
|
max_compute_nodes = 2 # Some tests depend on there being a single (available) compute node
|
2018-01-02 15:14:37 +00:00
|
|
|
|
2024-10-09 21:52:15 +00:00
|
|
|
Here is an example for a two-node DevStack deployment:
|
|
|
|
|
|
|
|
.. code-block:: ini
|
|
|
|
|
|
|
|
[whitebox]
|
|
|
|
nodes_yaml = /opt/stack/whitebox-tempest-plugin/nodes.yaml
|
|
|
|
ctlplane_ssh_username = vagrant
|
|
|
|
ctlplane_ssh_private_key_path = /home/vagrant/.ssh/id_rsa
|
|
|
|
|
|
|
|
with a ``nodes.yaml`` file that looks something like:
|
|
|
|
|
|
|
|
.. code-block:: yaml
|
|
|
|
|
|
|
|
controller:
|
|
|
|
services:
|
|
|
|
libvirt:
|
|
|
|
start-command: 'systemctl start libvirtd'
|
|
|
|
stop_command: 'systemctl stop libvirtd'
|
|
|
|
nova-compute:
|
|
|
|
config_path: '/etc/nova/nova-cpu.conf'
|
|
|
|
start_command: 'systemctl start devstack@n-cpu'
|
|
|
|
stop_command: 'systemctl stop devstack@n-cpu'
|
|
|
|
compute1:
|
|
|
|
services:
|
|
|
|
libvirt:
|
|
|
|
start-command: 'systemctl start libvirtd'
|
|
|
|
stop_command: 'systemctl stop libvirtd'
|
|
|
|
nova-compute:
|
|
|
|
config_path: '/etc/nova/nova-cpu.conf'
|
|
|
|
start_command: 'systemctl start devstack@n-cpu'
|
|
|
|
stop_command: 'systemctl stop devstack@n-cpu'
|
|
|
|
|
|
|
|
where ``controller`` is the hostname of the controller node and
|
|
|
|
``compute1`` is the hostname of the second node running nova-compute.
|
|
|
|
|
2018-01-02 15:14:37 +00:00
|
|
|
3. Execute the tests. ::
|
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
tempest run --serial --regex whitebox_tempest_plugin.
|
2016-07-21 14:17:04 +02:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
.. important::
|
2017-12-13 15:06:41 +00:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
Whitebox expects its tests to run one at a time. Make sure to pass
|
2024-04-19 17:48:59 +01:00
|
|
|
``--serial`` or ``--concurrency 1`` to ``tempest run``.
|
2017-12-13 15:06:41 +00:00
|
|
|
|
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
How to add a new test
|
|
|
|
---------------------
|
2017-12-13 15:06:41 +00:00
|
|
|
|
2020-03-30 14:37:50 -04:00
|
|
|
Tests should fit whitebox's scope. If a test intereacts with REST APIs and
|
|
|
|
nothing else, it is better suited for Tempest itself. New tests should be added
|
|
|
|
in their respective subdirectories. For example, tests that use the compute API
|
|
|
|
live in ``whitebox_tempest_plugin/api/compute``. Test code does not need unit
|
|
|
|
tests, but helpers or utilities do. Unit tests live in
|
|
|
|
``whitebox_tempest_plugin/tests``. Whitebox does not adhere to the `Tempest
|
|
|
|
plugin interface <https://docs.openstack.org/tempest/latest/plugin.html>`. As
|
|
|
|
mentioned, whitebox tests run one at a time, so it's safe for a test to modify
|
|
|
|
the environment and/or be destructive, as long as it cleans up after itself.
|
|
|
|
For example, changing Nova configuration values and/or restarting services is
|
|
|
|
acceptable, as long as the original values and service state are restored.
|