openstack/whitebox-tempest-plugin
Code Issues Proposed changes
Tempest plugin for whitebox testing. For testing things not exposed through the REST APIs.
367 Commits 1 Branch 3 Tags 23 MiB
Python 98.3%
Shell 1%
Jinja 0.7%
71c9c3eeb7
Go to file
Artom Lifshitz 71c9c3eeb7 Swtich to Ubuntu Noble image 
Besides it being just a good thing to move to the latest LTS, this
also resolved an issue in different behaviour in libvirt versions for
watchdog devices. The q35 machine type always includes an implicit
watchdog device, but that device didn't start showing up in the XML
and having configurable actions until 9.1.0. On Jammy, the libvirt
version is 8.0.0, but in downstream RHOSO 18 CI on RHEL 9.4, the
version is 10.0.0. Moving to Noble unifies the libvirt versions, and
makes the watchdog testing less complicated.

Changes necessary to make this work:

* Start installing crudini with `package` instead of `pip` - new `pip`
  complains about the externally managed environment, and tells us to
  either use pipx to install in a venv, pass --break-system-packages,
  or install using system packages. Let's try the latter.

* Start running our local download job on changes to .zuul.yaml as
  well.

* Disable the br-ex-tcpdump service. It appears to only be useful for
  debugging, and it's crashing on the compute host with Noble. Just
  disable it for now.

Change-Id: I538f900953be2304baf8f63693699651fe763402
2024-06-29 14:04:35 -04:00
devstack Add realtime mask as parameter 2024-06-11 15:55:22 -04:00
playbooks Swtich to Ubuntu Noble image 2024-06-29 14:04:35 -04:00
tools add twine check and pre-commit 2024-04-19 19:02:33 +01:00
whitebox_tempest_plugin Merge "extend pre-commit with basic python linting" 2024-06-21 17:51:34 +00:00
.gitignore add twine check and pre-commit 2024-04-19 19:02:33 +01:00
.gitreview Fix repo after rename 2020-06-13 07:08:00 +02:00
.pre-commit-config.yaml extend pre-commit with basic python linting 2024-06-04 08:11:46 -04:00
.stestr.conf Subclass API tests instead of scenario 2018-07-13 10:38:09 -04:00
.zuul.yaml Swtich to Ubuntu Noble image 2024-06-29 14:04:35 -04:00
dictionary.txt extend pre-commit with basic python linting 2024-06-04 08:11:46 -04:00
LICENSE enable release jobs 2024-04-19 18:58:15 +01:00
README.rst enable release jobs 2024-04-19 18:58:15 +01:00
requirements.txt Reduce ddt requirement to >=1.6.0 2024-06-18 15:10:15 -04:00
setup.cfg enable release jobs 2024-04-19 18:58:15 +01:00
setup.py Initial commit 2017-12-12 17:55:00 +01:00
test-requirements.txt Remove tempest and oslo.log from requirements.txt 2020-07-30 12:09:17 -04:00
tox.ini Add global nodes variable 2023-11-13 13:39:53 -05:00

README.rst

Whitebox Tempest plugin

This is a Tempest plugin for whitebox testing. While Tempest's scope is limited to only the REST APIs, whitebox allows tests to peak behind the curtain, similar to how a cloud admin might. Examining things on the compute host(s) and/or the controller(s) is not only allowed, it's required for a test to be in whitebox's scope. Whitebox tests must still be REST API-driven, however their assertions can involve things like the instance XML (if the Nova libvirt driver is in use) or the database.

Requirements

While Tempest is cloud-agnostic because all clouds expose the same OpenStack APIs (with some caveats around extensions), whitebox peaks behind the curtain, and thus is coupled to the way the cloud was deployed. Currently, devstack and TripleO (with undercloud and overcloud) are supported, with only devstack being tested in CI.

Some tests have specific hardware requirements. These should be documented as config options, and tests are expected to skip if their hardware requirements are not declared in the configuration.

Install, configure and run

  1. Tempest needs to be installed and configured.

  2. Install the plugin.

    This should be done from source. :

    .. code-block:: shell

    WORKSPACE=/some/directory cd $WORKSPACE git clone https://opendev.org/openstack/whitebox-tempest-plugin sudo pip install whitebox-tempest-plugin

  3. Configure Tempest.

    The exact configuration will depend on the deployment. There is no configuration reference yet, have a look at whitebox_tempest_plugin/config.py instead. As an example, here is a configuration for a multinode TripleO deployment:

    .. code-block:: ini

    [whitebox] ctlplane_addresses = compute-0.localdomain:192.168.24.6,compute-1.localdomain:192.168.24.12 ctlplane_ssh_username = heat-admin ctlplane_ssh_private_key_path = /home/stack/.ssh/id_rsa containers = true max_compute_nodes = 2 # Some tests depend on there being a single (available) compute node

  4. Execute the tests. :

    tempest run --serial --regex whitebox_tempest_plugin.

    Important

    Whitebox expects its tests to run one at a time. Make sure to pass --serial or --concurrency 1 to tempest run.

How to add a new test

Tests should fit whitebox's scope. If a test intereacts with REST APIs and nothing else, it is better suited for Tempest itself. New tests should be added in their respective subdirectories. For example, tests that use the compute API live in whitebox_tempest_plugin/api/compute. Test code does not need unit tests, but helpers or utilities do. Unit tests live in whitebox_tempest_plugin/tests. Whitebox does not adhere to the Tempest plugin interface <https://docs.openstack.org/tempest/latest/plugin.html>. As mentioned, whitebox tests run one at a time, so it's safe for a test to modify the environment and/or be destructive, as long as it cleans up after itself. For example, changing Nova configuration values and/or restarting services is acceptable, as long as the original values and service state are restored.