openstack
/
openstack-tempest-skiplist
Code Issues Proposed changes
RETIRED, A collection of tempest skipped tests shared across OpenStack projects.
198 Commits 1 Branch 0 Tags 4.2 MiB
Go to file
Brent Eagles ecb8c966af Re-enable skipped octavia test for scenario 10 
The octavia "smoke" test is skipped because of
https://bugs.launchpad.net/tripleo/+bug/1883687. The general Octavia
tests are rather extensive so this patch enables the smoke test and
relies on the job definitions to limit the list of tests to a subset.

Change-Id: I0436ab3b49ff1738832cee1579ce120c8cc4f00a
 		2021-01-25 09:19:08 -03:30
doc Create tox environment to build pdf documentation 2020-10-25 16:40:43 +00:00
playbooks Support non-zuul workflow for list_skipped_yaml_file 2020-07-20 21:13:25 +02:00
releasenotes/source Add releasenotes and py38 jobs 2020-10-23 13:09:01 +00:00
roles Re-enable skipped octavia test for scenario 10 2021-01-25 09:19:08 -03:30
tempest_skip Merge "Make releases field required" 2020-10-25 12:00:47 +00:00
.coveragerc Initial skeleton 2020-03-31 11:05:59 +02:00
.gitignore Add tox-pep8 job and .gitignore 2020-03-31 09:48:39 +00:00
.gitreview Added .gitreview and noop job 2020-03-30 17:39:49 +05:30
.zuul.yaml Add releasenotes and py38 jobs 2020-10-23 13:09:01 +00:00
README.rst Update README with quickstart 2020-11-18 13:23:57 +01:00
requirements.txt Create validate command 2020-04-08 14:13:38 +02:00
setup.cfg Add releasenotes and py38 jobs 2020-10-23 13:09:01 +00:00
setup.py Revert "Cleanup py27 support" 2020-06-05 18:28:33 +00:00
test-requirements.txt Fix the filter with --job option 2020-09-14 11:57:32 +02:00
tox.ini Create tox environment to build pdf documentation 2020-10-25 16:40:43 +00:00

README.rst

openstack-tempest-skiplist

Overview

openstack-tempest-skiplist will generate a skip list to be executed by tempest

Quickstart

Installation

Installing from git and virtualenv:

$ git clone https://opendev.org/openstack/openstack-tempest-skiplist
$ cd openstack-tempest-skiplist
$ virtualenv .venv
$ source .venv/bin/activate
$ pip install .

Validation

After edit your file, you can check if it is valid with the following command:

$ tempest-skiplist validate --file file.yaml

Examples

A simple example of the structure of the yaml file expected by tempest-skiplist. For more information about each field, visit the documentation <https://docs.openstack.org/openstack-tempest-skiplist/latest/yaml/formatting.html> website:

known_failures:
- test: 'full.tempest.test'
    bz: 'https://bugzilla.redhat.com/1'
    lp: 'https://launchpad.net/bugs/1'
    deployment:
      - 'undercloud'
      - 'overcloud'
    jobs:
      - job1
      - job2
    reason: 'default reason'
    releases:
      - name: master
        lp: 'https://launchpad.net/bugs/2'
        reason: 'Some reason'
      - name: train
        bz: 'https://bugzilla.redhat.com/train1'
      - name: ussuri
        bz: 'https://bugzilla.redhat.com/ussuri1'

In the above example, the test full.tempest.test will be executed by tempest in both deployments, undercloud, and overcloud (depending on the job). It will also be skipped in the releases master, train and ussuri, specifically in jobs job1 and job2. It will not be skipped in any other job, no matter what the release is.

Removing the list of jobs, means it will be skipped everywhere.