RETIRED, A collection of tempest skipped tests shared across OpenStack projects.

Go to file

Zuul aebe91c375 Merge "Revert skiplist for #1968732 "		2022-08-03 15:11:54 +00:00
doc	Add 'all' wildcard for releases	2021-09-23 15:55:40 +02:00
playbooks	Support non-zuul workflow for list_skipped_yaml_file	2020-07-20 21:13:25 +02:00
releasenotes/source	Add releasenotes and py38 jobs	2020-10-23 13:09:01 +00:00
roles	Merge "Revert skiplist for #1968732 "	2022-08-03 15:11:54 +00:00
tempest_skip	Check if `installers` argument is provided	2022-07-07 10:52:01 -07:00
.coveragerc	Initial skeleton	2020-03-31 11:05:59 +02:00
.gitignore	Add tox-pep8 job and .gitignore	2020-03-31 09:48:39 +00:00
.gitreview	Added .gitreview and noop job	2020-03-30 17:39:49 +05:30
.zuul.yaml	Use python jobs template for Zed	2022-05-23 19:18:18 +05:30
README.rst	Add doc example using installers	2021-07-15 15:30:50 -04:00
requirements.txt	Changing ruamel.yaml version to >= instad of ==	2021-06-25 14:30:29 +02:00
setup.cfg	setup.cfg: Replace dashes by underscores	2022-02-06 02:23:24 +09:00
setup.py	Revert "Cleanup py27 support"	2020-06-05 18:28:33 +00:00
tempest_skip.yml	make adding a skip more user friendly	2021-06-04 10:27:02 -06:00
test-requirements.txt	Add addtest command to tempest-skiplist	2021-01-26 10:02:27 +01:00
tox.ini	Create tox environment to build pdf documentation	2020-10-25 16:40:43 +00:00

README.rst

openstack-tempest-skiplist

Overview

openstack-tempest-skiplist will generate a skip list to be executed by tempest

Free software: Apache license
Documentation: https://docs.openstack.org/openstack-tempest-skiplist/latest/
Source: https://opendev.org/openstack/openstack-tempest-skiplist
Bugs: https://storyboard.openstack.org/#!/project/1173
Release notes: https://docs.openstack.org/releasenotes/openstack-tempest-skiplist

Quickstart

edit tempest_skip.yml
add the required fields, including the launchpad
execute tox tests
submit :)

Installation

Installing from git and virtualenv:

$ git clone https://opendev.org/openstack/openstack-tempest-skiplist
$ cd openstack-tempest-skiplist
$ virtualenv .venv
$ source .venv/bin/activate
$ pip install .

Validation

After edit your file, you can check if it is valid with the following command:

$ tempest-skip validate --file file.yaml

Examples

A simple example of the structure of the yaml file expected by tempest-skiplist. For more information about each field, visit the documentation <https://docs.openstack.org/openstack-tempest-skiplist/latest/yaml/formatting.html> website:

known_failures:
- test: 'full.tempest.test'
    bz: 'https://bugzilla.redhat.com/1'
    lp: 'https://launchpad.net/bugs/1'
    deployment:
      - 'undercloud'
      - 'overcloud'
    jobs:
      - job1
      - job2
    reason: 'default reason'
    releases:
      - name: master
        lp: 'https://launchpad.net/bugs/2'
        reason: 'Some reason'
      - name: train
        bz: 'https://bugzilla.redhat.com/train1'
        installers:
          - 'osp'
      - name: ussuri
        installers:
          - 'tripleo'
        bz: 'https://bugzilla.redhat.com/ussuri1'

In the above example, the test full.tempest.test will be executed by tempest in both deployments, undercloud, and overcloud (depending on the job). It will also be skipped in the releases master, train and ussuri, specifically in jobs job1 and job2. It will not be skipped in any other job, no matter what the release is.

Removing the list of jobs, means it will be skipped everywhere.