RETIRED, A collection of tempest skipped tests shared across OpenStack projects.

Go to file

Ronelle Landy 586c407637 Revert "Remove skip entries for bug 923194" This reverts commit `a675090161`. Reason for revert: test is still failing Change-Id: Ic9de7b65c9c574013c2190a9d1dc2c99ba6c6663		2021-11-30 01:28:32 +00:00
doc	Add 'all' wildcard for releases	2021-09-23 15:55:40 +02:00
playbooks	Support non-zuul workflow for list_skipped_yaml_file	2020-07-20 21:13:25 +02:00
releasenotes/source	Add releasenotes and py38 jobs	2020-10-23 13:09:01 +00:00
roles	Revert "Remove skip entries for bug 923194"	2021-11-30 01:28:32 +00:00
tempest_skip	Add 'all' wildcard for releases	2021-09-23 15:55:40 +02:00
.coveragerc	Initial skeleton	2020-03-31 11:05:59 +02:00
.gitignore	Add tox-pep8 job and .gitignore	2020-03-31 09:48:39 +00:00
.gitreview	Added .gitreview and noop job	2020-03-30 17:39:49 +05:30
.zuul.yaml	Add releasenotes and py38 jobs	2020-10-23 13:09:01 +00:00
README.rst	Add doc example using installers	2021-07-15 15:30:50 -04:00
requirements.txt	Changing ruamel.yaml version to >= instad of ==	2021-06-25 14:30:29 +02:00
setup.cfg	Allowed tests command added	2021-04-14 14:08:52 +02:00
setup.py	Revert "Cleanup py27 support"	2020-06-05 18:28:33 +00:00
tempest_skip.yml	make adding a skip more user friendly	2021-06-04 10:27:02 -06:00
test-requirements.txt	Add addtest command to tempest-skiplist	2021-01-26 10:02:27 +01:00
tox.ini	Create tox environment to build pdf documentation	2020-10-25 16:40:43 +00:00

README.rst

openstack-tempest-skiplist

Overview

openstack-tempest-skiplist will generate a skip list to be executed by tempest

Free software: Apache license
Documentation: https://docs.openstack.org/openstack-tempest-skiplist/latest/
Source: https://opendev.org/openstack/openstack-tempest-skiplist
Bugs: https://storyboard.openstack.org/#!/project/1173
Release notes: https://docs.openstack.org/releasenotes/openstack-tempest-skiplist

Quickstart

edit tempest_skip.yml
add the required fields, including the launchpad
execute tox tests
submit :)

Installation

Installing from git and virtualenv:

$ git clone https://opendev.org/openstack/openstack-tempest-skiplist
$ cd openstack-tempest-skiplist
$ virtualenv .venv
$ source .venv/bin/activate
$ pip install .

Validation

After edit your file, you can check if it is valid with the following command:

$ tempest-skip validate --file file.yaml

Examples

A simple example of the structure of the yaml file expected by tempest-skiplist. For more information about each field, visit the documentation <https://docs.openstack.org/openstack-tempest-skiplist/latest/yaml/formatting.html> website:

known_failures:
- test: 'full.tempest.test'
    bz: 'https://bugzilla.redhat.com/1'
    lp: 'https://launchpad.net/bugs/1'
    deployment:
      - 'undercloud'
      - 'overcloud'
    jobs:
      - job1
      - job2
    reason: 'default reason'
    releases:
      - name: master
        lp: 'https://launchpad.net/bugs/2'
        reason: 'Some reason'
      - name: train
        bz: 'https://bugzilla.redhat.com/train1'
        installers:
          - 'osp'
      - name: ussuri
        installers:
          - 'tripleo'
        bz: 'https://bugzilla.redhat.com/ussuri1'

In the above example, the test full.tempest.test will be executed by tempest in both deployments, undercloud, and overcloud (depending on the job). It will also be skipped in the releases master, train and ussuri, specifically in jobs job1 and job2. It will not be skipped in any other job, no matter what the release is.

Removing the list of jobs, means it will be skipped everywhere.