A collection of tempest skipped tests shared across OpenStack projects.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Douglas Viroel 1f2fdc5056 Revert "Add FloatingIpMultipleRoutersTest to skiplist" 4 days ago
doc Add 'all' wildcard for releases 1 month ago
playbooks Support non-zuul workflow for list_skipped_yaml_file 1 year ago
releasenotes/source Add releasenotes and py38 jobs 1 year ago
roles Revert "Add FloatingIpMultipleRoutersTest to skiplist" 4 days ago
tempest_skip Add 'all' wildcard for releases 1 month ago
.coveragerc Initial skeleton 2 years ago
.gitignore Add tox-pep8 job and .gitignore 2 years ago
.gitreview Added .gitreview and noop job 2 years ago
.zuul.yaml Add releasenotes and py38 jobs 1 year ago
README.rst Add doc example using installers 3 months ago
requirements.txt Changing ruamel.yaml version to >= instad of == 4 months ago
setup.cfg Allowed tests command added 6 months ago
setup.py Revert "Cleanup py27 support" 1 year ago
tempest_skip.yml make adding a skip more user friendly 5 months ago
test-requirements.txt Add addtest command to tempest-skiplist 9 months ago
tox.ini Create tox environment to build pdf documentation 1 year ago

README.rst

openstack-tempest-skiplist

Overview

openstack-tempest-skiplist will generate a skip list to be executed by tempest

Quickstart

  • edit tempest_skip.yml
  • add the required fields, including the launchpad
  • execute tox tests
  • submit :)

Installation

Installing from git and virtualenv:

$ git clone https://opendev.org/openstack/openstack-tempest-skiplist
$ cd openstack-tempest-skiplist
$ virtualenv .venv
$ source .venv/bin/activate
$ pip install .

Validation

After edit your file, you can check if it is valid with the following command:

$ tempest-skip validate --file file.yaml

Examples

A simple example of the structure of the yaml file expected by tempest-skiplist. For more information about each field, visit the documentation <https://docs.openstack.org/openstack-tempest-skiplist/latest/yaml/formatting.html> website:

known_failures:
- test: 'full.tempest.test'
    bz: 'https://bugzilla.redhat.com/1'
    lp: 'https://launchpad.net/bugs/1'
    deployment:
      - 'undercloud'
      - 'overcloud'
    jobs:
      - job1
      - job2
    reason: 'default reason'
    releases:
      - name: master
        lp: 'https://launchpad.net/bugs/2'
        reason: 'Some reason'
      - name: train
        bz: 'https://bugzilla.redhat.com/train1'
        installers:
          - 'osp'
      - name: ussuri
        installers:
          - 'tripleo'
        bz: 'https://bugzilla.redhat.com/ussuri1'

In the above example, the test full.tempest.test will be executed by tempest in both deployments, undercloud, and overcloud (depending on the job). It will also be skipped in the releases master, train and ussuri, specifically in jobs job1 and job2. It will not be skipped in any other job, no matter what the release is.

Removing the list of jobs, means it will be skipped everywhere.