openstack
/
openstack-tempest-skiplist
Code Issues Proposed changes
A collection of tempest skipped tests shared across OpenStack projects.
522 Commits
1 Branch
0 Tags
4.2 MiB
Python 100%
 
Go to file
Alan Pevec ec349ebef6 Revert "Add fs020-rbac to skiplist - BZ2211604" 
This reverts commit 30b34879ce.

Reason for revert: ovn22.12-22.12.0-94.el9fdp with the fix is in the latest compose

Change-Id: I5c96971efcb0e95b207ae514c376714ddbd193a9
 		2023-07-07 18:21:43 +02:00
doc Update docs for multiple groups with same name 2022-08-09 06:56:52 +00:00
openstack-operators Ignore api.compute.admin.test_networks.NetworksTest.test_list_all_networks 2023-06-20 18:36:56 +02:00
playbooks Support non-zuul workflow for list_skipped_yaml_file 2020-07-20 21:13:25 +02:00
releasenotes/source Add releasenotes and py38 jobs 2020-10-23 13:09:01 +00:00
roles Revert "Add fs020-rbac to skiplist - BZ2211604" 2023-07-07 18:21:43 +02:00
tempest_skip Fixes on tests and new lists for openstack-operators 2023-02-08 14:35:55 +01:00
.coveragerc Initial skeleton 2020-03-31 11:05:59 +02:00
.gitignore Add tox-pep8 job and .gitignore 2020-03-31 09:48:39 +00:00
.gitreview Added .gitreview and noop job 2020-03-30 17:39:49 +05:30
.zuul.yaml Use python jobs template for Zed 2022-05-23 19:18:18 +05:30
README.rst Add doc example using installers 2021-07-15 15:30:50 -04:00
requirements.txt Changing ruamel.yaml version to >= instad of == 2021-06-25 14:30:29 +02:00
setup.cfg Fixes on tests and new lists for openstack-operators 2023-02-08 14:35:55 +01:00
setup.py Revert "Cleanup py27 support" 2020-06-05 18:28:33 +00:00
tempest_skip.yml make adding a skip more user friendly 2021-06-04 10:27:02 -06:00
test-requirements.txt Add addtest command to tempest-skiplist 2021-01-26 10:02:27 +01:00
tox.ini Fixes on tests and new lists for openstack-operators 2023-02-08 14:35:55 +01:00

README.rst

openstack-tempest-skiplist

Overview

openstack-tempest-skiplist will generate a skip list to be executed by tempest

Quickstart

  • edit tempest_skip.yml
  • add the required fields, including the launchpad
  • execute tox tests
  • submit :)

Installation

Installing from git and virtualenv:

$ git clone https://opendev.org/openstack/openstack-tempest-skiplist
$ cd openstack-tempest-skiplist
$ virtualenv .venv
$ source .venv/bin/activate
$ pip install .

Validation

After edit your file, you can check if it is valid with the following command:

$ tempest-skip validate --file file.yaml

Examples

A simple example of the structure of the yaml file expected by tempest-skiplist. For more information about each field, visit the documentation <https://docs.openstack.org/openstack-tempest-skiplist/latest/yaml/formatting.html> website:

known_failures:
- test: 'full.tempest.test'
    bz: 'https://bugzilla.redhat.com/1'
    lp: 'https://launchpad.net/bugs/1'
    deployment:
      - 'undercloud'
      - 'overcloud'
    jobs:
      - job1
      - job2
    reason: 'default reason'
    releases:
      - name: master
        lp: 'https://launchpad.net/bugs/2'
        reason: 'Some reason'
      - name: train
        bz: 'https://bugzilla.redhat.com/train1'
        installers:
          - 'osp'
      - name: ussuri
        installers:
          - 'tripleo'
        bz: 'https://bugzilla.redhat.com/ussuri1'

In the above example, the test full.tempest.test will be executed by tempest in both deployments, undercloud, and overcloud (depending on the job). It will also be skipped in the releases master, train and ussuri, specifically in jobs job1 and job2. It will not be skipped in any other job, no matter what the release is.

Removing the list of jobs, means it will be skipped everywhere.