openstack
/
openstack-tempest-skiplist
Code Issues Proposed changes
RETIRED, A collection of tempest skipped tests shared across OpenStack projects.
287 Commits 1 Branch 0 Tags 4.2 MiB
Go to file
Arx Cruz 85a392c55f Add installer key to releases on skiplist 
Due the fact that one release might have different installers and
recently we were facing problems where add a test on the skiplist in
upstream jobs was also affecting downstream jobs, the best solution
right now is to add the key installers on the release for the test.
The default remains being 'tripleo' and 'osp', so we don't need to edit
all the tests to add the installers option. However, if you explicitly
set the installer, and you want it to be skipped in both tripleo and
other, you must also add the tripleo to the list.

Change-Id: I709a826f75e64ab2a7f741891e0a3cc4aa6edf32
 		2021-07-05 12:10:32 +02:00
doc Add installer key to releases on skiplist 2021-07-05 12:10:32 +02:00
playbooks Support non-zuul workflow for list_skipped_yaml_file 2020-07-20 21:13:25 +02:00
releasenotes/source Add releasenotes and py38 jobs 2020-10-23 13:09:01 +00:00
roles Add installer key to releases on skiplist 2021-07-05 12:10:32 +02:00
tempest_skip Add installer key to releases on skiplist 2021-07-05 12:10:32 +02:00
.coveragerc Initial skeleton 2020-03-31 11:05:59 +02:00
.gitignore Add tox-pep8 job and .gitignore 2020-03-31 09:48:39 +00:00
.gitreview Added .gitreview and noop job 2020-03-30 17:39:49 +05:30
.zuul.yaml Add releasenotes and py38 jobs 2020-10-23 13:09:01 +00:00
README.rst make adding a skip more user friendly 2021-06-04 10:27:02 -06:00
requirements.txt Changing ruamel.yaml version to >= instad of == 2021-06-25 14:30:29 +02:00
setup.cfg Allowed tests command added 2021-04-14 14:08:52 +02:00
setup.py Revert "Cleanup py27 support" 2020-06-05 18:28:33 +00:00
tempest_skip.yml make adding a skip more user friendly 2021-06-04 10:27:02 -06:00
test-requirements.txt Add addtest command to tempest-skiplist 2021-01-26 10:02:27 +01:00
tox.ini Create tox environment to build pdf documentation 2020-10-25 16:40:43 +00:00

README.rst

openstack-tempest-skiplist

Overview

openstack-tempest-skiplist will generate a skip list to be executed by tempest

Quickstart

  • edit tempest_skip.yml
  • add the required fields, including the launchpad
  • execute tox tests
  • submit :)

Installation

Installing from git and virtualenv:

$ git clone https://opendev.org/openstack/openstack-tempest-skiplist
$ cd openstack-tempest-skiplist
$ virtualenv .venv
$ source .venv/bin/activate
$ pip install .

Validation

After edit your file, you can check if it is valid with the following command:

$ tempest-skip validate --file file.yaml

Examples

A simple example of the structure of the yaml file expected by tempest-skiplist. For more information about each field, visit the documentation <https://docs.openstack.org/openstack-tempest-skiplist/latest/yaml/formatting.html> website:

known_failures:
- test: 'full.tempest.test'
    bz: 'https://bugzilla.redhat.com/1'
    lp: 'https://launchpad.net/bugs/1'
    deployment:
      - 'undercloud'
      - 'overcloud'
    jobs:
      - job1
      - job2
    reason: 'default reason'
    releases:
      - name: master
        lp: 'https://launchpad.net/bugs/2'
        reason: 'Some reason'
      - name: train
        bz: 'https://bugzilla.redhat.com/train1'
      - name: ussuri
        bz: 'https://bugzilla.redhat.com/ussuri1'

In the above example, the test full.tempest.test will be executed by tempest in both deployments, undercloud, and overcloud (depending on the job). It will also be skipped in the releases master, train and ussuri, specifically in jobs job1 and job2. It will not be skipped in any other job, no matter what the release is.

Removing the list of jobs, means it will be skipped everywhere.