tempest/zuul.d/integrated-gate.yaml
Ghanshyam Mann 14f199eca2 Run cindet-tempest-plugin tests in tempest-extra-tests
We have a lot of Tempest public interfaces which are used by the
tempest plugins and sometime it is not easy to know breaking changes
as we do not test all plugins test when any change merge in Tempest.
Example: https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/937725

To avoid these breaking changes at some extend, let's run the cinder
tempest plugin tests in one of the Temp[est job. Evantually, we can
add more plugins tests if needed but I think cinder-tempest-plugin
uses a good amount of interfaces.

Change-Id: I80c2b77936ac3220ff05232877eaf6fbac4743a5
2024-12-14 02:09:43 +00:00

734 lines
27 KiB
YAML

# NOTE(gmann): This file includes all integrated jobs definition which
# are supposed to be run by Tempest and other projects as
# integrated testing.
- job:
name: tempest-all
parent: devstack-tempest
description: |
Integration test that runs all tests.
Former name for this job was:
* legacy-periodic-tempest-dsvm-all-master
timeout: 10800
vars:
tox_envlist: all
tempest_test_regex: tempest
devstack_localrc:
MYSQL_REDUCE_MEMORY: true
# TODO(gmann): Enable File injection tests once nova bug is fixed
# https://bugs.launchpad.net/nova/+bug/1882421
# ENABLE_FILE_INJECTION: true
run_tempest_cleanup: true
run_tempest_cleanup_resource_list: true
devstack_local_conf:
test-config:
$TEMPEST_CONFIG:
DEFAULT:
record_resources: true
- job:
name: tempest-ipv6-only
parent: devstack-tempest-ipv6
description: |
Integration test of IPv6-only deployments. This job runs
smoke and IPv6 relates tests only. Basic idea is to test
whether OpenStack Services listen on IPv6 address or not.
timeout: 10800
vars:
tox_envlist: ipv6-only
- job:
name: tempest-extra-tests
parent: tempest-full-py3
description: |
This job runs the extra tests mentioned in
tools/tempest-extra-tests-list.txt.
# NOTE(gmann): We need c-t-p as this job run c-t-p tests also.
required-projects:
- opendev.org/openstack/cinder-tempest-plugin
vars:
tox_envlist: extra-tests
tempest_plugins:
- cinder-tempest-plugin
run_tempest_cleanup: true
run_tempest_cleanup_resource_list: true
run_tempest_dry_cleanup: true
devstack_localrc:
CINDER_ENFORCE_SCOPE: true
devstack_local_conf:
test-config:
$TEMPEST_CONFIG:
DEFAULT:
record_resources: true
- job:
name: tempest-full-py3
parent: devstack-tempest
# This job version is with swift enabled on py3
# as swift is ready on py3 from stable/ussuri onwards.
# As this use 'integrated-full' tox env which is not
# available in old tempest used till stable/wallaby,
# this job definition is only for stable/xena onwards
# and separate job definition until stable/wallaby
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
# NOTE(sean-k-mooney): this job and its descendants frequently times out
# run on rax-* providers with a timeout of 2 hours. temporary increase
# the timeout to 2.5 hours.
timeout: 9000
description: |
Base integration test with Neutron networking, horizon, swift enable,
and py3.
Former names for this job where:
* legacy-tempest-dsvm-py35
* gate-tempest-dsvm-py35
required-projects:
- openstack/horizon
vars:
# NOTE(gmann): Default concurrency is higher (number of cpu -2) which
# end up 6 in upstream CI. Higher concurrency means high parallel
# requests to services and can cause more oom issues. To avoid the
# oom issue, setting the concurrency to 4 in this job.
# NOTE(sean-k-mooney): now that we use zswap we should be able to
# increase the concurrency to 6.
tempest_concurrency: 4
tox_envlist: integrated-full
devstack_localrc:
USE_PYTHON3: true
FORCE_CONFIG_DRIVE: true
ENABLE_VOLUME_MULTIATTACH: true
GLANCE_USE_IMPORT_WORKFLOW: True
devstack_plugins:
neutron: https://opendev.org/openstack/neutron
devstack_services:
# Enable horizon so that we can run horizon test.
horizon: true
- job:
name: tempest-full-centos-9-stream
parent: tempest-full-py3
nodeset: devstack-single-node-centos-9-stream
# centos-9-stream is supported from yoga release onwards
branches:
regex: ^.*/(victoria|wallaby|xena)$
negate: true
description: |
Base integration test on CentOS 9 stream
vars:
# Required until bug/1949606 is resolved when using libvirt and QEMU
# >=5.0.0 with a [libvirt]virt_type of qemu (TCG).
configure_swap_size: 4096
tox_envlist: full
- job:
name: tempest-integrated-networking
parent: devstack-tempest
description: |
This job runs integration tests for networking. This is subset of
'tempest-full-py3' job and run only Neutron and Nova related tests.
This is meant to be run on neutron gate only.
vars:
tox_envlist: integrated-network
devstack_localrc:
USE_PYTHON3: true
FORCE_CONFIG_DRIVE: true
devstack_services:
s-account: false
s-container: false
s-object: false
s-proxy: false
c-bak: false
- job:
name: tempest-integrated-compute
parent: devstack-tempest
description: |
This job runs integration tests for compute. This is
subset of 'tempest-full-py3' job and run Nova, Neutron, Cinder (except backup tests)
and Glance related tests. This is meant to be run on Nova gate only.
# NOTE(sean-k-mooney): this job and its descendants frequently times out
# when run on rax-* providers, recent optimizations have reduced the
# runtime of the job but it still times out. temporary increase the
# timeout to 2.5 hours.
timeout: 9000
vars:
# NOTE(gmann): Default concurrency is higher (number of cpu -2) which
# end up 6 in upstream CI. Higher concurrency means high parallel
# requests to services and can cause more oom issues. To avoid the
# oom issue, setting the concurrency to 4 in this job.
# NOTE(sean-k-mooney): now that we use zswap we should be able to
# increase the concurrency to 6.
tempest_concurrency: 4
tox_envlist: integrated-compute
tempest_exclude_regex: ""
devstack_localrc:
USE_PYTHON3: true
FORCE_CONFIG_DRIVE: true
ENABLE_VOLUME_MULTIATTACH: true
devstack_services:
s-account: false
s-container: false
s-object: false
s-proxy: false
c-bak: false
- job:
name: tempest-integrated-compute-centos-9-stream
parent: tempest-integrated-compute
nodeset: devstack-single-node-centos-9-stream
# centos-9-stream is supported from yoga release onwards
branches:
regex: ^.*/(victoria|wallaby|xena)$
negate: true
description: |
This job runs integration tests for compute. This is
subset of 'tempest-full-py3' job and run Nova, Neutron, Cinder (except backup tests)
and Glance related tests. This is meant to be run on Nova gate only.
This version of the job also uses CentOS 9 stream.
vars:
# Required until bug/1949606 is resolved when using libvirt and QEMU
# >=5.0.0 with a [libvirt]virt_type of qemu (TCG).
configure_swap_size: 4096
- job:
name: tempest-integrated-placement
parent: devstack-tempest
description: |
This job runs integration tests for placement. This is
subset of 'tempest-full-py3' job and run Nova and Neutron
related tests. This is meant to be run on Placement gate only.
vars:
# NOTE(gmann): Default concurrency is higher (number of cpu -2) which
# end up 6 in upstream CI. Higher concurrency means high parallel
# requests to services and can cause more oom issues. To avoid the
# oom issue, setting the concurrency to 4 in this job.
tempest_concurrency: 4
tox_envlist: integrated-placement
devstack_localrc:
USE_PYTHON3: true
FORCE_CONFIG_DRIVE: true
ENABLE_VOLUME_MULTIATTACH: true
devstack_services:
s-account: false
s-container: false
s-object: false
s-proxy: false
c-bak: false
- job:
name: tempest-integrated-storage
parent: devstack-tempest
description: |
This job runs integration tests for image & block storage. This is
subset of 'tempest-full-py3' job and run Cinder, Glance, Swift and Nova
related tests. This is meant to be run on Cinder and Glance gate only.
vars:
tox_envlist: integrated-storage
devstack_localrc:
USE_PYTHON3: true
FORCE_CONFIG_DRIVE: true
ENABLE_VOLUME_MULTIATTACH: true
GLANCE_USE_IMPORT_WORKFLOW: True
- job:
name: tempest-integrated-object-storage
parent: devstack-tempest
description: |
This job runs integration tests for object storage. This is
subset of 'tempest-full-py3' job and run Swift, Cinder and Glance
related tests. This is meant to be run on Swift gate only.
vars:
tox_envlist: integrated-object-storage
devstack_localrc:
# NOTE(gmann): swift is not ready on python3 yet and devstack
# install it on python2.7 only. But setting the USE_PYTHON3
# for future once swift is ready on py3.
USE_PYTHON3: true
- job:
name: tempest-with-latest-microversion
parent: tempest-full-py3
description: |
This job runs compute, placement and volume API tests with 'latest'
API microversion (This can be extended to other services having API
microversion concept).
vars:
tox_envlist: api-microversion-tests
devstack_localrc:
TEMPEST_COMPUTE_MIN_MICROVERSION: 'latest'
TEMPEST_VOLUME_MIN_MICROVERSION: 'latest'
TEMPEST_PLACEMENT_MIN_MICROVERSION: 'latest'
- job:
name: tempest-multinode-full-py3
parent: tempest-multinode-full-base
nodeset: openstack-two-node-noble
# This job runs on ubuntu Noble from 2025.1 onwards.
branches:
regex: ^.*/(victoria|wallaby|xena|yoga|zed|2023.1|2023.2|2024.1|2024.2)$
negate: true
vars:
# NOTE(gmann): Default concurrency is higher (number of cpu -2) which
# end up 6 in upstream CI. Higher concurrency means high parallel
# requests to services and can cause more oom issues. To avoid the
# oom issue, setting the concurrency to 4 in this job.
tempest_concurrency: 4
tempest_set_src_dest_host: true
devstack_plugins:
neutron: https://opendev.org/openstack/neutron
devstack_services:
neutron-trunk: true
br-ex-tcpdump: true
br-int-flows: true
group-vars:
subnode:
devstack_services:
br-ex-tcpdump: true
br-int-flows: true
- job:
name: tempest-slow
parent: tempest-multinode-full
description: |
This multinode integration job will run all the tests tagged as slow.
It enables the lvm multibackend setup to cover few scenario tests.
This job will run only slow tests (API or Scenario) serially.
Former names for this job were:
* legacy-tempest-dsvm-neutron-scenario-multinode-lvm-multibackend
* tempest-scenario-multinode-lvm-multibackend
timeout: 10800
vars:
tox_envlist: slow-serial
devstack_localrc:
CINDER_ENABLED_BACKENDS: lvm:lvmdriver-1,lvm:lvmdriver-2
ENABLE_VOLUME_MULTIATTACH: true
devstack_plugins:
neutron: https://opendev.org/openstack/neutron
devstack_services:
neutron-placement: true
neutron-qos: true
group-vars:
# NOTE(mriedem): The ENABLE_VOLUME_MULTIATTACH variable is used on both
# the controller and subnode prior to Rocky so we have to make sure the
# variable is set in both locations.
subnode:
devstack_localrc:
ENABLE_VOLUME_MULTIATTACH: true
- job:
name: tempest-slow-py3
parent: tempest-multinode-full-py3
# This job version is with swift enabled on py3
# as swift is ready on py3 from stable/ussuri onwards.
timeout: 10800
# As the 'slow' tox env which is not available in old tempest used
# till stable/wallaby, this job definition is only for stable/xena
# onwards and separate job definition until stable/wallaby
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
vars:
tox_envlist: slow
devstack_localrc:
CINDER_ENABLED_BACKENDS: lvm:lvmdriver-1,lvm:lvmdriver-2
ENABLE_VOLUME_MULTIATTACH: true
GLANCE_ENFORCE_IMAGE_FORMAT: false
devstack_plugins:
neutron: https://opendev.org/openstack/neutron
devstack_services:
neutron-placement: true
neutron-qos: true
group-vars:
# NOTE(mriedem): The ENABLE_VOLUME_MULTIATTACH variable is used on both
# the controller and subnode prior to Rocky so we have to make sure the
# variable is set in both locations.
subnode:
devstack_localrc:
ENABLE_VOLUME_MULTIATTACH: true
# TODO(gmann): As per the 2025.1 testing runtime, we need to run at least
# one job set on Jammy. These jammy job can be removed in the next
# cycle(2025.2).
- job:
name: tempest-full-ubuntu-jammy
description: This is tempest-full python3 job on Ubuntu Jammy(22.04)
parent: tempest-full-py3
nodeset: openstack-single-node-jammy
- job:
name: tempest-multinode-full-ubuntu-jammy
description: This is tempest-multinode-full-py3 python3 job on Ubuntu Jammy(22.04)
parent: tempest-multinode-full-py3
nodeset: openstack-two-node-jammy
- job:
name: tempest-extra-tests-ubuntu-jammy
description: This is tempest-extra-tests python3 job on Ubuntu Jammy(22.04)
parent: tempest-extra-tests
nodeset: openstack-single-node-jammy
- job:
name: tempest-cinder-v2-api
parent: devstack-tempest
# NOTE(gmann): Cinder v2 APIs are available until
# stable/wallaby only.
override-checkout: stable/wallaby
description: |
This job runs the cinder API test against v2 endpoint.
vars:
tox_envlist: all
tempest_test_regex: api.*volume
devstack_localrc:
TEMPEST_VOLUME_TYPE: volumev2
- job:
name: tempest-centos9-stream-fips
parent: devstack-tempest
description: |
Integration testing for a FIPS enabled Centos 9 system
timeout: 10800
nodeset: devstack-single-node-centos-9-stream
vars:
tox_envlist: full
configure_swap_size: 4096
nslookup_target: 'opendev.org'
enable_fips: True
- job:
name: tempest-pg-full
parent: tempest-full-py3
description: |
Base integration test with Neutron networking and PostgreSQL.
Former name for this job was legacy-tempest-dsvm-neutron-pg-full.
vars:
devstack_localrc:
# TODO(gmann): Enable File injection tests once nova bug is fixed
# https://bugs.launchpad.net/nova/+bug/1882421
# ENABLE_FILE_INJECTION: true
DATABASE_TYPE: postgresql
- job:
name: tempest-full-enforce-scope-new-defaults
parent: tempest-full-py3
description: |
This job runs the Tempest tests with scope and new defaults enabled.
vars:
devstack_localrc:
KEYSTONE_ENFORCE_SCOPE: true
CINDER_ENFORCE_SCOPE: true
PLACEMENT_ENFORCE_SCOPE: true
- job:
name: tempest-all-rbac-old-defaults
parent: tempest-all
description: |
Integration test that runs all tests on RBAC old defaults.
vars:
devstack_localrc:
# NOTE(gmann): Nova, Glance, and Neutron have enabled the new defaults
# and scope by default in devstack so we need some jobs keep testing
# the old defaults until they are removed from service side.
NOVA_ENFORCE_SCOPE: false
GLANCE_ENFORCE_SCOPE: false
NEUTRON_ENFORCE_SCOPE: false
- project-template:
name: integrated-gate-networking
description: |
Run the python3 Tempest network integration tests (Nova and Neutron related)
in check and gate for the neutron integrated gate. This is meant to be
run on neutron gate only.
check:
jobs:
- grenade
# NOTE(gmann): These template are generic and used on stable branch
# as well as master testing. So grenade-skip-level on stable/2023.1
# which test stable/yoga to stable/2023.1 upgrade is non-voting.
- grenade-skip-level:
voting: false
branches:
- stable/2023.1
# on stable/2024.1(SLURP) grenade-skip-level is voting which test
# stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
# SLURP release only.
- grenade-skip-level:
branches:
- ^.*/2024.1
# on current master 2025.1(SLURP) grenade-skip-level-always is voting
# which test stable/2024.1 to 2025.1 upgrade.
- grenade-skip-level-always:
branches:
- master
- tempest-integrated-networking
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade
- tempest-integrated-networking
# on stable/2024.1(SLURP) grenade-skip-level is voting which test
# stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
# SLURP release only.
- grenade-skip-level:
branches:
- ^.*/2024.1
# on current master 2025.1(SLURP) grenade-skip-level-always is voting
# which test stable/2024.1 to 2025.1 upgrade.
- grenade-skip-level-always:
branches:
- master
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
- project-template:
name: integrated-gate-compute
description: |
Run the python3 Tempest compute integration tests
(Nova, Neutron, Cinder and Glance related) in check and gate
for the Nova integrated gate. This is meant to be
run on Nova gate only.
# NOTE(gmann): This template is used for stable branches also so when we
# add/remove jobs here we need to make sure we should not change the
# behaviour for stable branches. For example, with branch variant we need
# to make sure old job keep running on stable branches and the new one run
# only from master(or the branch it was meant to run).
check:
jobs:
# NOTE(gmann): These template are generic and used on stable branch
# as well as master testing. So grenade-skip-level on stable/2023.1
# which test stable/yoga to stable/2023.1 upgrade is non-voting.
- grenade-skip-level:
voting: false
branches:
- stable/2023.1
# NOTE(gmann): Nova decided to run grenade skip level testing always
# (on SLURP as well as non SLURP release) so we are adding grenade-skip-level-always
# job in integrated gate and we do not need to update skip level job
# here until Nova change the decision.
# This is added from 2023.2 release cycle onwards so we need to use branch variant
# to make sure we do not run this job on older than 2023.2 gate.
- grenade-skip-level-always:
branches:
- ^.*/2023.2
- ^.*/2024.1
- ^.*/2024.2
- master
- tempest-integrated-compute
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade-skip-level-always:
branches:
- ^.*/2023.2
- ^.*/2024.1
- ^.*/2024.2
- master
- tempest-integrated-compute
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
periodic-weekly:
jobs:
# centos-9-stream is tested from zed release onwards
- tempest-integrated-compute-centos-9-stream:
branches:
regex: ^.*/(victoria|wallaby|xena|yoga)$
negate: true
- project-template:
name: integrated-gate-placement
description: |
Run the python3 Tempest placement integration tests
(Nova and Neutron related) in check and gate
for the Placement integrated gate. This is meant to be
run on Placement gate only.
check:
jobs:
- grenade
# NOTE(gmann): These template are generic and used on stable branch
# as well as master testing. So grenade-skip-level on stable/2023.1
# which test stable/yoga to stable/2023.1 upgrade is non-voting.
- grenade-skip-level:
voting: false
branches:
- stable/2023.1
# on stable/2024.1(SLURP) grenade-skip-level is voting which test
# stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
# SLURP release only.
- grenade-skip-level:
branches:
- ^.*/2024.1
# on current master 2025.1(SLURP) grenade-skip-level-always is voting
# which test stable/2024.1 to 2025.1 upgrade.
- grenade-skip-level-always:
branches:
- master
- tempest-integrated-placement
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade
- tempest-integrated-placement
# on stable/2024.1(SLURP) grenade-skip-level is voting which test
# stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
# SLURP release only.
- grenade-skip-level:
branches:
- ^.*/2024.1
# on current master 2025.1(SLURP) grenade-skip-level-always is voting
# which test stable/2024.1 to 2025.1 upgrade.
- grenade-skip-level-always:
branches:
- master
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
- project-template:
name: integrated-gate-storage
description: |
Run the python3 Tempest image & block storage integration tests
(Cinder, Glance, Swift and Nova related) in check and gate
for the neutron integrated gate. This is meant to be
run on Cinder and Glance gate only.
check:
jobs:
- grenade
# NOTE(gmann): These template are generic and used on stable branch
# as well as master testing. So grenade-skip-level on stable/2023.1
# which test stable/yoga to stable/2023.1 upgrade is non-voting.
- grenade-skip-level:
voting: false
branches:
- stable/2023.1
# on stable/2024.1(SLURP) grenade-skip-level is voting which test
# stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
# SLURP release only.
- grenade-skip-level:
branches:
- ^.*/2024.1
# on current master 2025.1(SLURP) grenade-skip-level-always is voting
# which test stable/2024.1 to 2025.1 upgrade.
- grenade-skip-level-always:
branches:
- master
- tempest-integrated-storage
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade
# on stable/2024.1(SLURP) grenade-skip-level is voting which test
# stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
# SLURP release only.
- grenade-skip-level:
branches:
- ^.*/2024.1
# on current master 2025.1(SLURP) grenade-skip-level-always is voting
# which test stable/2024.1 to 2025.1 upgrade.
- grenade-skip-level-always:
branches:
- master
- tempest-integrated-storage
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
- project-template:
name: integrated-gate-object-storage
description: |
Run the python3 Tempest object storage integration tests
(Swift, Cinder and Glance related) in check and gate
for the swift integrated gate. This is meant to be
run on swift gate only.
check:
jobs:
- grenade
# on stable/2024.1(SLURP) grenade-skip-level is voting which test
# stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
# SLURP release only.
- grenade-skip-level:
branches:
- ^.*/2024.1
# on current master 2025.1(SLURP) grenade-skip-level-always is voting
# which test stable/2024.1 to 2025.1 upgrade.
- grenade-skip-level-always:
branches:
- master
- tempest-integrated-object-storage
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true
gate:
jobs:
- grenade
# on stable/2024.1(SLURP) grenade-skip-level is voting which test
# stable/2023.1 to stable/2024.1 upgrade. This is supposed to run on
# SLURP release only.
- grenade-skip-level:
branches:
- ^.*/2024.1
# on current master 2025.1(SLURP) grenade-skip-level-always is voting
# which test stable/2024.1 to 2025.1 upgrade.
- grenade-skip-level-always:
branches:
- master
- tempest-integrated-object-storage
# Do not run it on ussuri until below issue is fixed
# https://storyboard.openstack.org/#!/story/2010057
# and job is broken up to wallaby branch due to the issue
# described in https://review.opendev.org/872341
- openstacksdk-functional-devstack:
branches:
regex: ^.*/(victoria|wallaby)$
negate: true