tacker/.zuul.yaml

879 lines
31 KiB
YAML
Raw Normal View History

- nodeset:
name: openstack-4-nodes-jammy
nodes:
- name: controller
label: ubuntu-jammy
- name: controller-tacker
label: ubuntu-jammy
- name: compute1
label: ubuntu-jammy
- name: compute2
label: ubuntu-jammy
groups:
# Nodes running the compute service
- name: compute
nodes:
- compute1
- compute2
# Nodes that are not the controller
- name: subnode
nodes:
- controller-tacker
- compute1
- compute2
# Switch node for multinode networking setup
- name: switch
nodes:
- controller
# Peer nodes for multinode networking setup
- name: peers
nodes:
- controller-tacker
- compute1
- compute2
- nodeset:
Keep new RBAC disable by default & Gate fix for GLOBAL_VENV This commit squash the multiple changes: Changes 1: Keep new RBAC disable by default oslo.policy has enabled the new RBAC config options enforce_scope and enforce_new_defaults by default[1][2]. Tacker implemented the new RBAC in previous cycle with new RBAC disable by default. To give more time to operator, let's continue the same setting in this release also. Also, there are many test modification is needed for the new RBAC (using the new RBAC default role in tests) Unit tests log a lot of policy warnings, suppressing those[3] -https://fa3204066787dd37fd86-ea893277118f144d3b928cfbb4823c04.ssl.cf1.rackcdn.com/926089/1/check/openstack-tox-py311/9df6631/testr_results.html As oslo.policy enable them by default, we override the setting for the Tacker. NOTE: there is no change in behaviour, tacker continue with the old RBAC as default. ref: https://review.opendev.org/c/openstack/requirements/+/925464 [1] https://review.opendev.org/c/openstack/oslo.policy/+/924283 [2] https://review.opendev.org/c/openstack/releases/+/925032 Change 2: Co-Authored-By: Ayumu Ueha <ueha.ayumu@fujitsu.com> This fixes the issues when "Set GLOBAL_VENV=true". 1: When running python in the subprocess on venv, the library import may fail to execute properly. This patch changes the command passed to subprocess from "python" to "sys.executable" [4]. 2: Remove compute node from nodeset for kubernetes jobs because it is not used in kubernetes jobs and to save the resources. Also left "GLOBAL_VENV=false" on the jobs related kubernetes to avoid the issue of setup on kuryr-kubernetes. [4] https://docs.python.org/3/library/sys.html#sys.executable [3] ----------- File "/home/zuul/src/opendev.org/openstack/tacker/tacker/policy.py", line 209, in authorize result = _ENFORCER.authorize(action, target, credentials, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1189, in authorize return self.enforce( ^^^^^^^^^^^^^ File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1093, in enforce raise exc(*args, **kwargs) tacker.common.exceptions.PolicyNotAuthorized: Policy doesn't allow os_nfv_orchestration_api:vnf_instances:cancel to be performed. ----------- Change-Id: I4f736660e3b1079883a7434847222f6a6589377f
2024-08-13 10:45:26 -07:00
name: openstack-k8s-3-nodes-jammy
nodes:
- name: controller
label: ubuntu-jammy
- name: controller-tacker
label: ubuntu-jammy
- name: controller-k8s
label: ubuntu-jammy
groups:
- name: subnode
nodes:
- controller-tacker
- controller-k8s
- name: switch
nodes:
- controller
- name: peers
nodes:
- controller-tacker
- controller-k8s
- job:
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
name: tacker-functional-devstack-multinode-sol-parent
parent: devstack
description: |
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
Abstraction multinodes job for SOL devstack-based functional tests
nodeset: openstack-4-nodes-jammy
pre-run: playbooks/devstack/pre.yaml
run: playbooks/devstack/run.yaml
post-run: playbooks/devstack/post.yaml
roles:
- zuul: openstack-infra/devstack
timeout: 10800
required-projects:
- openstack/cinder
- openstack/glance
- openstack/keystone
- openstack/neutron
- openstack/nova
- openstack/placement
- openstack/aodh
- openstack/horizon
- openstack/barbican
- openstack/heat
- openstack/networking-sfc
- openstack/python-barbicanclient
- openstack/python-tackerclient
- openstack/tacker
- openstack/tacker-horizon
vars:
devstack_localrc:
CELLSV2_SETUP: singleconductor
Add a seperate controller subnode to subdivide the memory load The number of required plugins has increased to the extent that controller on Zuul FT infrastructure almost runs out of memory [1]. This potentially induces various problems such as FT failure noises or POST_FAILURE at the ansible task 'export-devstack-journal'. To request expanded images such as ubuntu-focal-expanded (16GB) or ubuntu-focal-32GB would be an option, but it turns out that would lead us to another problematic situation. [2] This patch, instead, addresses the issue by subdividing the memory load. As a first step, introduce a new subnode 'tacker-controller', on which both tacker-server and tacker-conductor are located. Note: * when we re-locate some other components to this new subnode, it might better to rename it. * `devstack_local_conf: {}` in .zuul.yaml is to cancel out the global job.vars devstack_local_conf.post-config.$NEUTRON_DHCP_CONF, which is not present on 'tacker-controller' in the first place. * TACKER_MODE is set to 'standalone'. 'all-in-one' supposes core services like nova, neutron, keystone, etc. api servers are located on the same host as tacker-server. * in devstack/lib/tacker:create_tacker_accounts, SERVICE_HOST should have been TACKER_HOST. this minor fix is included. * in roles/setup-default-vim/tasks/main.yaml, the same where conditions were scattered but all tasks in it just needed to run on 'controller' only. so let us wrap them all in a block. * renamed devstack/plugin.sh:tacker_register_default_vim for clarity. * policy file modification for Heat is now done by an ansible task. it frees us from the co-location requirements for Tacker and Heat. * drop devstack/lib/tacker:is_tacker_enabled as it's no longer needed. [1]: we investigated how severe the memory load on 'controller' was on Zuul FT infrastructure: * The highest memory-consuming processes in desc order: 808.70MB (9.87%) 828112 /usr/sbin/mysqld 179.81MB (2.19%) 184124 ... /usr/local/bin/tacker-server ... 152.57MB (1.86%) 156232 ... /usr/local/bin/tacker-conductor . 146.67MB (1.79%) 150188 ... /usr/local/bin/neutron-server ... 132.96MB (1.62%) 136148 ... /usr/local/bin/neutron-server ... 129.08MB (1.58%) 132180 ... /usr/local/bin/heat-engine ... 127.48MB (1.56%) 130544 ... /usr/local/bin/heat-engine ... 122.16MB (1.49%) 125092 nova-apiuWSGI worker 1 121.00MB (1.48%) 123900 neutron-openvswitch-agent ... 119.50MB (1.46%) 122368 cinder-apiuWSGI worker 1 ---(snip)--- * `free -m` output total used free shared buff/cache available Mem: 7955 7427 196 16 331 219 Swap: 1022 1019 3 [2]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ %23openstack-infra.2020-11-25.log.html Change-Id: I030ffd5fd11b7ca9abca56e85e449ed4c4d709bd
2020-11-26 11:14:06 +09:00
PHYSICAL_NETWORK: mgmtphysnet0
OVS_BRIDGE_MAPPINGS: public:br-ex,mgmtphysnet0:br-infra
DATABASE_TYPE: mysql
Add a seperate controller subnode to subdivide the memory load The number of required plugins has increased to the extent that controller on Zuul FT infrastructure almost runs out of memory [1]. This potentially induces various problems such as FT failure noises or POST_FAILURE at the ansible task 'export-devstack-journal'. To request expanded images such as ubuntu-focal-expanded (16GB) or ubuntu-focal-32GB would be an option, but it turns out that would lead us to another problematic situation. [2] This patch, instead, addresses the issue by subdividing the memory load. As a first step, introduce a new subnode 'tacker-controller', on which both tacker-server and tacker-conductor are located. Note: * when we re-locate some other components to this new subnode, it might better to rename it. * `devstack_local_conf: {}` in .zuul.yaml is to cancel out the global job.vars devstack_local_conf.post-config.$NEUTRON_DHCP_CONF, which is not present on 'tacker-controller' in the first place. * TACKER_MODE is set to 'standalone'. 'all-in-one' supposes core services like nova, neutron, keystone, etc. api servers are located on the same host as tacker-server. * in devstack/lib/tacker:create_tacker_accounts, SERVICE_HOST should have been TACKER_HOST. this minor fix is included. * in roles/setup-default-vim/tasks/main.yaml, the same where conditions were scattered but all tasks in it just needed to run on 'controller' only. so let us wrap them all in a block. * renamed devstack/plugin.sh:tacker_register_default_vim for clarity. * policy file modification for Heat is now done by an ansible task. it frees us from the co-location requirements for Tacker and Heat. * drop devstack/lib/tacker:is_tacker_enabled as it's no longer needed. [1]: we investigated how severe the memory load on 'controller' was on Zuul FT infrastructure: * The highest memory-consuming processes in desc order: 808.70MB (9.87%) 828112 /usr/sbin/mysqld 179.81MB (2.19%) 184124 ... /usr/local/bin/tacker-server ... 152.57MB (1.86%) 156232 ... /usr/local/bin/tacker-conductor . 146.67MB (1.79%) 150188 ... /usr/local/bin/neutron-server ... 132.96MB (1.62%) 136148 ... /usr/local/bin/neutron-server ... 129.08MB (1.58%) 132180 ... /usr/local/bin/heat-engine ... 127.48MB (1.56%) 130544 ... /usr/local/bin/heat-engine ... 122.16MB (1.49%) 125092 nova-apiuWSGI worker 1 121.00MB (1.48%) 123900 neutron-openvswitch-agent ... 119.50MB (1.46%) 122368 cinder-apiuWSGI worker 1 ---(snip)--- * `free -m` output total used free shared buff/cache available Mem: 7955 7427 196 16 331 219 Swap: 1022 1019 3 [2]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ %23openstack-infra.2020-11-25.log.html Change-Id: I030ffd5fd11b7ca9abca56e85e449ed4c4d709bd
2020-11-26 11:14:06 +09:00
KEYSTONE_SERVICE_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
MYSQL_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
GNOCCHI_SERVICE_HOST: "{{ hostvars['controller-tacker']['nodepool']['private_ipv4'] }}"
Q_SERVICE_PLUGIN_CLASSES: ovn-router,networking_sfc.services.flowclassifier.plugin.FlowClassifierPlugin,neutron.services.qos.qos_plugin.QoSPlugin,qos
L2_AGENT_EXTENSIONS: qos
DEVSTACK_PARALLEL: True
OVN_L3_CREATE_PUBLIC_NETWORK: true
OVN_DBS_LOG_LEVEL: dbg
Q_ML2_PLUGIN_MECHANISM_DRIVERS: ovn,logger
test_matrix_configs: [neutron]
zuul_work_dir: src/opendev.org/openstack/tacker
host-vars:
controller:
devstack_local_conf:
post-config:
# NOTE: workaround for Heat timeout due to communication
# problem between nova and neutron.
$NEUTRON_CONF:
DEFAULT:
client_socket_timeout: 60
$NEUTRON_DHCP_CONF:
DEFAULT:
enable_isolated_metadata: True
# NOTE: workaround for Cinder image volume cache problem.
# https://bugs.launchpad.net/cinder/+bug/1953704
$CINDER_CONF:
lvmdriver-1:
image_volume_cache_enabled: False
devstack_plugins:
heat: https://opendev.org/openstack/heat
networking-sfc: https://opendev.org/openstack/networking-sfc
aodh: https://opendev.org/openstack/aodh
barbican: https://opendev.org/openstack/barbican
devstack_services:
# Core services enabled for this branch.
# This list replaces the test-matrix.
# Shared services
mysql: true
rabbit: true
tls-proxy: false
# Keystone services
key: true
# Glance services
g-api: true
# Nova services
n-api: true
n-api-meta: true
n-cond: true
n-cpu: false
n-novnc: true
n-sch: true
placement-api: true
# OVN services
ovn-controller: true
ovn-northd: true
ovs-vswitchd: true
ovsdb-server: true
# Neutron services
q-svc: true
q-qos: true
q-ovn-metadata-agent: true
# Cinder services
c-api: true
c-sch: true
c-vol: true
cinder: true
# Services we don't need.
# This section is not really needed, it's for readability.
horizon: false
tempest: false
swift: false
s-account: false
s-container: false
s-object: false
s-proxy: false
c-bak: false
tox_install_siblings: false
Add a seperate controller subnode to subdivide the memory load The number of required plugins has increased to the extent that controller on Zuul FT infrastructure almost runs out of memory [1]. This potentially induces various problems such as FT failure noises or POST_FAILURE at the ansible task 'export-devstack-journal'. To request expanded images such as ubuntu-focal-expanded (16GB) or ubuntu-focal-32GB would be an option, but it turns out that would lead us to another problematic situation. [2] This patch, instead, addresses the issue by subdividing the memory load. As a first step, introduce a new subnode 'tacker-controller', on which both tacker-server and tacker-conductor are located. Note: * when we re-locate some other components to this new subnode, it might better to rename it. * `devstack_local_conf: {}` in .zuul.yaml is to cancel out the global job.vars devstack_local_conf.post-config.$NEUTRON_DHCP_CONF, which is not present on 'tacker-controller' in the first place. * TACKER_MODE is set to 'standalone'. 'all-in-one' supposes core services like nova, neutron, keystone, etc. api servers are located on the same host as tacker-server. * in devstack/lib/tacker:create_tacker_accounts, SERVICE_HOST should have been TACKER_HOST. this minor fix is included. * in roles/setup-default-vim/tasks/main.yaml, the same where conditions were scattered but all tasks in it just needed to run on 'controller' only. so let us wrap them all in a block. * renamed devstack/plugin.sh:tacker_register_default_vim for clarity. * policy file modification for Heat is now done by an ansible task. it frees us from the co-location requirements for Tacker and Heat. * drop devstack/lib/tacker:is_tacker_enabled as it's no longer needed. [1]: we investigated how severe the memory load on 'controller' was on Zuul FT infrastructure: * The highest memory-consuming processes in desc order: 808.70MB (9.87%) 828112 /usr/sbin/mysqld 179.81MB (2.19%) 184124 ... /usr/local/bin/tacker-server ... 152.57MB (1.86%) 156232 ... /usr/local/bin/tacker-conductor . 146.67MB (1.79%) 150188 ... /usr/local/bin/neutron-server ... 132.96MB (1.62%) 136148 ... /usr/local/bin/neutron-server ... 129.08MB (1.58%) 132180 ... /usr/local/bin/heat-engine ... 127.48MB (1.56%) 130544 ... /usr/local/bin/heat-engine ... 122.16MB (1.49%) 125092 nova-apiuWSGI worker 1 121.00MB (1.48%) 123900 neutron-openvswitch-agent ... 119.50MB (1.46%) 122368 cinder-apiuWSGI worker 1 ---(snip)--- * `free -m` output total used free shared buff/cache available Mem: 7955 7427 196 16 331 219 Swap: 1022 1019 3 [2]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ %23openstack-infra.2020-11-25.log.html Change-Id: I030ffd5fd11b7ca9abca56e85e449ed4c4d709bd
2020-11-26 11:14:06 +09:00
controller-tacker:
devstack_plugins:
tacker: https://opendev.org/openstack/tacker
Add a seperate controller subnode to subdivide the memory load The number of required plugins has increased to the extent that controller on Zuul FT infrastructure almost runs out of memory [1]. This potentially induces various problems such as FT failure noises or POST_FAILURE at the ansible task 'export-devstack-journal'. To request expanded images such as ubuntu-focal-expanded (16GB) or ubuntu-focal-32GB would be an option, but it turns out that would lead us to another problematic situation. [2] This patch, instead, addresses the issue by subdividing the memory load. As a first step, introduce a new subnode 'tacker-controller', on which both tacker-server and tacker-conductor are located. Note: * when we re-locate some other components to this new subnode, it might better to rename it. * `devstack_local_conf: {}` in .zuul.yaml is to cancel out the global job.vars devstack_local_conf.post-config.$NEUTRON_DHCP_CONF, which is not present on 'tacker-controller' in the first place. * TACKER_MODE is set to 'standalone'. 'all-in-one' supposes core services like nova, neutron, keystone, etc. api servers are located on the same host as tacker-server. * in devstack/lib/tacker:create_tacker_accounts, SERVICE_HOST should have been TACKER_HOST. this minor fix is included. * in roles/setup-default-vim/tasks/main.yaml, the same where conditions were scattered but all tasks in it just needed to run on 'controller' only. so let us wrap them all in a block. * renamed devstack/plugin.sh:tacker_register_default_vim for clarity. * policy file modification for Heat is now done by an ansible task. it frees us from the co-location requirements for Tacker and Heat. * drop devstack/lib/tacker:is_tacker_enabled as it's no longer needed. [1]: we investigated how severe the memory load on 'controller' was on Zuul FT infrastructure: * The highest memory-consuming processes in desc order: 808.70MB (9.87%) 828112 /usr/sbin/mysqld 179.81MB (2.19%) 184124 ... /usr/local/bin/tacker-server ... 152.57MB (1.86%) 156232 ... /usr/local/bin/tacker-conductor . 146.67MB (1.79%) 150188 ... /usr/local/bin/neutron-server ... 132.96MB (1.62%) 136148 ... /usr/local/bin/neutron-server ... 129.08MB (1.58%) 132180 ... /usr/local/bin/heat-engine ... 127.48MB (1.56%) 130544 ... /usr/local/bin/heat-engine ... 122.16MB (1.49%) 125092 nova-apiuWSGI worker 1 121.00MB (1.48%) 123900 neutron-openvswitch-agent ... 119.50MB (1.46%) 122368 cinder-apiuWSGI worker 1 ---(snip)--- * `free -m` output total used free shared buff/cache available Mem: 7955 7427 196 16 331 219 Swap: 1022 1019 3 [2]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ %23openstack-infra.2020-11-25.log.html Change-Id: I030ffd5fd11b7ca9abca56e85e449ed4c4d709bd
2020-11-26 11:14:06 +09:00
devstack_services:
# OVN services
ovn-controller: true
ovs-vswitchd: true
ovsdb-server: true
# Neutron services
q-ovn-metadata-agent: true
Add a seperate controller subnode to subdivide the memory load The number of required plugins has increased to the extent that controller on Zuul FT infrastructure almost runs out of memory [1]. This potentially induces various problems such as FT failure noises or POST_FAILURE at the ansible task 'export-devstack-journal'. To request expanded images such as ubuntu-focal-expanded (16GB) or ubuntu-focal-32GB would be an option, but it turns out that would lead us to another problematic situation. [2] This patch, instead, addresses the issue by subdividing the memory load. As a first step, introduce a new subnode 'tacker-controller', on which both tacker-server and tacker-conductor are located. Note: * when we re-locate some other components to this new subnode, it might better to rename it. * `devstack_local_conf: {}` in .zuul.yaml is to cancel out the global job.vars devstack_local_conf.post-config.$NEUTRON_DHCP_CONF, which is not present on 'tacker-controller' in the first place. * TACKER_MODE is set to 'standalone'. 'all-in-one' supposes core services like nova, neutron, keystone, etc. api servers are located on the same host as tacker-server. * in devstack/lib/tacker:create_tacker_accounts, SERVICE_HOST should have been TACKER_HOST. this minor fix is included. * in roles/setup-default-vim/tasks/main.yaml, the same where conditions were scattered but all tasks in it just needed to run on 'controller' only. so let us wrap them all in a block. * renamed devstack/plugin.sh:tacker_register_default_vim for clarity. * policy file modification for Heat is now done by an ansible task. it frees us from the co-location requirements for Tacker and Heat. * drop devstack/lib/tacker:is_tacker_enabled as it's no longer needed. [1]: we investigated how severe the memory load on 'controller' was on Zuul FT infrastructure: * The highest memory-consuming processes in desc order: 808.70MB (9.87%) 828112 /usr/sbin/mysqld 179.81MB (2.19%) 184124 ... /usr/local/bin/tacker-server ... 152.57MB (1.86%) 156232 ... /usr/local/bin/tacker-conductor . 146.67MB (1.79%) 150188 ... /usr/local/bin/neutron-server ... 132.96MB (1.62%) 136148 ... /usr/local/bin/neutron-server ... 129.08MB (1.58%) 132180 ... /usr/local/bin/heat-engine ... 127.48MB (1.56%) 130544 ... /usr/local/bin/heat-engine ... 122.16MB (1.49%) 125092 nova-apiuWSGI worker 1 121.00MB (1.48%) 123900 neutron-openvswitch-agent ... 119.50MB (1.46%) 122368 cinder-apiuWSGI worker 1 ---(snip)--- * `free -m` output total used free shared buff/cache available Mem: 7955 7427 196 16 331 219 Swap: 1022 1019 3 [2]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ %23openstack-infra.2020-11-25.log.html Change-Id: I030ffd5fd11b7ca9abca56e85e449ed4c4d709bd
2020-11-26 11:14:06 +09:00
# Tacker services
tacker: true
tacker-conductor: true
group-vars:
compute:
# Since a VirtualInterfaceCreateException occurs during a test,
# the setting of network-vif-plugged is changed by the reference of
# the following URL.
# https://bugs.launchpad.net/heat/+bug/1694371
devstack_local_conf:
post-config:
$NOVA_CONF:
DEFAULT:
vif_plugging_is_fatal: False
subnode:
devstack_localrc:
CELLSV2_SETUP: singleconductor
PHYSICAL_NETWORK: mgmtphysnet0
OVS_BRIDGE_MAPPINGS: public:br-ex,mgmtphysnet0:br-infra
DATABASE_TYPE: mysql
Add a seperate controller subnode to subdivide the memory load The number of required plugins has increased to the extent that controller on Zuul FT infrastructure almost runs out of memory [1]. This potentially induces various problems such as FT failure noises or POST_FAILURE at the ansible task 'export-devstack-journal'. To request expanded images such as ubuntu-focal-expanded (16GB) or ubuntu-focal-32GB would be an option, but it turns out that would lead us to another problematic situation. [2] This patch, instead, addresses the issue by subdividing the memory load. As a first step, introduce a new subnode 'tacker-controller', on which both tacker-server and tacker-conductor are located. Note: * when we re-locate some other components to this new subnode, it might better to rename it. * `devstack_local_conf: {}` in .zuul.yaml is to cancel out the global job.vars devstack_local_conf.post-config.$NEUTRON_DHCP_CONF, which is not present on 'tacker-controller' in the first place. * TACKER_MODE is set to 'standalone'. 'all-in-one' supposes core services like nova, neutron, keystone, etc. api servers are located on the same host as tacker-server. * in devstack/lib/tacker:create_tacker_accounts, SERVICE_HOST should have been TACKER_HOST. this minor fix is included. * in roles/setup-default-vim/tasks/main.yaml, the same where conditions were scattered but all tasks in it just needed to run on 'controller' only. so let us wrap them all in a block. * renamed devstack/plugin.sh:tacker_register_default_vim for clarity. * policy file modification for Heat is now done by an ansible task. it frees us from the co-location requirements for Tacker and Heat. * drop devstack/lib/tacker:is_tacker_enabled as it's no longer needed. [1]: we investigated how severe the memory load on 'controller' was on Zuul FT infrastructure: * The highest memory-consuming processes in desc order: 808.70MB (9.87%) 828112 /usr/sbin/mysqld 179.81MB (2.19%) 184124 ... /usr/local/bin/tacker-server ... 152.57MB (1.86%) 156232 ... /usr/local/bin/tacker-conductor . 146.67MB (1.79%) 150188 ... /usr/local/bin/neutron-server ... 132.96MB (1.62%) 136148 ... /usr/local/bin/neutron-server ... 129.08MB (1.58%) 132180 ... /usr/local/bin/heat-engine ... 127.48MB (1.56%) 130544 ... /usr/local/bin/heat-engine ... 122.16MB (1.49%) 125092 nova-apiuWSGI worker 1 121.00MB (1.48%) 123900 neutron-openvswitch-agent ... 119.50MB (1.46%) 122368 cinder-apiuWSGI worker 1 ---(snip)--- * `free -m` output total used free shared buff/cache available Mem: 7955 7427 196 16 331 219 Swap: 1022 1019 3 [2]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ %23openstack-infra.2020-11-25.log.html Change-Id: I030ffd5fd11b7ca9abca56e85e449ed4c4d709bd
2020-11-26 11:14:06 +09:00
KEYSTONE_SERVICE_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
MYSQL_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
GNOCCHI_SERVICE_HOST: "{{ hostvars['controller-tacker']['nodepool']['private_ipv4'] }}"
Add a seperate controller subnode to subdivide the memory load The number of required plugins has increased to the extent that controller on Zuul FT infrastructure almost runs out of memory [1]. This potentially induces various problems such as FT failure noises or POST_FAILURE at the ansible task 'export-devstack-journal'. To request expanded images such as ubuntu-focal-expanded (16GB) or ubuntu-focal-32GB would be an option, but it turns out that would lead us to another problematic situation. [2] This patch, instead, addresses the issue by subdividing the memory load. As a first step, introduce a new subnode 'tacker-controller', on which both tacker-server and tacker-conductor are located. Note: * when we re-locate some other components to this new subnode, it might better to rename it. * `devstack_local_conf: {}` in .zuul.yaml is to cancel out the global job.vars devstack_local_conf.post-config.$NEUTRON_DHCP_CONF, which is not present on 'tacker-controller' in the first place. * TACKER_MODE is set to 'standalone'. 'all-in-one' supposes core services like nova, neutron, keystone, etc. api servers are located on the same host as tacker-server. * in devstack/lib/tacker:create_tacker_accounts, SERVICE_HOST should have been TACKER_HOST. this minor fix is included. * in roles/setup-default-vim/tasks/main.yaml, the same where conditions were scattered but all tasks in it just needed to run on 'controller' only. so let us wrap them all in a block. * renamed devstack/plugin.sh:tacker_register_default_vim for clarity. * policy file modification for Heat is now done by an ansible task. it frees us from the co-location requirements for Tacker and Heat. * drop devstack/lib/tacker:is_tacker_enabled as it's no longer needed. [1]: we investigated how severe the memory load on 'controller' was on Zuul FT infrastructure: * The highest memory-consuming processes in desc order: 808.70MB (9.87%) 828112 /usr/sbin/mysqld 179.81MB (2.19%) 184124 ... /usr/local/bin/tacker-server ... 152.57MB (1.86%) 156232 ... /usr/local/bin/tacker-conductor . 146.67MB (1.79%) 150188 ... /usr/local/bin/neutron-server ... 132.96MB (1.62%) 136148 ... /usr/local/bin/neutron-server ... 129.08MB (1.58%) 132180 ... /usr/local/bin/heat-engine ... 127.48MB (1.56%) 130544 ... /usr/local/bin/heat-engine ... 122.16MB (1.49%) 125092 nova-apiuWSGI worker 1 121.00MB (1.48%) 123900 neutron-openvswitch-agent ... 119.50MB (1.46%) 122368 cinder-apiuWSGI worker 1 ---(snip)--- * `free -m` output total used free shared buff/cache available Mem: 7955 7427 196 16 331 219 Swap: 1022 1019 3 [2]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/ %23openstack-infra.2020-11-25.log.html Change-Id: I030ffd5fd11b7ca9abca56e85e449ed4c4d709bd
2020-11-26 11:14:06 +09:00
TACKER_HOST: "{{ hostvars['controller-tacker']['nodepool']['private_ipv4'] }}"
TACKER_MODE: standalone
IS_ZUUL_FT: True
Q_SERVICE_PLUGIN_CLASSES: "qos,\
networking_sfc.services.flowclassifier.plugin.FlowClassifierPlugin,\
neutron.services.qos.qos_plugin.QoSPlugin,\
ovn-router"
L2_AGENT_EXTENSIONS: qos
ENABLE_CHASSIS_AS_GW: false
Q_ML2_PLUGIN_MECHANISM_DRIVERS: ovn,logger
devstack_services:
# To override the parent job's definitions.
tls-proxy: false
# OVN services
ovn-controller: true
ovs-vswitchd: true
ovsdb-server: true
# Neutron services
q-ovn-metadata-agent: true
- job:
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
name: tacker-functional-devstack-multinode-sol-legacy-nfvo
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests for legacy_nfvo
host-vars:
controller-tacker:
tox_envlist: dsvm-functional-sol-legacy-nfvo
- job:
name: tacker-functional-devstack-multinode-sol-vnflcm
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests for vnflcm
host-vars:
controller-tacker:
Fix broken jobs in V1 API gates This patch fixes the following issues: 1. Change the QueuePool settings to use the values recommended. The following error occurred in sol-vnflcm and sol-kubernetes job: ``` tacker.wsgi sqlalchemy.exc.TimeoutError: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10.00 ``` Change to remove the process of overriding the default value of oslo.db config. In Neutron, the process of overriding the default value of oslo.db has already been removed [1]. However, errors still occur when using the recommended value, so temporarily set `max_pool_size` to unlimited in FT. 2. Fix issue of `Show subscription (V1 API)`. A previous patch [2] included `sqlalchemy.row.LegacyRow` for backwards compatibility with oslo.db, which caused the error "Show subscription (V1 API)" to fail by upgrading SQLAlchemy. Fix it to work correctly by removing `row.LegacyRow`. 3. Fix to upload image successfully in FT. There was a process to get the image from the glance and upload the copy image, but it was not working properly. Changed the argument to glance so that it uploads correctly. 4. Increase wait time Terminate after Heal in sol-vnflcm job. A 409 error occurred at the time of Terminate because the process was to wait a fixed number of seconds after Heal before Terminate. The current 20 seconds may not be enough, so increase it to 60 seconds. This will be fixed to check if OpOcc is COMPLETED status in future work. 5. Split test jobs for sol-vnflcm. Split test jobs for sol-vnflcm because it takes longer to run and, it is executed using the full limit time. [1] https://review.opendev.org/c/openstack/neutron/+/456419 [2] https://review.opendev.org/c/openstack/tacker/+/910720 Closes-Bug: #2066047 Change-Id: If1cd835c8875fa844f9eac5bff5ba85103ad91aa
2024-07-26 09:56:04 +00:00
devstack_local_conf:
post-config:
$TACKER_CONF:
database:
max_pool_size: 0
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
tox_envlist: dsvm-functional-sol-vnflcm
Fix broken jobs in V1 API gates This patch fixes the following issues: 1. Change the QueuePool settings to use the values recommended. The following error occurred in sol-vnflcm and sol-kubernetes job: ``` tacker.wsgi sqlalchemy.exc.TimeoutError: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10.00 ``` Change to remove the process of overriding the default value of oslo.db config. In Neutron, the process of overriding the default value of oslo.db has already been removed [1]. However, errors still occur when using the recommended value, so temporarily set `max_pool_size` to unlimited in FT. 2. Fix issue of `Show subscription (V1 API)`. A previous patch [2] included `sqlalchemy.row.LegacyRow` for backwards compatibility with oslo.db, which caused the error "Show subscription (V1 API)" to fail by upgrading SQLAlchemy. Fix it to work correctly by removing `row.LegacyRow`. 3. Fix to upload image successfully in FT. There was a process to get the image from the glance and upload the copy image, but it was not working properly. Changed the argument to glance so that it uploads correctly. 4. Increase wait time Terminate after Heal in sol-vnflcm job. A 409 error occurred at the time of Terminate because the process was to wait a fixed number of seconds after Heal before Terminate. The current 20 seconds may not be enough, so increase it to 60 seconds. This will be fixed to check if OpOcc is COMPLETED status in future work. 5. Split test jobs for sol-vnflcm. Split test jobs for sol-vnflcm because it takes longer to run and, it is executed using the full limit time. [1] https://review.opendev.org/c/openstack/neutron/+/456419 [2] https://review.opendev.org/c/openstack/tacker/+/910720 Closes-Bug: #2066047 Change-Id: If1cd835c8875fa844f9eac5bff5ba85103ad91aa
2024-07-26 09:56:04 +00:00
- job:
name: tacker-functional-devstack-multinode-sol-vnflcm-userdata
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests for vnflcm with userdata
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
database:
max_pool_size: 0
tox_envlist: dsvm-functional-sol-vnflcm-userdata
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
- job:
name: tacker-functional-devstack-multinode-sol-vnfpkgm
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests for vnfpkgm
host-vars:
controller-tacker:
tox_envlist: dsvm-functional-sol-vnfpkgm
- job:
name: tacker-functional-devstack-multinode-sol-v2-basic
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL V2 devstack-based functional tests
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
server_notification:
server_notification: true
prometheus_plugin:
auto_scaling: true
auto_healing: true
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
v2_vnfm:
# Notes: The List API will return 'Link' Header in response
# body only if page_size is not 0. This job will check 'Link'
# Header, so these page_size parameters are set to 1.
vnf_instance_page_size: 1
lcm_op_occ_page_size: 1
tox_envlist: dsvm-functional-sol-v2-basic
- job:
name: tacker-functional-devstack-multinode-sol-v2-vnflcm
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL V2 devstack-based functional tests
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
server_notification:
server_notification: true
prometheus_plugin:
auto_scaling: true
auto_healing: true
tox_envlist: dsvm-functional-sol-v2-vnflcm
voting: false
- job:
name: tacker-functional-devstack-multinode-sol-v2-notification
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL V2 devstack-based functional tests
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
server_notification:
server_notification: true
tox_envlist: dsvm-functional-sol-v2-notification
- job:
name: tacker-functional-devstack-multinode-sol-v2-prometheus
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL V2 devstack-based functional tests
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
prometheus_plugin:
auto_scaling: true
auto_healing: true
tox_envlist: dsvm-functional-sol-v2-prometheus
- job:
name: tacker-functional-devstack-multinode-sol-multi-tenant
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL Multi tenant devstack-based functional tests
host-vars:
controller-tacker:
tox_envlist: dsvm-functional-sol-multi-tenant
vars:
setup_multi_tenant: true
- job:
name: tacker-functional-devstack-multinode-sol-separated-nfvo
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests
with separated NFVO
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
connect_vnf_packages:
base_url: http://127.0.0.1:9990/vnfpkgm/v1/vnf_packages
pipeline: package_content,vnfd,artifacts
connect_grant:
base_url: http://127.0.0.1:9990/grant/v1/grants
openstack_vim:
stack_retries: 120
tox_envlist: dsvm-functional-sol-separated-nfvo
- job:
name: tacker-functional-devstack-multinode-sol-separated-nfvo-v2
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests
with separated V2 NFVO
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
v2_nfvo:
use_external_nfvo: True
endpoint: http://127.0.0.1:9990
token_endpoint: http://127.0.0.1:9990/token
client_id: 229ec984de7547b2b662e968961af5a4
client_password: devstack
use_client_secret_basic: True
tox_envlist: dsvm-functional-sol-separated-nfvo-v2
- job:
name: tacker-functional-devstack-multinode-sol-v2-individual-vnfc-mgmt
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinode job for SOL V2 devstack-based individual vnfc mgmt functional
tests
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
v2_vnfm:
# Notes: The List API will return 'Link' Header in response
# body only if page_size is not 0. This job will check 'Link'
# Header, so these page_size parameters are set to 1.
vnf_instance_page_size: 1
lcm_op_occ_page_size: 1
tox_envlist: dsvm-functional-sol-v2-individual-vnfc-mgmt
- job:
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
name: tacker-functional-devstack-multinode-sol-kubernetes-parent
parent: devstack
description: |
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
Abstraction multinodes job for SOL devstack-based kubernetes functional
tests
Keep new RBAC disable by default & Gate fix for GLOBAL_VENV This commit squash the multiple changes: Changes 1: Keep new RBAC disable by default oslo.policy has enabled the new RBAC config options enforce_scope and enforce_new_defaults by default[1][2]. Tacker implemented the new RBAC in previous cycle with new RBAC disable by default. To give more time to operator, let's continue the same setting in this release also. Also, there are many test modification is needed for the new RBAC (using the new RBAC default role in tests) Unit tests log a lot of policy warnings, suppressing those[3] -https://fa3204066787dd37fd86-ea893277118f144d3b928cfbb4823c04.ssl.cf1.rackcdn.com/926089/1/check/openstack-tox-py311/9df6631/testr_results.html As oslo.policy enable them by default, we override the setting for the Tacker. NOTE: there is no change in behaviour, tacker continue with the old RBAC as default. ref: https://review.opendev.org/c/openstack/requirements/+/925464 [1] https://review.opendev.org/c/openstack/oslo.policy/+/924283 [2] https://review.opendev.org/c/openstack/releases/+/925032 Change 2: Co-Authored-By: Ayumu Ueha <ueha.ayumu@fujitsu.com> This fixes the issues when "Set GLOBAL_VENV=true". 1: When running python in the subprocess on venv, the library import may fail to execute properly. This patch changes the command passed to subprocess from "python" to "sys.executable" [4]. 2: Remove compute node from nodeset for kubernetes jobs because it is not used in kubernetes jobs and to save the resources. Also left "GLOBAL_VENV=false" on the jobs related kubernetes to avoid the issue of setup on kuryr-kubernetes. [4] https://docs.python.org/3/library/sys.html#sys.executable [3] ----------- File "/home/zuul/src/opendev.org/openstack/tacker/tacker/policy.py", line 209, in authorize result = _ENFORCER.authorize(action, target, credentials, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1189, in authorize return self.enforce( ^^^^^^^^^^^^^ File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1093, in enforce raise exc(*args, **kwargs) tacker.common.exceptions.PolicyNotAuthorized: Policy doesn't allow os_nfv_orchestration_api:vnf_instances:cancel to be performed. ----------- Change-Id: I4f736660e3b1079883a7434847222f6a6589377f
2024-08-13 10:45:26 -07:00
nodeset: openstack-k8s-3-nodes-jammy
pre-run: playbooks/devstack/pre.yaml
run: playbooks/devstack/run.yaml
post-run: playbooks/devstack/post.yaml
roles:
- zuul: openstack-infra/devstack
timeout: 10800
required-projects:
- openstack/barbican
- openstack/cinder
- openstack/devstack-plugin-container
- openstack/glance
- openstack/heat
- openstack/horizon
- openstack/keystone
- openstack/kuryr-kubernetes
- openstack/neutron
- openstack/nova
- openstack/octavia
- openstack/placement
- openstack/python-barbicanclient
- openstack/python-octaviaclient
- openstack/python-tackerclient
- openstack/tacker
- openstack/tacker-horizon
host-vars:
controller:
devstack_plugins:
barbican: https://opendev.org/openstack/barbican
heat: https://opendev.org/openstack/heat
neutron: https://opendev.org/openstack/neutron
octavia: https://opendev.org/openstack/octavia
devstack_services:
base: false
c-api: true
c-bak: false
c-sch: true
c-vol: true
cinder: true
coredns: false
etcd3: true
g-api: true
g-reg: true
horizon: false
key: true
mysql: true
n-api-meta: true
n-api: true
n-cond: true
n-cpu: false
n-novnc: true
n-sch: true
neutron: true
o-api: true
o-cw: true
o-hk: true
o-hm: true
octavia: true
placement-api: true
placement-client: true
ovn-controller: true
ovn-northd: true
ovs-vswitchd: true
ovsdb-server: true
q-qos: true
q-svc: true
q-ovn-metadata-agent: true
rabbit: true
s-account: false
s-container: false
s-object: false
s-proxy: false
swift: false
tempest: false
tls-proxy: false
tox_install_siblings: false
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
kubernetes_vim:
stack_retries: 120
devstack_services:
ovn-controller: true
ovn-northd: true
ovs-vswitchd: true
ovsdb-server: true
q-ovn-metadata-agent: true
tacker: true
tacker-conductor: true
devstack_plugins:
tacker: https://opendev.org/openstack/tacker
controller-k8s:
devstack_local_conf: {}
devstack_plugins:
devstack-plugin-container: https://opendev.org/openstack/devstack-plugin-container
kuryr-kubernetes: https://opendev.org/openstack/kuryr-kubernetes
devstack_services:
etcd3: false
kubernetes-master: true
kuryr-daemon: true
kuryr-kubernetes: true
octavia: false
ovn-controller: true
ovn-northd: true
ovs-vswitchd: true
ovsdb-server: true
q-ovn-metadata-agent: true
tox_install_siblings: false
group-vars:
subnode:
devstack_localrc:
CELLSV2_SETUP: singleconductor
DATABASE_TYPE: mysql
IS_ZUUL_FT: True
K8S_API_SERVER_IP: "{{ hostvars['controller-k8s']['nodepool']['private_ipv4'] }}"
KEYSTONE_SERVICE_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
KURYR_FORCE_IMAGE_BUILD: true
KURYR_K8S_API_PORT: 6443
KURYR_K8S_API_URL: "https://{{ hostvars['controller-k8s']['nodepool']['private_ipv4'] }}:${KURYR_K8S_API_PORT}"
KURYR_K8S_CONTAINERIZED_DEPLOYMENT: false
KURYR_NEUTRON_DEFAULT_SUBNETPOOL_ID: shared-default-subnetpool-v4
# NOTES:
# - In Bobcat cycle, Kubernetes version is updated to 1.26.
# https://blueprints.launchpad.net/tacker/+spec/update-k8s-helm-prometheus
KURYR_KUBERNETES_VERSION: 1.26.8
CONTAINER_ENGINE: crio
CRIO_VERSION: 1.26
MYSQL_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
OCTAVIA_AMP_IMAGE_FILE: "/tmp/test-only-amphora-x64-haproxy-ubuntu-bionic.qcow2"
OCTAVIA_AMP_IMAGE_NAME: "test-only-amphora-x64-haproxy-ubuntu-bionic"
OCTAVIA_AMP_IMAGE_SIZE: 3
OVS_BRIDGE_MAPPINGS: public:br-ex,mgmtphysnet0:br-infra
PHYSICAL_NETWORK: mgmtphysnet0
TACKER_HOST: "{{ hostvars['controller-tacker']['nodepool']['private_ipv4'] }}"
TACKER_MODE: standalone
USE_PYTHON3: true
ENABLE_CHASSIS_AS_GW: false
Q_ML2_PLUGIN_MECHANISM_DRIVERS: ovn,logger
Keep new RBAC disable by default & Gate fix for GLOBAL_VENV This commit squash the multiple changes: Changes 1: Keep new RBAC disable by default oslo.policy has enabled the new RBAC config options enforce_scope and enforce_new_defaults by default[1][2]. Tacker implemented the new RBAC in previous cycle with new RBAC disable by default. To give more time to operator, let's continue the same setting in this release also. Also, there are many test modification is needed for the new RBAC (using the new RBAC default role in tests) Unit tests log a lot of policy warnings, suppressing those[3] -https://fa3204066787dd37fd86-ea893277118f144d3b928cfbb4823c04.ssl.cf1.rackcdn.com/926089/1/check/openstack-tox-py311/9df6631/testr_results.html As oslo.policy enable them by default, we override the setting for the Tacker. NOTE: there is no change in behaviour, tacker continue with the old RBAC as default. ref: https://review.opendev.org/c/openstack/requirements/+/925464 [1] https://review.opendev.org/c/openstack/oslo.policy/+/924283 [2] https://review.opendev.org/c/openstack/releases/+/925032 Change 2: Co-Authored-By: Ayumu Ueha <ueha.ayumu@fujitsu.com> This fixes the issues when "Set GLOBAL_VENV=true". 1: When running python in the subprocess on venv, the library import may fail to execute properly. This patch changes the command passed to subprocess from "python" to "sys.executable" [4]. 2: Remove compute node from nodeset for kubernetes jobs because it is not used in kubernetes jobs and to save the resources. Also left "GLOBAL_VENV=false" on the jobs related kubernetes to avoid the issue of setup on kuryr-kubernetes. [4] https://docs.python.org/3/library/sys.html#sys.executable [3] ----------- File "/home/zuul/src/opendev.org/openstack/tacker/tacker/policy.py", line 209, in authorize result = _ENFORCER.authorize(action, target, credentials, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1189, in authorize return self.enforce( ^^^^^^^^^^^^^ File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1093, in enforce raise exc(*args, **kwargs) tacker.common.exceptions.PolicyNotAuthorized: Policy doesn't allow os_nfv_orchestration_api:vnf_instances:cancel to be performed. ----------- Change-Id: I4f736660e3b1079883a7434847222f6a6589377f
2024-08-13 10:45:26 -07:00
# TODO(ueha): Remove this workarround if the Zuul jobs succeed with GLOBAL_VENV=true
GLOBAL_VENV: false
devstack_services:
dstat: false
horizon: false
n-api-meta: false
n-api: false
n-cauth: false
n-cond: false
n-cpu: true
n-novnc: false
n-obj: false
n-sch: false
ovn-controller: true
ovs-vswitchd: true
ovsdb-server: true
q-ovn-metadata-agent: true
tls-proxy: false
vars:
devstack_localrc:
CELLSV2_SETUP: singleconductor
DATABASE_TYPE: mysql
ETCD_USE_RAMDISK: true
IS_ZUUL_FT: True
KEYSTONE_SERVICE_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
L2_AGENT_EXTENSIONS: qos
MYSQL_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
OCTAVIA_AMP_IMAGE_FILE: "/tmp/test-only-amphora-x64-haproxy-ubuntu-bionic.qcow2"
OCTAVIA_AMP_IMAGE_NAME: "test-only-amphora-x64-haproxy-ubuntu-bionic"
OCTAVIA_AMP_IMAGE_SIZE: 3
OVS_BRIDGE_MAPPINGS: public:br-ex,mgmtphysnet0:br-infra
PHYSICAL_NETWORK: mgmtphysnet0
Q_SERVICE_PLUGIN_CLASSES: ovn-router,neutron.services.qos.qos_plugin.QoSPlugin,qos
TACKER_HOST: "{{ hostvars['controller-tacker']['nodepool']['private_ipv4'] }}"
TACKER_MODE: standalone
USE_PYTHON3: true
DEVSTACK_PARALLEL: True
OVN_L3_CREATE_PUBLIC_NETWORK: true
OVN_DBS_LOG_LEVEL: dbg
Q_ML2_PLUGIN_MECHANISM_DRIVERS: ovn,logger
Keep new RBAC disable by default & Gate fix for GLOBAL_VENV This commit squash the multiple changes: Changes 1: Keep new RBAC disable by default oslo.policy has enabled the new RBAC config options enforce_scope and enforce_new_defaults by default[1][2]. Tacker implemented the new RBAC in previous cycle with new RBAC disable by default. To give more time to operator, let's continue the same setting in this release also. Also, there are many test modification is needed for the new RBAC (using the new RBAC default role in tests) Unit tests log a lot of policy warnings, suppressing those[3] -https://fa3204066787dd37fd86-ea893277118f144d3b928cfbb4823c04.ssl.cf1.rackcdn.com/926089/1/check/openstack-tox-py311/9df6631/testr_results.html As oslo.policy enable them by default, we override the setting for the Tacker. NOTE: there is no change in behaviour, tacker continue with the old RBAC as default. ref: https://review.opendev.org/c/openstack/requirements/+/925464 [1] https://review.opendev.org/c/openstack/oslo.policy/+/924283 [2] https://review.opendev.org/c/openstack/releases/+/925032 Change 2: Co-Authored-By: Ayumu Ueha <ueha.ayumu@fujitsu.com> This fixes the issues when "Set GLOBAL_VENV=true". 1: When running python in the subprocess on venv, the library import may fail to execute properly. This patch changes the command passed to subprocess from "python" to "sys.executable" [4]. 2: Remove compute node from nodeset for kubernetes jobs because it is not used in kubernetes jobs and to save the resources. Also left "GLOBAL_VENV=false" on the jobs related kubernetes to avoid the issue of setup on kuryr-kubernetes. [4] https://docs.python.org/3/library/sys.html#sys.executable [3] ----------- File "/home/zuul/src/opendev.org/openstack/tacker/tacker/policy.py", line 209, in authorize result = _ENFORCER.authorize(action, target, credentials, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1189, in authorize return self.enforce( ^^^^^^^^^^^^^ File "/home/zuul/src/opendev.org/openstack/tacker/.tox/py311/lib/python3.11/site-packages/oslo_policy/policy.py", line 1093, in enforce raise exc(*args, **kwargs) tacker.common.exceptions.PolicyNotAuthorized: Policy doesn't allow os_nfv_orchestration_api:vnf_instances:cancel to be performed. ----------- Change-Id: I4f736660e3b1079883a7434847222f6a6589377f
2024-08-13 10:45:26 -07:00
# TODO(ueha): Remove this workarround if the Zuul jobs succeed with GLOBAL_VENV=true
GLOBAL_VENV: false
devstack_local_conf:
post-config:
$NEUTRON_DHCP_CONF:
DEFAULT:
enable_isolated_metadata: True
$OCTAVIA_CONF:
controller_worker:
amp_active_retries: 9999
kuryr_k8s_api_url: "https://{{ hostvars['controller-k8s']['nodepool']['private_ipv4'] }}:6443"
k8s_ssl_verify: true
# NOTES:
# - In Bobcat cycle, Helm version is updated to 3.11.
# https://blueprints.launchpad.net/tacker/+spec/update-k8s-helm-prometheus
helm_version: "3.11.3"
test_matrix_configs: [neutron]
zuul_work_dir: src/opendev.org/openstack/tacker
zuul_copy_output:
'{{ devstack_log_dir }}/kubernetes': 'logs'
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
- job:
name: tacker-functional-devstack-multinode-sol-kubernetes
parent: tacker-functional-devstack-multinode-sol-kubernetes-parent
description: |
Multinodes job for SOL Kubernetes Multi tenant devstack-based functional tests
host-vars:
controller-tacker:
Fix broken jobs in V1 API gates This patch fixes the following issues: 1. Change the QueuePool settings to use the values recommended. The following error occurred in sol-vnflcm and sol-kubernetes job: ``` tacker.wsgi sqlalchemy.exc.TimeoutError: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10.00 ``` Change to remove the process of overriding the default value of oslo.db config. In Neutron, the process of overriding the default value of oslo.db has already been removed [1]. However, errors still occur when using the recommended value, so temporarily set `max_pool_size` to unlimited in FT. 2. Fix issue of `Show subscription (V1 API)`. A previous patch [2] included `sqlalchemy.row.LegacyRow` for backwards compatibility with oslo.db, which caused the error "Show subscription (V1 API)" to fail by upgrading SQLAlchemy. Fix it to work correctly by removing `row.LegacyRow`. 3. Fix to upload image successfully in FT. There was a process to get the image from the glance and upload the copy image, but it was not working properly. Changed the argument to glance so that it uploads correctly. 4. Increase wait time Terminate after Heal in sol-vnflcm job. A 409 error occurred at the time of Terminate because the process was to wait a fixed number of seconds after Heal before Terminate. The current 20 seconds may not be enough, so increase it to 60 seconds. This will be fixed to check if OpOcc is COMPLETED status in future work. 5. Split test jobs for sol-vnflcm. Split test jobs for sol-vnflcm because it takes longer to run and, it is executed using the full limit time. [1] https://review.opendev.org/c/openstack/neutron/+/456419 [2] https://review.opendev.org/c/openstack/tacker/+/910720 Closes-Bug: #2066047 Change-Id: If1cd835c8875fa844f9eac5bff5ba85103ad91aa
2024-07-26 09:56:04 +00:00
devstack_local_conf:
post-config:
$TACKER_CONF:
database:
max_pool_size: 0
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
tox_envlist: dsvm-functional-sol-kubernetes
- job:
name: tacker-functional-devstack-multinode-sol-kubernetes-multi-tenant
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-kubernetes-parent
description: |
Multinodes job for SOL Kubernetes Multi tenant devstack-based functional tests
host-vars:
controller-tacker:
tox_envlist: dsvm-functional-sol-kubernetes-multi-tenant
vars:
setup_multi_tenant: true
- job:
name: tacker-functional-devstack-multinode-sol-kubernetes-v2
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-kubernetes-parent
description: |
Multinodes job for SOL Kubernetes V2 devstack-based functional tests
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
v2_vnfm:
kubernetes_vim_rsc_wait_timeout: 800
prometheus_plugin:
fault_management: true
performance_management: true
auto_scaling: true
auto_healing: true
test_rule_with_promtool: true
tox_envlist: dsvm-functional-sol-kubernetes-v2
vars:
prometheus_setup: true
- job:
name: tacker-functional-devstack-multinode-sol-https-v2
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests
with https request
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
v2_vnfm:
notification_verify_cert: true
notification_ca_cert_file: /etc/https_server/ssl/ca.crt
use_oauth2_mtls_for_heat: false
heat_verify_cert: true
heat_ca_cert_file: /etc/https_server/ssl/ca.crt
prometheus_plugin:
fault_management: true
performance_management: true
auto_scaling: true
performance_management_package: tacker.sol_refactored.common.monitoring_plugin_base
performance_management_class: MonitoringPluginStub
v2_nfvo:
use_external_nfvo: true
endpoint: https://localhost:9990
token_endpoint: https://localhost:9990/token
client_id: 229ec984de7547b2b662e968961af5a4
client_password: devstack
nfvo_verify_cert: true
nfvo_ca_cert_file: /etc/https_server/ssl/ca.crt
use_client_secret_basic: true
tox_envlist: dsvm-functional-sol-https-v2
vars:
https_setup: true
- job:
name: tacker-functional-devstack-multinode-sol-encrypt-cred-barbican
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests
with encrypt credentials
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
DEFAULT:
use_credential_encryption: true
keymanager_type: "barbican"
v2_nfvo:
use_external_nfvo: true
endpoint: https://localhost:9990
token_endpoint: https://localhost:9990/token
client_id: 229ec984de7547b2b662e968961af5a4
client_password: devstack
nfvo_ca_cert_file: /etc/https_server/ssl/ca.crt
nfvo_verify_cert: true
use_client_secret_basic: true
tox_envlist: dsvm-functional-sol-encrypt-cred-barbican
vars:
https_setup: true
- job:
name: tacker-functional-devstack-multinode-sol-encrypt-cred-local
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based functional tests
with encrypt credentials
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
DEFAULT:
use_credential_encryption: true
keymanager_type: "local"
crypt_key_dir: "/etc/tacker/crypt/fernet_keys"
v2_nfvo:
use_external_nfvo: true
endpoint: https://localhost:9990
token_endpoint: https://localhost:9990/token
client_id: 229ec984de7547b2b662e968961af5a4
client_password: devstack
nfvo_ca_cert_file: /etc/https_server/ssl/ca.crt
nfvo_verify_cert: true
use_client_secret_basic: true
tox_envlist: dsvm-functional-sol-encrypt-cred-local
vars:
https_setup: true
- job:
name: tacker-functional-devstack-kubernetes-oidc-auth
parent: tacker-functional-devstack-multinode-sol-kubernetes-v2
description: |
Multinodes job for Kubernetes OIDC Auth tests
host-vars:
controller-tacker:
tox_envlist: dsvm-functional-sol_kubernetes_oidc_auth
vars:
keycloak_host: "{{ hostvars['controller-k8s']['nodepool']['private_ipv4'] }}"
keycloak_http_port: 8080
keycloak_https_port: 8443
- job:
name: tacker-functional-devstack-multinode-sol-v2-az-retry
parent: tacker-functional-devstack-multinode-sol-v2
description: |
Multinodes job for retry of AZ selection in SOL V2 devstack-based functional tests
host-vars:
controller-tacker:
devstack_local_conf:
post-config:
$TACKER_CONF:
v2_nfvo:
test_grant_zone_list: az-1
v2_vnfm:
placement_fallback_best_effort: true
enable_rollback_stack: true
devstack_services:
n-cpu: true
placement-client: true
tox_envlist: dsvm-functional-sol-v2-az-retry
vars:
setup_multi_az: true
controller_tacker_hostname: "{{ hostvars['controller-tacker']['ansible_hostname'] }}"
- job:
name: tacker-functional-devstack-enhanced-policy-sol
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Enhanced policy job for SOL devstack-based functional tests
host-vars:
controller-tacker:
tox_envlist: dsvm-functional-enhanced-policy-sol
devstack_local_conf:
post-config:
$TACKER_CONF:
oslo_policy:
enhanced_tacker_policy: True
vars:
config_enhanced_policy: true
- job:
name: tacker-functional-devstack-enhanced-policy-sol-kubernetes
parent: tacker-functional-devstack-multinode-sol-kubernetes-v2
description: |
Enhanced policy job for SOL Kubernetes devstack-based functional tests
host-vars:
controller-tacker:
tox_envlist: dsvm-functional-enhanced-policy-sol-kubernetes
devstack_local_conf:
post-config:
$TACKER_CONF:
oslo_policy:
enhanced_tacker_policy: True
vars:
config_enhanced_policy: true
- job:
name: tacker-functional-devstack-multinode-sol-terraform-v2
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL Terraform devstack-based functional tests
attempts: 1
host-vars:
controller-tacker:
tox_envlist: dsvm-functional-sol-terraform-v2
devstack_local_conf:
post-config:
$TACKER_CONF:
v2_vnfm:
tf_file_dir: /tmp/tacker/terraform
vars:
terraform_setup: true
- job:
name: tacker-compliance-devstack-multinode-sol
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
parent: tacker-functional-devstack-multinode-sol-parent
description: |
Multinodes job for SOL devstack-based compliance tests
host-vars:
controller-tacker:
tox_envlist: dsvm-compliance-sol-api
voting: false
- project:
templates:
- check-requirements
- openstack-cover-jobs
- openstack-python3-jobs
- publish-openstack-docs-pti
- release-notes-jobs-python3
check:
jobs:
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
- tacker-functional-devstack-multinode-sol-legacy-nfvo
- tacker-functional-devstack-multinode-sol-vnflcm
Fix broken jobs in V1 API gates This patch fixes the following issues: 1. Change the QueuePool settings to use the values recommended. The following error occurred in sol-vnflcm and sol-kubernetes job: ``` tacker.wsgi sqlalchemy.exc.TimeoutError: QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10.00 ``` Change to remove the process of overriding the default value of oslo.db config. In Neutron, the process of overriding the default value of oslo.db has already been removed [1]. However, errors still occur when using the recommended value, so temporarily set `max_pool_size` to unlimited in FT. 2. Fix issue of `Show subscription (V1 API)`. A previous patch [2] included `sqlalchemy.row.LegacyRow` for backwards compatibility with oslo.db, which caused the error "Show subscription (V1 API)" to fail by upgrading SQLAlchemy. Fix it to work correctly by removing `row.LegacyRow`. 3. Fix to upload image successfully in FT. There was a process to get the image from the glance and upload the copy image, but it was not working properly. Changed the argument to glance so that it uploads correctly. 4. Increase wait time Terminate after Heal in sol-vnflcm job. A 409 error occurred at the time of Terminate because the process was to wait a fixed number of seconds after Heal before Terminate. The current 20 seconds may not be enough, so increase it to 60 seconds. This will be fixed to check if OpOcc is COMPLETED status in future work. 5. Split test jobs for sol-vnflcm. Split test jobs for sol-vnflcm because it takes longer to run and, it is executed using the full limit time. [1] https://review.opendev.org/c/openstack/neutron/+/456419 [2] https://review.opendev.org/c/openstack/tacker/+/910720 Closes-Bug: #2066047 Change-Id: If1cd835c8875fa844f9eac5bff5ba85103ad91aa
2024-07-26 09:56:04 +00:00
- tacker-functional-devstack-multinode-sol-vnflcm-userdata
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
- tacker-functional-devstack-multinode-sol-vnfpkgm
- tacker-functional-devstack-multinode-sol-separated-nfvo
- tacker-functional-devstack-multinode-sol-kubernetes
Quick fix to migrate to SQLAlchemy 2.0 It's just a quick fix and intended to be not a drastic or complete update but step-by-step. Since the recent update of versions of sqlalchemy[1], several util features such as autocommit or subtransaction have been dropped, or some functions and attributes in alembic migration scripts called from `tacker-db-manage` haven't work anymore. All the things should be updated are described in the migration guide[2] and will completed later. This update is for following fixes, and including a fix of CI's update [3] which is required to this update. * Remove 'subtransaction=True' in `context.session.begin()`, or replace with 'nested=True' for subtransaction part itself. * Remove autocommit in `context_manager` and get_session(). * Add `commit()` to all sessions explicitly instead of dropping autocommit. `flush()` is also replaced with `commit()`. * In alembic migration scripts, change a way of creating session in a manner of SQLAlchemy 2.0. * Update `TackerBase` to add `__allow_unmapped__`. * Replace argument of `joinedload()` from string to model object because it's not supported in SQLAlchemy 2.0 anymore. * Reemove argument table's `insert()`. * Update usage of `orm.query.Query.join()` for SQLAlchemy 2.0. * Move to non-voting for unknown failures emerged recently. * tacker-functional-devstack-multinode-sol-vnflcm * tacker-functional-devstack-multinode-sol-kubernetes * tacker-functional-devstack-multinode-sol-v2-vnflcm * Skip functions for some failure of timeout cannot find out the cause and fix. * test_subscription_functionality in the job tacker-functional-devstack-multinode-sol-multi-tenant * test_inst_heal_term in the job tacker-functional-devstack-multinode-sol-separated-nfvo * Divide test scenarios under 'tacker/tests/functional/sol' and 'tacker/tests/functional/sol_v2' because the total time has become over the limitation. * Revise logging for better analysis. [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html [3] https://review.opendev.org/c/openstack/tacker/+/921528 Closes-Bug: #2066047 Change-Id: I8aec097da27b1c30f1291e5d10878ea07d26d007
2024-05-15 11:43:30 +00:00
- tacker-functional-devstack-multinode-sol-v2-basic
- tacker-functional-devstack-multinode-sol-v2-vnflcm
- tacker-functional-devstack-multinode-sol-v2-notification
- tacker-functional-devstack-multinode-sol-v2-prometheus
- tacker-functional-devstack-multinode-sol-separated-nfvo-v2
- tacker-functional-devstack-multinode-sol-v2-individual-vnfc-mgmt
- tacker-functional-devstack-multinode-sol-kubernetes-v2
- tacker-functional-devstack-multinode-sol-multi-tenant
- tacker-functional-devstack-multinode-sol-https-v2
- tacker-functional-devstack-multinode-sol-encrypt-cred-barbican
- tacker-functional-devstack-multinode-sol-encrypt-cred-local
- tacker-functional-devstack-multinode-sol-kubernetes-multi-tenant
- tacker-functional-devstack-kubernetes-oidc-auth
- tacker-functional-devstack-multinode-sol-v2-az-retry
- tacker-functional-devstack-enhanced-policy-sol
- tacker-functional-devstack-enhanced-policy-sol-kubernetes
- tacker-compliance-devstack-multinode-sol
- tacker-functional-devstack-multinode-sol-terraform-v2