Add support for OVS as a virtual switch

To support testing the upcoming standalone networking
feature it is necessary to have a virtual switch that
allows port VLAN configuration to be changed by the
networking generic switch driver.

Setting the test_vm_bridge_type to 'ovs' will create
3 separate VLANs for testing.  One will be dedicated
as an inspection network, another to the final
'tenant' network for the node, and the remaining one
will be used for all other network types (i.e.,
cleaning, rescuing, servicing, etc...).

Related-Bug: 2113769
Assisted-by: Claude Code/claude-sonnet-4
Change-Id: I54b154a28dcbb4f89b368deaa7c16792630f6564
Signed-off-by: Allain Legacy <alegacy@redhat.com>
This commit is contained in:
Allain Legacy
2025-09-12 10:16:17 -04:00
parent b7bb21c652
commit 71bebff81e
19 changed files with 396 additions and 6 deletions

View File

@@ -188,3 +188,17 @@ sushy-tools_ is also installed.
.. _VirtualBMC: https://docs.openstack.org/virtualbmc/
.. _sushy-tools: https://docs.openstack.org/sushy-tools/
Virtual Switching
-----------------
By default, Bifrost sets up a Linux bridge as the virtual switch
interconnecting the virtual machines that implement the nodes. To support
more complex test scenarios, it is possible to configure OVS as the virtual
switch. This enables updates to port VLAN assignments to test complex
networking scenarios.
The virtual switch type can be controlled by modifying the
``test_vm_switch_type`` variable via ansible extra vars supplied to the Ansible
commands or via bifrost-cli's ``-e`` option. Setting the variable to 'ovs'
enables the OVS switch type.

View File

@@ -12,6 +12,10 @@ The following packages are required and ensured to be present:
- qemu-kvm
- sgabios (except on CentOS Stream 10 / Rocky Linux 10)
Additional packages required when using test_vm_switch_type: 'ovs':
- openvswitch-switch (Debian/Ubuntu)
- openvswitch (RedHat/CentOS)
Warning
-------
@@ -150,6 +154,31 @@ test_vm_network_dhcp_end: End of DHCP range for 'test_vm_network'.
from scratch and when
'test_vm_network_enable_dhcp' is enabled.
test_vm_switch_type: Type of virtual switch to use for test VMs.
Defaults to 'linux_bridge'.
Set to 'ovs' to use Open vSwitch with VLAN support
for testing networking features.
test_ovs_bridge_name: Name of the OVS bridge to create when using
test_vm_switch_type: 'ovs'.
Defaults to 'brtest'.
test_ovs_host_vlans: List of VLAN IDs to configure on the OVS bridge.
Defaults to ['10', '20', '30'].
Creates separate VLANs for inspection, tenant, and
other network types (cleaning, rescuing, servicing).
VLAN IDs must be 1-255.
test_ovs_vm_initial_vlan: Initial VLAN ID for test VMs on OVS bridge.
Defaults to '10'.
VMs start on this VLAN and can be moved between
VLANs by the networking driver.
test_ovs_user: Username for OVS restricted user access.
Defaults to 'ovsuser'.
Uses SSH key-based authentication (password login is disabled).
Used for controlled VLAN management operations.
Dependencies
------------
@@ -158,6 +187,8 @@ None at this time.
Example Playbook
----------------
Basic usage with default Linux bridge:
- hosts: localhost
connection: local
become: yes
@@ -165,6 +196,19 @@ Example Playbook
roles:
- role: bifrost-create-vm-nodes
Using Open vSwitch for testing standalone networking features:
- hosts: localhost
connection: local
become: yes
gather_facts: yes
roles:
- role: bifrost-create-vm-nodes
vars:
test_vm_switch_type: ovs
test_ovs_host_vlans: ['10', '20', '30']
test_ovs_vm_initial_vlan: '10'
License
-------

View File

@@ -97,3 +97,18 @@ efi_nvram_locations_secboot:
- /usr/share/OVMF/OVMF_VARS.secboot.fd
efi_nvram_locations: >-
{{ efi_nvram_locations_secboot if test_vm_secure_boot | bool else efi_nvram_locations_normal }}
# Switch type configuration (default: linux_bridge)
test_vm_switch_type: linux_bridge
# OVS-specific configuration
test_ovs_bridge_name: brtest
# Simple VLAN configuration
# NOTE: VLAN IDs must be 1-255 when used for IP subnets (192.168.{VLAN}.0/24)
test_ovs_host_vlans: ['10', '20', '30']
test_ovs_vm_initial_vlan: '10'
# OVS restricted user configuration
# Uses SSH key-based authentication (password authentication is disabled)
test_ovs_user: ovsuser

View File

@@ -0,0 +1,17 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
- name: Restart sshd
systemd:
name: sshd
state: restarted

View File

@@ -19,6 +19,7 @@
vm_name: "{{ item }}"
vm_log_file: "{{ test_vm_logdir }}/{{ item }}_console.log"
vm_host_group: "{{ test_vm_default_groups }}"
vm_port_name: "{{ item }}-port0"
- set_fact:
vm_host_group: "{{ test_vm_default_groups | union(test_vm_groups[vm_name]) }}"
@@ -41,6 +42,28 @@
command: list_vms
register: existing_vms
# Create OVS port with VLAN tag for VM when using OVS
- name: check if OVS port already exists
shell:
cmd: |
set -eo pipefail
ovs-vsctl list-ports {{ test_ovs_bridge_name }} | grep -x {{ vm_port_name }} || echo "not_found"
register: ovs_port_check
when: test_vm_switch_type == 'ovs'
- name: create OVS port with VLAN tag
shell: |
ovs-vsctl add-port {{ test_ovs_bridge_name }} {{ vm_port_name }} tag={{ test_ovs_vm_initial_vlan }} -- set interface {{ vm_port_name }} type=internal
when:
- test_vm_switch_type == 'ovs'
- ovs_port_check.stdout.strip() == "not_found"
- name: configure OVS linux interface
shell: |
ovs-vsctl set interface {{ vm_port_name }} lldp:enable=true
ip link set {{ vm_port_name }} up
when: test_vm_switch_type == 'ovs'
# NOTE(pas-ha) wrapping in block/rescue to have diagnostic output, requires Ansible>=2
- when: vm_name not in existing_vms.list_vms
block:
@@ -129,6 +152,20 @@
set_fact:
vm_mac: "{{ (testvm_xml.get_xml | regex_findall(\"<mac address='.*'/>\") | first).split('=') | last | regex_replace(\"['/>]\", '') }}"
- name: set VM network configuration for OVS
set_fact:
vm_network_base: "192.168.{{ 100 + test_ovs_vm_initial_vlan | int }}."
vm_ip_offset: "{{ 2 + (testvm_json_data | length) }}"
mgmt_network_ip: "192.168.{{ 100 + test_ovs_vm_initial_vlan | int }}.1"
when: test_vm_switch_type == 'ovs'
- name: set VM network configuration for bridge
set_fact:
vm_network_base: "192.168.122."
vm_ip_offset: "{{ 2 + (testvm_json_data | length) }}"
mgmt_network_ip: "192.168.122.1"
when: test_vm_switch_type == 'linux_bridge'
# NOTE(pas-ha) using default username and password set by virtualbmc - "admin" and "password" respectively
# see vbmc add --help
- name: set the json entry for vm
@@ -139,7 +176,7 @@
host_groups: "{{ vm_host_group }}"
driver: "{{ test_vm_node_driver }}"
driver_info:
ipmi_address: "192.168.122.1"
ipmi_address: "{{ mgmt_network_ip }}"
ipmi_port: "{{ virtual_ipmi_port }}"
ipmi_username: "admin"
ipmi_password: "password"
@@ -149,8 +186,8 @@
redfish_password: "password"
nics:
- mac: "{{ vm_mac }}"
ansible_ssh_host: "192.168.122.{{ testvm_json_data | length + 2 }}"
ipv4_address: "192.168.122.{{ testvm_json_data | length + 2 }}"
ansible_ssh_host: "{{ vm_network_base }}{{ vm_ip_offset }}"
ipv4_address: "{{ vm_network_base }}{{ vm_ip_offset }}"
properties:
cpu_arch: "{{ test_vm_arch }}"
ram: "{{ test_vm_memory_size }}"
@@ -161,7 +198,7 @@
uuid: "{{ vm_name | to_uuid }}"
driver: "{{ test_vm_node_driver }}"
driver_info:
ipmi_address: "192.168.122.1"
ipmi_address: "{{ mgmt_network_ip }}"
ipmi_port: "{{ virtual_ipmi_port }}"
ipmi_username: "admin"
ipmi_password: "password"

View File

@@ -85,6 +85,9 @@
group: "{{ ansible_user_gid }}"
when: copy_from_local_path | bool
- import_tasks: prepare_ovs.yml
when: test_vm_switch_type == 'ovs'
- import_tasks: prepare_libvirt.yml
- name: truncate explicit list of vm names

View File

@@ -104,7 +104,7 @@
virt_net:
name: "{{ test_vm_network }}"
state: present
xml: "{{ lookup('template', 'net.xml.j2') }}"
xml: "{{ lookup('template', 'ovs-net.xml.j2' if test_vm_switch_type == 'ovs' else 'net.xml.j2') }}"
uri: "{{ test_vm_libvirt_uri }}"
- name: find facts on libvirt networks

View File

@@ -0,0 +1,120 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Setup OVS bridge with VLAN interfaces and DHCP services
---
- name: enable NFV repository for OVS on CentOS Stream 10
package:
name: centos-release-nfv-openvswitch
state: present
when:
- ansible_distribution == "CentOS"
- ansible_distribution_major_version|int >= 10
- name: install OVS packages
package:
name: "{{ ovs_packages }}"
state: present
- name: ensure OVS services are started and enabled
systemd:
name: "{{ ovs_service_name }}"
state: started
enabled: yes
- name: create OVS bridge
openvswitch.openvswitch.openvswitch_bridge:
bridge: "{{ test_ovs_bridge_name }}"
state: present
- name: bring up OVS bridge
command: ip link set {{ test_ovs_bridge_name }} up
- name: create VLAN interfaces on OVS bridge
shell: |
ovs-vsctl add-port {{ test_ovs_bridge_name }} {{ test_ovs_bridge_name }}.{{ item }} tag={{ item }} -- set interface {{ test_ovs_bridge_name }}.{{ item }} type=internal
ip addr add 192.168.{{ 100 + item | int }}.1/24 dev {{ test_ovs_bridge_name }}.{{ item }}
ip link set {{ test_ovs_bridge_name }}.{{ item }} up
loop: "{{ test_ovs_host_vlans }}"
ignore_errors: yes
- name: enable IP forwarding for OVS bridge
sysctl:
name: "net.ipv4.ip_forward"
value: 1
sysctl_set: yes
state: present
reload: yes
- name: ensure .ssh directory exists for OVS user
file:
path: /home/{{ test_ovs_user }}/.ssh
state: directory
owner: "{{ test_ovs_user }}"
mode: '0700'
- name: create OVS user
user:
name: "{{ test_ovs_user }}"
password: '!' # Disabled password
shell: /bin/bash
home: /home/{{ test_ovs_user }}
create_home: yes
groups: openvswitch
state: present
- name: generate SSH key pair for OVS user
user:
name: "{{ test_ovs_user }}"
generate_ssh_key: yes
ssh_key_type: ed25519
ssh_key_file: .ssh/id_ed25519
- name: read OVS user public key
slurp:
src: /home/{{ test_ovs_user }}/.ssh/id_ed25519.pub
register: ovs_user_pubkey
- name: add public key to authorized_keys for OVS user
authorized_key:
user: "{{ test_ovs_user }}"
key: "{{ ovs_user_pubkey['content'] | b64decode }}"
state: present
- name: set OVS socket group permissions
file:
path: /var/run/openvswitch/db.sock
group: openvswitch
mode: '0660'
# TODO(alegacy): this could be refined so that access is restricted to a
# specific set of OVS commands only using something like rbash
- name: add OVS user to sudoers for privileged access
copy:
dest: /etc/sudoers.d/{{ test_ovs_user }}-ovs
mode: '0440'
content: |
# Allow {{ test_ovs_user }} to run OVS commands as root without password
{{ test_ovs_user }} ALL=(ALL) NOPASSWD: /bin/bash
- name: Restrict OVS user SSH access from localhost only and disable password auth
ansible.builtin.blockinfile:
path: /etc/ssh/sshd_config
block: |
Match User {{ test_ovs_user }}
AllowUsers {{ test_ovs_user }}@localhost {{ test_ovs_user }}@127.0.0.1 {{ test_ovs_user }}@::1
PasswordAuthentication no
PubkeyAuthentication yes
marker: "# {mark} ANSIBLE MANAGED BLOCK FOR {{ test_ovs_user }}"
validate: 'sshd -t -f %s'
notify: Restart sshd

View File

@@ -0,0 +1,6 @@
<network>
<name>{{ test_vm_network }}</name>
<forward mode='bridge'/>
<bridge name='{{ test_ovs_bridge_name }}'/>
<virtualport type='openvswitch'/>
</network>

View File

@@ -36,6 +36,16 @@
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
<controller type='sata' index='0'/>
{% if test_vm_switch_type == 'ovs' %}
<interface type='direct'>
<source dev='{{ vm_port_name }}' mode='passthrough'/>
<virtualport type='openvswitch'/>
<model type='{{ test_vm_nic }}'/>
{% if default_boot_mode == 'uefi' %}
<boot order='1'/>
{% endif %}
</interface>
{% else %}
<interface type='network'>
<source network='{{ test_vm_network }}'/>
<model type='{{ test_vm_nic }}'/>
@@ -43,6 +53,7 @@
<boot order='1'/>
{% endif %}
</interface>
{% endif %}
<input type='mouse' bus='ps2'/>
<serial type='file'>
<source path='{{ vm_log_file }}'/>

View File

@@ -15,3 +15,8 @@ required_packages:
- ovmf
- ebtables
- dnsmasq
ovs_packages:
- openvswitch-switch
- python3-openvswitch
ovs_service_name: openvswitch-switch

View File

@@ -16,4 +16,11 @@ required_packages:
- libxslt-devel
- libxml2-devel
- edk2-ovmf
ovs_packages:
# TODO(alegacy): Update versioned package names periodically
- openvswitch3.5
- python3-openvswitch3.5
ovs_service_name: openvswitch
test_vm_emulator: "/usr/libexec/qemu-kvm"

View File

@@ -409,6 +409,10 @@ keystone:
# Timeout for gathering facts.
fact_gather_timeout: "{{ lookup('config', 'DEFAULT_GATHER_TIMEOUT', on_missing='skip') | default(omit, true) }}"
# Custom switch support (duplicated from create-vm-nodes)
test_vm_switch_type: linux_bridge
test_ovs_bridge_name: brtest
# Enable TLS support.
enable_tls: false
vmedia_enable_tls: "{{ enable_tls }}"

View File

@@ -297,13 +297,21 @@
itf_infos: "{{ internal_interface }}"
dhcp_netaddr: "{{ dhcp_pool_start }}/{{ dhcp_static_mask }}"
when: enable_dhcp | bool
- name: "Compute interface and DHCP network information"
set_fact:
itf_netaddr1: "{{ itf_infos['address'] }}/{{ itf_infos['netmask'] }}"
itf_netaddr2: "{{ itf_infos['network'] }}/{{ itf_infos['netmask'] }}"
itf_broadcast: "{{ itf_infos['broadcast'] }}/{{ itf_infos['netmask'] }}"
dhcp_netaddr: "{{ dhcp_netaddr | ansible.utils.ipaddr('network') }}/{{ dhcp_static_mask }}"
when: enable_dhcp | bool
- name: "Compute broadcast address for interface"
set_fact:
# NOTE: VLAN interfaces (e.g., brtest.10) may have empty broadcast field in Ansible facts.
# If broadcast is empty, compute it from the address/netmask to avoid validation failures.
itf_broadcast: "{{ (itf_infos['broadcast'] | length > 0) | ternary(itf_infos['broadcast'] + '/' + itf_infos['netmask'], itf_netaddr1 | ansible.utils.ipaddr('broadcast') + '/' + itf_infos['netmask']) }}"
when: enable_dhcp | bool
- name: "Validate interface network addresses"
fail:
msg: >
@@ -313,6 +321,7 @@
when:
- enable_dhcp | bool
- itf_netaddr1 | ansible.utils.ipaddr('network') != itf_netaddr2 | ansible.utils.ipaddr('network')
- name: "Validate interface broadcast addresses"
fail:
msg: >
@@ -322,6 +331,7 @@
when:
- enable_dhcp | bool
- itf_netaddr1 | ansible.utils.ipaddr('broadcast') != itf_broadcast | ansible.utils.ipaddr('broadcast')
- name: "Validate DHCP and interface addresses"
debug:
msg: >
@@ -332,6 +342,7 @@
when:
- enable_dhcp | bool
- itf_netaddr2 | ansible.utils.ipaddr('network') != dhcp_netaddr | ansible.utils.ipaddr('network')
- name: "Computing new DHCP information"
set_fact:
dhcp_start_ip: "{{ dhcp_pool_start.split('.')[-1] }}"
@@ -340,6 +351,7 @@
when:
- enable_dhcp | bool
- itf_netaddr2 | ansible.utils.ipaddr('network') != dhcp_netaddr | ansible.utils.ipaddr('network')
# Note(olivierbourdon38): we could do much more complex network
# computation to derive exact (or way closer to exact) range for
# the new network depending on netmasks and indexes.

View File

@@ -34,6 +34,13 @@
include_tasks: bootstrap.yml
when: not skip_bootstrap | bool
- name: "Bootstrap OVS"
include_tasks: ovs_bootstrap.yml
when:
- not skip_bootstrap | bool
- testing | bool
- test_vm_switch_type == 'ovs'
- name: "Start Ironic services"
include_tasks: start.yml
when: not skip_start | bool

View File

@@ -0,0 +1,62 @@
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
- name: "Collect configured OVS VLANs from the bridge"
shell:
cmd: |
set -eo pipefail
ovs-vsctl list-ports {{ test_ovs_bridge_name }} | grep -E '\.([0-9]+)$' | sed 's/.*\.\([0-9]\+\)$/\1/' | sort -n
register: ovs_configured_vlans
when: enable_dhcp | bool
failed_when: false
changed_when: false
- name: "Set variable with configured OVS VLAN IDs"
set_fact:
ovs_vlan_ids: "{{ ovs_configured_vlans.stdout_lines | default([]) }}"
when: enable_dhcp | bool
- name: "Configure OVS VLAN DHCP in main dnsmasq"
template:
src: ovs-vlans-dhcp.conf.j2
dest: "/etc/dnsmasq.d/ovs-vlans.conf"
mode: "0644"
when: enable_dhcp | bool
- name: "Get OVS ports with Linux interfaces"
shell:
cmd: |
set -eo pipefail
for port in $(ovs-vsctl list-ports {{ test_ovs_bridge_name }}); do
if ovs-vsctl get interface $port type 2>/dev/null | grep -q "internal\|\"\""; then
if ip link show $port >/dev/null 2>&1; then
echo $port
fi
fi
done
register: ovs_linux_interfaces
when: use_firewalld | bool
- name: "Add OVS Linux interfaces to firewall zone"
firewalld:
zone: "{{ 'libvirt' if testing | bool else firewalld_internal_zone }}"
interface: "{{ item }}"
state: enabled
permanent: yes
immediate: yes
loop: "{{ ovs_linux_interfaces.stdout_lines | default([]) }}"
when:
- use_firewalld | bool
- ovs_linux_interfaces.stdout_lines is defined
- ovs_linux_interfaces.stdout_lines | length > 0

View File

@@ -13,7 +13,9 @@ port=53
port=0
{% endif %}
{% if test_vm_switch_type == 'linux_bridge' %}
listen-address={{ internal_ip }}
{% endif %}
# On systems which support it, dnsmasq binds the wildcard address,
# even when it is listening on only some interfaces. It then discards
@@ -61,6 +63,7 @@ domain={{ domain }}
# a lease time. If you have more than one network, you will need to
# repeat this for each network on which you want to supply DHCP
# service.
{% if test_vm_switch_type == 'linux_bridge' %}
{% if testing | bool == true %}
dhcp-range=192.168.122.2,192.168.122.254,12h
{% elif inventory_dhcp | bool == true %}
@@ -68,6 +71,7 @@ dhcp-range={{dhcp_pool_start}},static,{{dhcp_static_mask}},{{dhcp_lease_time}}
{% else %}
dhcp-range={{dhcp_pool_start}},{{dhcp_pool_end}},{% if dhcp_pool_mask is defined %}{{dhcp_pool_mask}},{% endif %}{{dhcp_lease_time}}
{% endif %}
{% endif %}
# Override the default route supplied by dnsmasq, which assumes the
# router is the same machine as the one running dnsmasq.

View File

@@ -0,0 +1,12 @@
# OVS VLAN DHCP configuration for main dnsmasq
# Generated by bifrost-ironic-install role during installation
{% for vlan in ovs_vlan_ids | default([]) %}
{% set subnet_id = 100 + vlan|int %}
# DHCP configuration for VLAN {{ vlan }} (subnet 192.168.{{ subnet_id }}.0/24)
interface={{ test_ovs_bridge_name }}.{{ vlan }}
dhcp-range=tag:{{ test_ovs_bridge_name }}.{{ vlan }},192.168.{{ subnet_id }}.10,192.168.{{ subnet_id }}.100,255.255.255.0,12h
dhcp-option=tag:{{ test_ovs_bridge_name }}.{{ vlan }},3,192.168.{{ subnet_id }}.1
dhcp-option=tag:{{ test_ovs_bridge_name }}.{{ vlan }},6,192.168.{{ subnet_id }}.1
{% endfor %}

View File

@@ -0,0 +1,10 @@
---
features:
- |
Adds support for using Open vSwitch (OVS) as a virtual switch for testing
environments. Setting ``test_vm_switch_type`` to ``ovs`` creates 3 separate
VLANs for comprehensive network testing: one dedicated as an inspection
network, another for the final tenant network, and a third for all other
network types (cleaning, rescuing, servicing, etc.). This enhancement
enables testing of the standalone networking feature with proper VLAN
configuration support through the networking generic switch driver.