Add support for Gateway API

We recently introduced extraObjects section for
all chart's values. This makes it possible to
deploy arbitrary K8s objects as part of a Helm release.

The Openstack-Helm legacy ingress implementation
is very opinionated and we don't want to do the same for
Gateway API objects. Instead users are supposed to
add Gateway API objects (Gateway, HTTPRoute, etc.) manually
using this extraObjects value.

The range of infrastructure use cases is very wide
and it is better to let users to choose on their own
how they are going to provide access to deployed Openstack services.

Regarding this PS:

* Add tasks to the deploy-env role to deploy Envoy Gateway on
  test env.
* Add gateway.yaml overrides for Keystone, Nova, Neutron, Placement, Glance,
  Heat. These overrides disable rendering Ingress related
  objects and add HTTPRoute for public endpoints.

Later gateway overrides will be added for other charts and
deployment scripts will be updated appropriately

Signed-off-by: Vladimir Kozhukalov <kozhukalov@gmail.com>
Change-Id: I8043206136bf6513e2ba2b978510662b655a368f

doc/source/install/openstack.rst

@@ -12,12 +12,32 @@ At this point we assume all the prerequisites listed below are met:
 - The OpenStack-Helm repositories are enabled, OpenStack-Helm
   plugin is installed and necessary environment variables are set.
 - The ``openstack`` namespace is created.
-- Ingress controller is deployed in the ``openstack`` namespace.
-- MetalLB is deployed and configured. The service of type
-  ``LoadBalancer`` is created and DNS is configured to resolve the
-  Openstack endpoint names to the IP address of the service.
+- MetalLB is deployed. A Gateway API controller is installed and a
+  ``Gateway`` resource is created. The controller will provision a
+  ``LoadBalancer`` service automatically. DNS is configured to resolve
+  the OpenStack endpoint names to the external IP of that service.
+  We recommend `Envoy Gateway`_ as the Gateway API implementation.
 - Ceph is deployed and enabled for using by OpenStack-Helm.
 
+.. _Envoy Gateway: https://gateway.envoyproxy.io/
+
+.. note::
+
+    The recommended way to expose OpenStack services externally is through
+    the `Kubernetes Gateway API`_. The gateway controller (e.g. Envoy Gateway)
+    is deployed in its own namespace (``envoy-gateway-system`` for Envoy Gateway)
+    and creates a ``LoadBalancer`` service backed by MetalLB. Traffic is routed
+    to backend services via ``HTTPRoute`` resources.
+
+    How exactly users expose their workloads may vary. ``HTTPRoute`` objects
+    and additional ``Service`` resources can be added to any chart via the
+    ``.Values.extraObjects`` field available in all OpenStack-Helm charts.
+    For an example see ``values_overrides/nova/gateway.yaml``.
+
+    Legacy ``Ingress`` resources are still supported.
+
+.. _Kubernetes Gateway API: https://gateway-api.sigs.k8s.io/
+
 .. _kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/
 .. _helm: https://helm.sh/docs/intro/install/
 

doc/source/install/prerequisites.rst

@@ -1,69 +1,106 @@
 Kubernetes prerequisites
 ========================
 
-Ingress controller
-------------------
+Gateway API
+-----------
 
-Ingress controller when deploying OpenStack on Kubernetes
-is essential to ensure proper external access for the OpenStack services.
+The `Kubernetes Gateway API`_ is the recommended way to expose OpenStack
+services externally. It provides an expressive and extensible routing model.
+We recommend using `Envoy Gateway`_ as the Gateway API implementation.
 
-We recommend using the `haproxy-ingress`_ because it is simple and provides
-all necessary features. It utilizes HAProxy as a reverse proxy backend.
-Here is how to deploy it.
+Below we describe how we deploy the Gateway API and Envoy Gateway in the cluster on
+test clusters.
 
-First, let's create a namespace for the OpenStack workloads. The ingress
-controller must be deployed in the same namespace because OpenStack-Helm charts
-create service resources pointing to the ingress controller pods which
-in turn redirect traffic to particular Openstack API pods.
+First, install the Gateway API CRDs and `Envoy Gateway`_:
 
 .. code-block:: bash
 
-    tee > /tmp/openstack_namespace.yaml <<EOF
-    apiVersion: v1
-    kind: Namespace
+    helm install eg oci://docker.io/envoyproxy/gateway-helm \
+      --version v1.7.0 \
+      --namespace envoy-gateway-system \
+      --create-namespace
+
+Next, create the ``EnvoyProxy`` custom resource, the ``Gateway``.
+The ``EnvoyProxy`` tells Envoy Gateway
+how to configure the data-plane pods and the ``LoadBalancer`` service
+(backed by MetalLB, see below). The ``Gateway`` defines the listener that
+accepts traffic. The gateway controller will automatically create a
+``LoadBalancer`` service.
+
+.. code-block:: bash
+
+    GATEWAY_IP=<metallb_ip_for_gateway>
+
+    tee > /tmp/gatewayapi_envoy_default.yaml <<EOF
+    ---
+    apiVersion: gateway.envoyproxy.io/v1alpha1
+    kind: EnvoyProxy
     metadata:
-      name: openstack
+      name: gateway-proxy-default
+      namespace: envoy-gateway-system
+    spec:
+      provider:
+        type: Kubernetes
+        kubernetes:
+          envoyService:
+            type: LoadBalancer
+            externalTrafficPolicy: Cluster
+            annotations:
+              metallb.universe.tf/loadBalancerIPs: "${GATEWAY_IP}"
+            patch:
+              type: StrategicMerge
+              value:
+                spec:
+                  externalTrafficPolicy: Cluster
+    ---
+    apiVersion: gateway.networking.k8s.io/v1
+    kind: Gateway
+    metadata:
+      name: gateway-default
+      namespace: envoy-gateway-system
+    spec:
+      gatewayClassName: default
+      infrastructure:
+        parametersRef:
+          group: gateway.envoyproxy.io
+          kind: EnvoyProxy
+          name: gateway-proxy-default
+      listeners:
+        - name: http
+          protocol: HTTP
+          port: 80
+          allowedRoutes:
+            namespaces:
+              from: All
     EOF
-    kubectl apply -f /tmp/openstack_namespace.yaml
+    kubectl apply -f /tmp/gatewayapi_envoy_default.yaml
 
-Next, deploy the ingress controller in the ``openstack`` namespace:
+Kubernetes workloads that need to reach public OpenStack endpoints
+(e.g. Octavia workers calling the Nova API) send requests to the cluster
+Coredns service which by default forwards requests to the DNS server
+configured in the /etc/resolv.conf file on the cluster nodes. We configure
+the cluster nodes to use the Dnsmasq running on the control plane node
+as the default nameserver. The Dnsmasq is configured to resolve
+the Openstack public names to the LB IP.
 
-.. code-block:: bash
+How exactly users expose their workloads may vary. ``HTTPRoute`` objects
+can be added to any OpenStack-Helm chart via
+the ``.Values.extraObjects`` field. For a complete example see
+``values_overrides/nova/gateway.yaml``.
 
-    helm repo add haproxy-ingress https://haproxy-ingress.github.io/charts
-    helm upgrade --install haproxy-ingress haproxy-ingress/haproxy-ingress \
-      --version 0.15.0 \
-      --namespace=openstack \
-      --set controller.kind=Deployment \
-      --set controller.ingressClassResource.enabled="true" \
-      --set controller.ingressClass=ingress-openstack \
-      --set controller.podLabels.app=ingress-api
+.. _Kubernetes Gateway API: https://gateway-api.sigs.k8s.io/
+.. _Envoy Gateway: https://gateway.envoyproxy.io/
 
-You can deploy any other ingress controller that suits your needs best.
-See for example the list of available `ingress controllers`_.
-Ensure that the ingress controller pods are deployed with the ``app: ingress-api``
-label which is used by the OpenStack-Helm as a selector for the Kubernetes
-service resources.
-
-For example, the OpenStack-Helm ``keystone`` chart by default creates a service
-that redirects traffic to the ingress controller pods selected using the
-``app: ingress-api`` label. Then it also creates an ``Ingress`` resource which
-the ingress controller then uses to configure its reverse proxy
-backend (HAProxy) which eventually routes the traffic to the Keystone API
-service which works as an endpoint for Keystone API pods.
-
-.. image:: ingress.jpg
+.. image:: OSH_GatewayAPI.svg
     :width: 100%
     :align: center
-    :alt: ingress scheme
+    :alt: Gateway API traffic flow
 
 .. note::
-    For exposing the OpenStack services to the external world, we can a create
-    service of type ``LoadBalancer`` or ``NodePort`` with the selector pointing to
-    the ingress controller pods.
+    Legacy ``Ingress`` resources are still supported. If you prefer to use an
+    ingress controller instead of the Gateway API, deploy one in the
+    ``openstack`` namespace with pods labeled ``app: ingress-api``.
 
-.. _haproxy-ingress: https://haproxy-ingress.github.io/
-.. _ingress controllers: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
 
 MetalLB
 -------

neutron/values.yaml

@@ -297,7 +297,7 @@ dependencies:
           service: network
         - endpoint: internal
           service: compute
-        - endpoint: public
+        - endpoint: internal
           service: compute_metadata
     ovn_metadata:
       pod:

roles/deploy-env/defaults/main.yaml

@@ -27,6 +27,10 @@ cilium_version: "1.17.4"
 flannel_setup: false
 flannel_version: v0.26.7
 
+gatewayapi_setup: true
+gatewayapi_implementation: "envoy"  # options: envoy
+gatewayapi_envoy_version: "v1.7.0"
+
 ingress_setup: false
 ingress_implementation: "haproxy"  # options: haproxy, nginx
 ingress_nginx_version: "4.12.2"
@@ -71,16 +75,21 @@ coredns_extra_config: ""
 metallb_setup: true
 metallb_version: "0.15.3"
 metallb_pool_cidr: "172.24.128.0/24"
-metallb_openstack_endpoint_cidr: "172.24.128.100/24"
-metallb_osh_infra_endpoint_cidr: "172.24.128.101/24"
+metallb_ingress_openstack_endpoint_cidr: "172.24.128.100/24"
+metallb_ingress_osh_infra_endpoint_cidr: "172.24.128.101/24"
+metallb_gatewayapi_endpoint_cidr: "172.24.128.102/24"
 
 client_cluster_ssh_setup: true
 client_ssh_user: zuul
 cluster_ssh_user: zuul
 
-openstack_provider_gateway_setup: false
-openstack_provider_network_cidr: "172.24.4.0/24"
-openstack_provider_gateway_cidr: "172.24.4.1/24"
+floating_network_setup: false
+floating_network_cidr: "172.24.4.0/24"
+floating_network_gateway_cidr: "172.24.4.1/24"
+
+tcpproxy_cidr: "172.24.6.0/24"
+tcpproxy_gatewayapi_cidr: "172.24.6.1/24"
+tcpproxy_ingress_openstack_cidr: "172.24.6.2/24"
 
 tunnel_network_cidr: "172.24.5.0/24"
 tunnel_client_cidr: "172.24.5.2/24"

roles/deploy-env/files/calico_patch.yaml

@@ -8,5 +8,5 @@ spec:
             # we need Calico to skip this interface while discovering the
             # network changes on the host to prevent announcing unnecessary networks.
             - name: IP_AUTODETECTION_METHOD
-              value: "skip-interface=br-ex|provider.*|client.*|o-hm.*|o-w.*"
+              value: "skip-interface=br-ex|tcpproxy.*|provider.*|client.*|o-hm.*|o-w.*"
 ...

roles/deploy-env/files/nginx_tcp_proxy.conf

@@ -11,15 +11,29 @@ events {
 stream {
     access_log off;
 
+    # Ingress controller proxy (for services not using Gateway API)
     server {
-        listen {{ openstack_provider_gateway_cidr | ipaddr('address') }}:80;
-        proxy_pass {{ metallb_openstack_endpoint_cidr | ipaddr('address') }}:80;
-        proxy_bind {{ openstack_provider_gateway_cidr | ipaddr('address') }} transparent;
+        listen {{ tcpproxy_ingress_openstack_cidr | ipaddr('address') }}:80;
+        proxy_pass {{ metallb_ingress_openstack_endpoint_cidr | ipaddr('address') }}:80;
+        proxy_bind {{ tcpproxy_ingress_openstack_cidr | ipaddr('address') }} transparent;
     }
 
     server {
-        listen {{ openstack_provider_gateway_cidr | ipaddr('address') }}:443;
-        proxy_pass {{ metallb_openstack_endpoint_cidr | ipaddr('address') }}:443;
-        proxy_bind {{ openstack_provider_gateway_cidr | ipaddr('address') }} transparent;
+        listen {{ tcpproxy_ingress_openstack_cidr | ipaddr('address') }}:443;
+        proxy_pass {{ metallb_ingress_openstack_endpoint_cidr | ipaddr('address') }}:443;
+        proxy_bind {{ tcpproxy_ingress_openstack_cidr | ipaddr('address') }} transparent;
+    }
+
+    # Gateway API proxy (for services using HTTPRoute)
+    server {
+        listen {{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}:80;
+        proxy_pass {{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}:80;
+        proxy_bind {{ tcpproxy_gatewayapi_cidr | ipaddr('address') }} transparent;
+    }
+
+    server {
+        listen {{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}:443;
+        proxy_pass {{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}:443;
+        proxy_bind {{ tcpproxy_gatewayapi_cidr | ipaddr('address') }} transparent;
     }
 }

roles/deploy-env/tasks/floating_network.yaml

@@ -20,15 +20,23 @@
   shell: |
     ip tuntap add name provider1 mode tap
     ip link set provider1 up
-    ip addr add {{ openstack_provider_gateway_cidr }} dev provider1
+    ip addr add {{ floating_network_gateway_cidr }} dev provider1
 
 - name: Set up SNAT for packets going outside the cluster
   shell: |
-    iptables -t nat -A POSTROUTING -o {{ cluster_default_dev }} -s {{ openstack_provider_network_cidr }} -j MASQUERADE
+    iptables -t nat -A POSTROUTING -o {{ cluster_default_dev }} -s {{ floating_network_cidr }} -j MASQUERADE
 
 - name: Set up FORWARD for packets going from VMs
   shell: |
-    iptables -t filter -I FORWARD -s {{ openstack_provider_network_cidr }} -j ACCEPT
+    iptables -t filter -I FORWARD -s {{ floating_network_cidr }} -j ACCEPT
+
+
+- name: Set up tcp proxy TAP interfaces on cluster control-plane node
+  shell: |
+    ip tuntap add name tcpproxy1 mode tap
+    ip link set tcpproxy1 up
+    ip addr add {{ tcpproxy_gatewayapi_cidr }} dev tcpproxy1
+    ip addr add {{ tcpproxy_ingress_openstack_cidr }} dev tcpproxy1
 
 # We use tcp proxy to forward traffic to make it possible to connect
 # to the Openstack public endpoint (managed by Metallb) from VMs.
@@ -73,8 +81,19 @@
       --keep-in-foreground
       --no-hosts
       --bind-interfaces
-      --address="/openstack.svc.cluster.local/{{ openstack_provider_gateway_cidr | ipaddr('address') }}"
-      --listen-address="{{ openstack_provider_gateway_cidr | ipaddr('address') }}"
+      --address="/keystone.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/placement.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/glance.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/nova.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/metadata.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/novncproxy.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/serialproxy.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/spiceproxy.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/neutron.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/heat.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/cloudformation.openstack-helm.org/{{ tcpproxy_gatewayapi_cidr | ipaddr('address') }}"
+      --address="/openstack.svc.cluster.local/{{ tcpproxy_ingress_openstack_cidr | ipaddr('address') }}"
+      --listen-address="{{ floating_network_gateway_cidr | ipaddr('address') }}"
       --no-resolv
       --server={{ dnsmasq_dns_server }}
       {{ dnsmasq_extra_args | default('') }}

roles/deploy-env/tasks/gatewayapi_envoy.yaml

@@ -0,0 +1,44 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+---
+- name: Deploy Envoy Gateway
+  become: false
+  block:
+    - name: Install Envoy Gateway
+      become_user: "{{ kubectl.user }}"
+      shell: |
+        helm upgrade --install envoy-gateway oci://docker.io/envoyproxy/gateway-helm \
+          --version {{ gatewayapi_envoy_version }} \
+          --namespace envoy-gateway-system \
+          --create-namespace \
+          --wait
+
+    - name: Sleep before checking Envoy Gateway pods
+      pause:
+        seconds: 30
+
+    - name: Wait for Envoy Gateway pods to be ready
+      command: kubectl -n envoy-gateway-system wait --timeout=5m --for=condition=Available deployment/envoy-gateway
+
+    - name: Create GatewayClass for Envoy Gateway
+      shell: |
+        tee > /tmp/gatewayapi_envoy_class.yaml <<EOF
+        ---
+        apiVersion: gateway.networking.k8s.io/v1
+        kind: GatewayClass
+        metadata:
+          name: default
+        spec:
+          controllerName: gateway.envoyproxy.io/gatewayclass-controller
+        EOF
+        kubectl apply -f /tmp/gatewayapi_envoy_class.yaml

roles/deploy-env/tasks/ingress_metallb_endpoint.yaml

@@ -1,113 +0,0 @@
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
----
-- name: Create openstack ingress service
-  become: false
-  shell: |
-    tee > /tmp/openstack_endpoint_service.yaml <<EOF
-    ---
-    apiVersion: v1
-    kind: Namespace
-    metadata:
-      labels:
-        kubernetes.io/metadata.name: openstack
-        name: openstack
-      name: openstack
-    ---
-    kind: Service
-    apiVersion: v1
-    metadata:
-      name: public-openstack
-      namespace: openstack
-      annotations:
-        metallb.universe.tf/loadBalancerIPs: "{{ metallb_openstack_endpoint_cidr | ipaddr('address') }}"
-    spec:
-      externalTrafficPolicy: Cluster
-      type: LoadBalancer
-      selector:
-        app: ingress-api
-      ports:
-        - name: http
-          port: 80
-        - name: https
-          port: 443
-    EOF
-    kubectl apply -f /tmp/openstack_endpoint_service.yaml
-
-- name: Create openstack ingress service
-  when: ingress_osh_infra_setup
-  become: false
-  shell: |
-    tee > /tmp/osh_infra_endpoint_service.yaml <<EOF
-    ---
-    apiVersion: v1
-    kind: Namespace
-    metadata:
-      labels:
-        kubernetes.io/metadata.name: osh-infra
-        name: osh-infra
-      name: osh-infra
-    ---
-    kind: Service
-    apiVersion: v1
-    metadata:
-      name: public-osh-infra
-      namespace: osh-infra
-      annotations:
-        metallb.universe.tf/loadBalancerIPs: "{{ metallb_osh_infra_endpoint_cidr | ipaddr('address') }}"
-    spec:
-      externalTrafficPolicy: Cluster
-      type: LoadBalancer
-      selector:
-        app: ingress-api
-      ports:
-        - name: http
-          port: 80
-        - name: https
-          port: 443
-    EOF
-    kubectl apply -f /tmp/osh_infra_endpoint_service.yaml
-
-- name: Set dnsmasq listen ip
-  set_fact:
-    nameserver_ip: "{{ (groups['primary'] | map('extract', hostvars, ['ansible_default_ipv4', 'address']))[0] }}"
-
-- name: Start dnsmasq
-  docker_container:
-    name: endpoint_dnsmasq
-    image: "{{ dnsmasq_image }}"
-    network_mode: host
-    capabilities:
-      - NET_ADMIN
-    entrypoint: dnsmasq
-    command: >-
-      --keep-in-foreground
-      --no-hosts
-      --bind-interfaces
-      --address="/openstack.svc.cluster.local/{{ metallb_openstack_endpoint_cidr | ipaddr('address') }}"
-      --address="/osh-infra.svc.cluster.local/{{ metallb_osh_infra_endpoint_cidr | ipaddr('address') }}"
-      --listen-address="{{ nameserver_ip }}"
-      --no-resolv
-      --server={{ dnsmasq_dns_server }}
-      {{ dnsmasq_extra_args | default('') }}
-    state: started
-    recreate: yes
-
-- name: Configure /etc/resolv.conf
-  template:
-    src: files/resolv.conf
-    dest: /etc/resolv.conf
-    owner: root
-    group: root
-    mode: 0644
-...

roles/deploy-env/tasks/main.yaml

@@ -84,11 +84,11 @@
     file: coredns_resolver.yaml
   when: coredns_resolver_setup
 
-- name: Include Openstack provider gateway tasks
+- name: Include floating network tasks
   include_tasks:
-    file: openstack_provider_gateway.yaml
+    file: floating_network.yaml
   when:
-    - openstack_provider_gateway_setup
+    - floating_network_setup
     - inventory_hostname in (groups['k8s_control_plane'] | default([]))
 
 - name: Include Metallb tasks
@@ -96,13 +96,6 @@
     file: metallb.yaml
   when: metallb_setup
 
-- name: Include Ingress Metallb endpoint tasks
-  include_tasks:
-    file: ingress_metallb_endpoint.yaml
-  when:
-    - metallb_setup
-    - inventory_hostname in (groups['primary'] | default([]))
-
 - name: Include client-to-cluster tunnel tasks
   include_tasks:
     file: client_cluster_tunnel.yaml
@@ -129,6 +122,21 @@
     - ingress_implementation == "haproxy"
     - inventory_hostname in (groups['primary'] | default([]))
 
+- name: Include Envoy Gateway tasks
+  include_tasks:
+    file: gatewayapi_envoy.yaml
+  when:
+    - gatewayapi_setup
+    - gatewayapi_implementation == "envoy"
+    - inventory_hostname in (groups['primary'] | default([]))
+
+- name: Include public endpoints tasks
+  include_tasks:
+    file: public_endpoints.yaml
+  when:
+    - ingress_setup or gatewayapi_setup
+    - metallb_setup
+
 - name: Include env inventory tasks
   include_tasks:
     file: env_inventory.yaml

roles/deploy-env/tasks/metallb.yaml

@@ -26,14 +26,11 @@
         helm upgrade --install metallb metallb/metallb \
           --version {{ metallb_version }} \
           --namespace metallb-system \
-          --create-namespace
+          --create-namespace \
+          --wait --timeout 5m
 
-    - name: Sleep before trying to check MetalLB pods
-      pause:
-        seconds: 30
-
-    - name: Wait for MetalLB pods ready
-      command: kubectl -n metallb-system wait --timeout=240s --for=condition=Ready pods -l 'app.kubernetes.io/name=metallb'
+    - name: Wait for MetalLB webhook to become ready
+      command: kubectl -n metallb-system wait --timeout=120s --for=condition=Ready pods -l 'app.kubernetes.io/component=controller'
 
     - name: Create MetalLB address pool
       shell: |
@@ -62,4 +59,8 @@
             - public
         EOF
         kubectl apply -f /tmp/metallb_l2advertisement.yaml
+      retries: 6
+      delay: 10
+      register: metallb_pool_result
+      until: metallb_pool_result.rc == 0
 ...

roles/deploy-env/tasks/public_endpoints.yaml

@@ -0,0 +1,194 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+---
+- name: Create openstack ingress service
+  when:
+    - ingress_setup
+    - ingress_openstack_setup
+    - inventory_hostname in (groups['primary'] | default([]))
+  become: false
+  shell: |
+    tee > /tmp/ingress_openstack_endpoint_service.yaml <<EOF
+    ---
+    apiVersion: v1
+    kind: Namespace
+    metadata:
+      labels:
+        kubernetes.io/metadata.name: openstack
+        name: openstack
+      name: openstack
+    ---
+    kind: Service
+    apiVersion: v1
+    metadata:
+      name: public-openstack
+      namespace: openstack
+      annotations:
+        metallb.universe.tf/loadBalancerIPs: "{{ metallb_ingress_openstack_endpoint_cidr | ipaddr('address') }}"
+    spec:
+      externalTrafficPolicy: Cluster
+      type: LoadBalancer
+      selector:
+        app: ingress-api
+      ports:
+        - name: http
+          port: 80
+        - name: https
+          port: 443
+    EOF
+    kubectl apply -f /tmp/ingress_openstack_endpoint_service.yaml
+
+- name: Create osh-infra ingress service
+  when:
+    - ingress_setup
+    - ingress_osh_infra_setup
+    - inventory_hostname in (groups['primary'] | default([]))
+  become: false
+  shell: |
+    tee > /tmp/ingress_osh_infra_endpoint_service.yaml <<EOF
+    ---
+    apiVersion: v1
+    kind: Namespace
+    metadata:
+      labels:
+        kubernetes.io/metadata.name: osh-infra
+        name: osh-infra
+      name: osh-infra
+    ---
+    kind: Service
+    apiVersion: v1
+    metadata:
+      name: public-osh-infra
+      namespace: osh-infra
+      annotations:
+        metallb.universe.tf/loadBalancerIPs: "{{ metallb_ingress_osh_infra_endpoint_cidr | ipaddr('address') }}"
+    spec:
+      externalTrafficPolicy: Cluster
+      type: LoadBalancer
+      selector:
+        app: ingress-api
+      ports:
+        - name: http
+          port: 80
+        - name: https
+          port: 443
+    EOF
+    kubectl apply -f /tmp/ingress_osh_infra_endpoint_service.yaml
+
+- name: Create Gateway for OpenStack services
+  when:
+    - gatewayapi_setup
+    - gatewayapi_implementation == "envoy"
+    - metallb_setup
+    - inventory_hostname in (groups['primary'] | default([]))
+  become: false
+  shell: |
+    tee > /tmp/gatewayapi_envoy_default.yaml <<EOF
+    ---
+    apiVersion: gateway.envoyproxy.io/v1alpha1
+    kind: EnvoyProxy
+    metadata:
+      name: gateway-proxy-default
+      namespace: envoy-gateway-system
+    spec:
+      provider:
+        type: Kubernetes
+        kubernetes:
+          envoyService:
+            type: LoadBalancer
+            externalTrafficPolicy: Cluster
+            annotations:
+              metallb.universe.tf/loadBalancerIPs: "{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+            patch:
+              type: StrategicMerge
+              value:
+                spec:
+                  externalTrafficPolicy: Cluster
+    ---
+    apiVersion: gateway.networking.k8s.io/v1
+    kind: Gateway
+    metadata:
+      name: gateway-default
+      namespace: envoy-gateway-system
+    spec:
+      gatewayClassName: default
+      infrastructure:
+        parametersRef:
+          group: gateway.envoyproxy.io
+          kind: EnvoyProxy
+          name: gateway-proxy-default
+      listeners:
+        - name: http
+          protocol: HTTP
+          port: 80
+          allowedRoutes:
+            namespaces:
+              from: All
+    EOF
+    kubectl apply -f /tmp/gatewayapi_envoy_default.yaml
+
+- name: Set dnsmasq listen ip
+  set_fact:
+    nameserver_ip: "{{ overlay_network_prefix }}{{ (groups['all'] | sort).index(groups['primary'][0]) + 1 }}"
+
+- name: Start dnsmasq
+  when:
+    - inventory_hostname in (groups['primary'] | default([]))
+  docker_container:
+    name: endpoint_dnsmasq
+    image: "{{ dnsmasq_image }}"
+    network_mode: host
+    capabilities:
+      - NET_ADMIN
+    entrypoint: dnsmasq
+    command: >-
+      --keep-in-foreground
+      --no-hosts
+      --bind-interfaces
+      --address="/keystone.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/placement.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/glance.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/nova.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/metadata.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/novncproxy.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/serialproxy.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/spiceproxy.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/neutron.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/heat.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/cloudformation.openstack-helm.org/{{ metallb_gatewayapi_endpoint_cidr | ipaddr('address') }}"
+      --address="/openstack.svc.cluster.local/{{ metallb_ingress_openstack_endpoint_cidr | ipaddr('address') }}"
+      --address="/osh-infra.svc.cluster.local/{{ metallb_ingress_osh_infra_endpoint_cidr | ipaddr('address') }}"
+      --listen-address="{{ nameserver_ip }}"
+      --no-resolv
+      --server={{ dnsmasq_dns_server }}
+      {{ dnsmasq_extra_args | default('') }}
+    state: started
+    recreate: yes
+
+- name: Configure /etc/resolv.conf
+  template:
+    src: files/resolv.conf
+    dest: /etc/resolv.conf
+    owner: root
+    group: root
+    mode: 0644
+  vars:
+    nameserver_ip: "{{ nameserver_ip }}"
+
+- name: Restart coredns to re-read resolv.conf changes
+  become: false
+  shell: |
+    kubectl rollout restart -n kube-system deployment/coredns
+    kubectl rollout status -n kube-system deployment/coredns --timeout=120s
+  when: inventory_hostname in (groups['primary'] | default([]))
+...

tools/deployment/common/openstack-exporter.sh

@@ -1,64 +0,0 @@
-#!/bin/bash
-
-#    Licensed under the Apache License, Version 2.0 (the "License"); you may
-#    not use this file except in compliance with the License. You may obtain
-#    a copy of the License at
-#
-#         http://www.apache.org/licenses/LICENSE-2.0
-#
-#    Unless required by applicable law or agreed to in writing, software
-#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-#    License for the specific language governing permissions and limitations
-#    under the License.
-
-
-set -xe
-
-# Check if Keystone API DNS and HTTP endpoint are available; skip deployment if not
-KEYSTONE_HOST="keystone-api.openstack.svc.cluster.local"
-KEYSTONE_PORT=5000
-KEYSTONE_URL="http://$KEYSTONE_HOST:$KEYSTONE_PORT/v3"
-TIMEOUT=${TIMEOUT:-60}
-INTERVAL=2
-start_time=$(date +%s)
-
-# DNS check
-while ! getent hosts "$KEYSTONE_HOST" >/dev/null; do
-    now=$(date +%s)
-    elapsed=$((now - start_time))
-    if [ $elapsed -ge $TIMEOUT ]; then
-        echo "[INFO] Keystone API DNS not found after $TIMEOUT seconds, skipping prometheus-openstack-exporter deployment."
-        exit 0
-    fi
-    echo "[INFO] Waiting for Keystone DNS... ($elapsed/$TIMEOUT)"
-    sleep $INTERVAL
-done
-
-# HTTP check
-while ! curl -sf "$KEYSTONE_URL" >/dev/null; do
-    now=$(date +%s)
-    elapsed=$((now - start_time))
-    if [ $elapsed -ge $TIMEOUT ]; then
-        echo "[INFO] Keystone API not responding after $TIMEOUT seconds, skipping prometheus-openstack-exporter deployment."
-        exit 0
-    fi
-    echo "[INFO] Waiting for Keystone API... ($elapsed/$TIMEOUT)"
-    sleep $INTERVAL
-done
-
-echo "[INFO] Keystone API is available. Proceeding with exporter deployment."
-
-#NOTE: Define variables
-: ${OSH_HELM_REPO:="../openstack-helm"}
-: ${OSH_VALUES_OVERRIDES_PATH:="../openstack-helm/values_overrides"}
-: ${OSH_EXTRA_HELM_ARGS_OSEXPORTER:="$(helm osh get-values-overrides ${DOWNLOAD_OVERRIDES:-} -p ${OSH_VALUES_OVERRIDES_PATH} -c prometheus-openstack-exporter ${FEATURES})"}
-
-#NOTE: Deploy command
-helm upgrade --install prometheus-openstack-exporter ${OSH_HELM_REPO}/prometheus-openstack-exporter \
-    --namespace=openstack \
-    ${OSH_EXTRA_HELM_ARGS:=} \
-    ${OSH_EXTRA_HELM_ARGS_OSEXPORTER}
-
-#NOTE: Wait for deploy
-helm osh wait-for-pods openstack

tools/deployment/common/setup-client.sh

@@ -29,7 +29,7 @@ if [[ ${FEATURES//,/ } =~ (^|[[:space:]])tls($|[[:space:]]) ]]; then
         project_name: 'admin'
         project_domain_name: 'default'
         user_domain_name: 'default'
-        auth_url: 'https://keystone.openstack.svc.cluster.local/v3'
+        auth_url: 'https://keystone.openstack-helm.org/v3'
 EOF
 else
   tee /etc/openstack/clouds.yaml << EOF
@@ -43,7 +43,7 @@ else
         project_name: 'admin'
         project_domain_name: 'default'
         user_domain_name: 'default'
-        auth_url: 'http://keystone.openstack.svc.cluster.local/v3'
+        auth_url: 'http://keystone.openstack-helm.org/v3'
 EOF
 fi
 

tools/deployment/common/use-it.sh

@@ -145,7 +145,7 @@ ssh -o "StrictHostKeyChecking no" -i ${SSH_DIR}/osh_key ${IMAGE_USER}@${FLOATING
 ssh -i ${SSH_DIR}/osh_key ${IMAGE_USER}@${FLOATING_IP} curl --verbose --connect-timeout 5 169.254.169.254
 
 # Check the VM can reach the keystone server
-ssh -i ${SSH_DIR}/osh_key ${IMAGE_USER}@${FLOATING_IP} curl --verbose --connect-timeout 5 keystone.openstack.svc.cluster.local
+ssh -i ${SSH_DIR}/osh_key ${IMAGE_USER}@${FLOATING_IP} curl --verbose --connect-timeout 5 keystone.openstack-helm.org
 
 # Check to see if cinder has been deployed, if it has then perform a volume attach.
 if openstack service list -f value -c Type | grep -q "^volume"; then

tools/deployment/component/compute-kit/compute-kit.sh

@@ -30,6 +30,7 @@ fi
 
 #NOTE: Deploy placement
 helm upgrade --install placement ${OSH_HELM_REPO}/placement --namespace=openstack \
+    --values=${OSH_VALUES_OVERRIDES_PATH}/placement/gateway.yaml \
     ${OSH_EXTRA_HELM_ARGS:=} \
     ${OSH_EXTRA_HELM_ARGS_PLACEMENT}
 
@@ -50,6 +51,7 @@ EOF
 helm upgrade --install nova ${OSH_HELM_REPO}/nova \
     --namespace=openstack \
     --values=/tmp/nova.yaml \
+    --values=${OSH_VALUES_OVERRIDES_PATH}/nova/gateway.yaml \
     ${OSH_EXTRA_HELM_ARGS:=} \
     ${OSH_EXTRA_HELM_ARGS_NOVA}
 
@@ -93,6 +95,7 @@ EOF
 helm upgrade --install neutron ${OSH_HELM_REPO}/neutron \
     --namespace=openstack \
     --values=/tmp/neutron.yaml \
+    --values=${OSH_VALUES_OVERRIDES_PATH}/neutron/gateway.yaml \
     ${OSH_EXTRA_HELM_ARGS:=} \
     ${OSH_EXTRA_HELM_ARGS_NEUTRON}
 

tools/deployment/component/freezer/freezer_smoke_test.sh

@@ -24,7 +24,7 @@ set -ex
 )
 
 unset OS_DOMAIN_NAME
-export OS_AUTH_URL=http://keystone.openstack.svc.cluster.local/v3
+export OS_AUTH_URL=http://keystone.openstack-helm.org/v3
 export OS_PROJECT_NAME=admin
 export OS_USERNAME=admin
 export OS_PASSWORD=password

tools/deployment/component/glance/glance.sh

@@ -29,6 +29,7 @@ EOF
 helm upgrade --install glance ${OSH_HELM_REPO}/glance \
   --namespace=openstack \
   --values=/tmp/glance.yaml \
+  --values=${OSH_VALUES_OVERRIDES_PATH}/glance/gateway.yaml \
   --timeout=800s \
   ${OSH_EXTRA_HELM_ARGS:=} \
   ${OSH_EXTRA_HELM_ARGS_GLANCE}

tools/deployment/component/heat/heat.sh

@@ -21,6 +21,7 @@ set -xe
 #NOTE: Deploy command
 helm upgrade --install heat ${OSH_HELM_REPO}/heat \
   --namespace=openstack \
+  --values=${OSH_VALUES_OVERRIDES_PATH}/heat/gateway.yaml \
   ${OSH_EXTRA_HELM_ARGS:=} \
   ${OSH_EXTRA_HELM_ARGS_HEAT}
 
@@ -36,5 +37,5 @@ sleep 30 #NOTE(portdirect): Wait for ingress controller to update rules and rest
 openstack orchestration service list
 
 if [[ ${FEATURES//,/ } =~ (^|[[:space:]])tls($|[[:space:]]) ]]; then
-  curl --cacert /etc/openstack-helm/certs/ca/ca.pem -L https://heat.openstack.svc.cluster.local
+  curl --cacert /etc/openstack-helm/certs/ca/ca.pem -L https://heat.openstack-helm.org
 fi

tools/deployment/component/keystone/keystone.sh

@@ -23,6 +23,7 @@ set -xe
 #NOTE: Deploy command
 helm upgrade --install keystone ${OSH_HELM_REPO}/keystone \
     --namespace=openstack \
+    --values=${OSH_VALUES_OVERRIDES_PATH}/keystone/gateway.yaml \
     ${OSH_EXTRA_HELM_ARGS:=} \
     ${OSH_EXTRA_HELM_ARGS_KEYSTONE}
 
@@ -50,13 +51,13 @@ if [[ ${FEATURES//,/ } =~ (^|[[:space:]])ldap($|[[:space:]]) ]]; then
 
   #NOTE: Testing we can auth against the LDAP user
   unset OS_CLOUD
-  openstack --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-username bob --os-password password --os-user-domain-name ${domain} --os-identity-api-version 3 token issue
+  openstack --os-auth-url http://keystone.openstack-helm.org/v3 --os-username bob --os-password password --os-user-domain-name ${domain} --os-identity-api-version 3 token issue
 
   #NOTE: Test the domain specific thing works
   curl --verbose -X GET \
     -H "Content-Type: application/json" \
     -H "X-Auth-Token: $token" \
-    http://keystone.openstack.svc.cluster.local/v3/domains/${domainId}/config
+    http://keystone.openstack-helm.org/v3/domains/${domainId}/config
 fi
 
 if [ "x${RUN_HELM_TESTS}" != "xno" ]; then
@@ -64,5 +65,5 @@ if [ "x${RUN_HELM_TESTS}" != "xno" ]; then
 fi
 
 if [[ ${FEATURES//,/ } =~ (^|[[:space:]])tls($|[[:space:]]) ]]; then
-  curl --cacert /etc/openstack-helm/certs/ca/ca.pem -L https://keystone.openstack.svc.cluster.local
+  curl --cacert /etc/openstack-helm/certs/ca/ca.pem -L https://keystone.openstack-helm.org
 fi

values_overrides/glance/gateway.yaml

@@ -0,0 +1,40 @@
+# Gateway API overrides for Glance.
+#
+# Public endpoints use *.openstack-helm.org FQDNs which are resolved by
+# dnsmasq to the MetalLB Gateway VIP. In-cluster pods reach the Gateway
+# through this DNS path so no ExternalName services are needed.
+---
+endpoints:
+  image:
+    host_fqdn_override:
+      public:
+        host: glance.openstack-helm.org
+
+manifests:
+  ingress_api: false
+  service_ingress_api: false
+
+extraObjects:
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: glance-route
+      namespace: openstack
+    spec:
+      hostnames:
+        - "glance"
+        - "glance.openstack"
+        - "glance.openstack.svc.cluster.local"
+        - "glance.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: glance-api
+              port: 9292
+...

values_overrides/heat/gateway.yaml

@@ -0,0 +1,68 @@
+# Gateway API overrides for Heat.
+#
+# Public endpoints use *.openstack-helm.org FQDNs which are resolved by
+# dnsmasq to the MetalLB Gateway VIP. In-cluster pods reach the Gateway
+# through this DNS path so no ExternalName services are needed.
+---
+endpoints:
+  orchestration:
+    host_fqdn_override:
+      public:
+        host: heat.openstack-helm.org
+  cloudformation:
+    host_fqdn_override:
+      public:
+        host: cloudformation.openstack-helm.org
+
+manifests:
+  ingress_api: false
+  ingress_cfn: false
+  service_ingress_api: false
+  service_ingress_cfn: false
+
+extraObjects:
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: heat-api-route
+      namespace: openstack
+    spec:
+      hostnames:
+        - "heat"
+        - "heat.openstack"
+        - "heat.openstack.svc.cluster.local"
+        - "heat.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: heat-api
+              port: 8004
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: heat-cfn-route
+      namespace: openstack
+    spec:
+      hostnames:
+        - "heat-cfn"
+        - "heat-cfn.openstack"
+        - "heat-cfn.openstack.svc.cluster.local"
+        - "cloudformation.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: heat-cfn
+              port: 8000
+...

values_overrides/keystone/gateway.yaml

@@ -0,0 +1,40 @@
+# Gateway API overrides for Keystone.
+#
+# Public endpoints use *.openstack-helm.org FQDNs which are resolved by
+# dnsmasq to the MetalLB Gateway VIP. In-cluster pods reach the Gateway
+# through this DNS path so no ExternalName services are needed.
+---
+endpoints:
+  identity:
+    host_fqdn_override:
+      public:
+        host: keystone.openstack-helm.org
+
+manifests:
+  ingress_api: false
+  service_ingress_api: false
+
+extraObjects:
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: keystone-route
+      namespace: openstack
+    spec:
+      parentRefs:
+      - name: gateway-default
+        namespace: envoy-gateway-system
+      hostnames:
+      - "keystone"
+      - "keystone.openstack"
+      - "keystone.openstack.svc.cluster.local"
+      - "keystone.openstack-helm.org"
+      rules:
+      - matches:
+        - path:
+            type: PathPrefix
+            value: /
+        backendRefs:
+        - name: keystone-api
+          port: 5000
+...

values_overrides/neutron/gateway.yaml

@@ -0,0 +1,40 @@
+# Gateway API overrides for Neutron.
+#
+# Public endpoints use *.openstack-helm.org FQDNs which are resolved by
+# dnsmasq to the MetalLB Gateway VIP. In-cluster pods reach the Gateway
+# through this DNS path so no ExternalName services are needed.
+---
+endpoints:
+  network:
+    host_fqdn_override:
+      public:
+        host: neutron.openstack-helm.org
+
+manifests:
+  ingress_server: false
+  service_ingress_server: false
+
+extraObjects:
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: neutron-route
+      namespace: openstack
+    spec:
+      hostnames:
+        - "neutron"
+        - "neutron.openstack"
+        - "neutron.openstack.svc.cluster.local"
+        - "neutron.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: neutron-server
+              port: 9696
+...

values_overrides/nova/gateway.yaml

@@ -0,0 +1,156 @@
+# Gateway API overrides for Nova.
+#
+# Public endpoints use *.openstack-helm.org FQDNs which are resolved by
+# dnsmasq to the MetalLB Gateway VIP. In-cluster pods reach the Gateway
+# through this DNS path so no ExternalName services are needed.
+---
+endpoints:
+  compute:
+    host_fqdn_override:
+      public:
+        host: nova.openstack-helm.org
+  compute_metadata:
+    host_fqdn_override:
+      public:
+        host: metadata.openstack-helm.org
+  compute_novnc_proxy:
+    host_fqdn_override:
+      public:
+        host: novncproxy.openstack-helm.org
+  compute_serial_proxy:
+    host_fqdn_override:
+      public:
+        host: serialproxy.openstack-helm.org
+  compute_spice_proxy:
+    host_fqdn_override:
+      public:
+        host: spiceproxy.openstack-helm.org
+
+manifests:
+  ingress_metadata: false
+  ingress_novncproxy: false
+  ingress_serialproxy: false
+  ingress_spiceproxy: false
+  ingress_osapi: false
+  service_ingress_metadata: false
+  service_ingress_novncproxy: false
+  service_ingress_serialproxy: false
+  service_ingress_spiceproxy: false
+  service_ingress_osapi: false
+
+extraObjects:
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: nova-route-osapi
+      namespace: openstack
+    spec:
+      hostnames:
+        - "nova"
+        - "nova.openstack"
+        - "nova.openstack.svc.cluster.local"
+        - "nova.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: nova-api
+              port: 8774
+
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: nova-route-metadata
+      namespace: openstack
+    spec:
+      hostnames:
+        - "metadata"
+        - "metadata.openstack"
+        - "metadata.openstack.svc.cluster.local"
+        - "metadata.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: nova-metadata
+              port: 8775
+
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: nova-route-novncproxy
+      namespace: openstack
+    spec:
+      hostnames:
+        - "novncproxy"
+        - "novncproxy.openstack"
+        - "novncproxy.openstack.svc.cluster.local"
+        - "novncproxy.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: nova-novncproxy
+              port: 6080
+
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: nova-route-serialproxy
+      namespace: openstack
+    spec:
+      hostnames:
+        - "serialproxy"
+        - "serialproxy.openstack"
+        - "serialproxy.openstack.svc.cluster.local"
+        - "serialproxy.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: nova-serialproxy
+              port: 6083
+
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: nova-route-spiceproxy
+      namespace: openstack
+    spec:
+      hostnames:
+        - "spiceproxy"
+        - "spiceproxy.openstack"
+        - "spiceproxy.openstack.svc.cluster.local"
+        - "spiceproxy.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: nova-spiceproxy
+              port: 6082
+...

values_overrides/placement/gateway.yaml

@@ -0,0 +1,40 @@
+# Gateway API overrides for Placement.
+#
+# Public endpoints use *.openstack-helm.org FQDNs which are resolved by
+# dnsmasq to the MetalLB Gateway VIP. In-cluster pods reach the Gateway
+# through this DNS path so no ExternalName services are needed.
+---
+endpoints:
+  placement:
+    host_fqdn_override:
+      public:
+        host: placement.openstack-helm.org
+
+manifests:
+  ingress: false
+  service_ingress: false
+
+extraObjects:
+  - apiVersion: gateway.networking.k8s.io/v1
+    kind: HTTPRoute
+    metadata:
+      name: placement-route
+      namespace: openstack
+    spec:
+      hostnames:
+        - "placement"
+        - "placement.openstack"
+        - "placement.openstack.svc.cluster.local"
+        - "placement.openstack-helm.org"
+      parentRefs:
+        - name: gateway-default
+          namespace: envoy-gateway-system
+      rules:
+        - matches:
+            - path:
+                type: PathPrefix
+                value: /
+          backendRefs:
+            - name: placement-api
+              port: 8778
+...

zuul.d/base.yaml

@@ -114,9 +114,12 @@
       coredns_resolver_setup: false
       ingress_setup: true
       ingress_implementation: "haproxy"
+      gatewayapi_setup: true
+      gatewayapi_implementation: "envoy"
+      gatewayapi_envoy_version: "v1.7.0"
       crictl_version: "v1.35.0"
       run_helm_tests: "no"
-      openstack_provider_gateway_setup: true
+      floating_network_setup: true
 
 - job:
     name: openstack-helm-deploy-kubespray