ext-gw-multihoming: api-def and api-ref

API additions for [1].

* Added a new router attribute: external_gateways;
* Added new API definitions for:
    PUT add_external_gateways
    PUT update_external_gateways
    PUT remove_external_gateways
* Added extensions for each of the new router-level attributes:
  * enable_default_route_ecmp
  * enable_default_route_bfd
* Combined the validation logic for the external_gateway_info type
  across extensions (l3_ext_gw_mode, qos_gateway_ip and the new
  extension called l3_ext_gw_multihoming).

[1] https://review.opendev.org/c/openstack/neutron-specs/+/870030/

Change-Id: I2618475636b2bb9bfd743a62f5d4859d4f68a547
Related-Bug: #2002687

api-ref/source/v2/parameters.yaml

@@ -5971,6 +5971,22 @@ router-distributed-request:
   in: body
   required: false
   type: boolean
+router-enable_default_route_bfd:
+  description: |
+    ``true`` indicates that Neutron will enable BFD sessions for default routes
+    inferred from the external gateway port subnets.
+    Available when ``external-gateway-multihoming`` extension is enabled.
+  in: body
+  required: false
+  type: boolean
+router-enable_default_route_ecmp:
+  description: |
+    ``true`` indicates that Neutron will add ECMP default routes if multiple
+    are available via different gateway ports.
+    Available when ``external-gateway-multihoming`` extension is enabled.
+  in: body
+  required: false
+  type: boolean
 router-enable_ndp_proxy:
   description: |
     Enable NDP proxy attribute. ``true`` means NDP proxy is enabled for the
@@ -6032,8 +6048,9 @@ router-external_gateway_info:
   description: |
     The external gateway information of the router.
     If the router has an external gateway, this would be a dict with
-    ``network_id``, ``enable_snat``, ``external_fixed_ips`` and
+    ``network_id``, ``enable_snat``, ``external_fixed_ips``,
-    ``qos_policy_id``.
+    ``qos_policy_id``, ``enable_default_route_ecmp`` and
+    ``enable_default_route_bfd``.
     Otherwise, this would be ``null``.
   in: body
   required: true
@@ -6054,6 +6071,12 @@ router-external_gateway_ports:
   in: body
   required: true
   type: string
+router-external_gateways:
+  description: |
+    The list of external gateways of the router.
+  in: body
+  required: true
+  type: array
 router-flavor_id:
   description: |
     The ID of the flavor associated with the router.

api-ref/source/v2/routers.inc

@@ -68,6 +68,14 @@ modes, adds the ``external_gateway_info`` attribute to ``routers``
 and allows definitions for ``network_id``, ``enable_snat`` and
 ``external_fixed_ips``.
 
+L3 external gateway multihoming extension (``external-gateway-multihoming``)
+============================================================================
+
+The ``external-gateway-multihoming`` extension allows a router to have
+multiple external gateway ports and to have a policy specified on how
+to handle ECMP and BFD for default routes inferred from the subnets
+associated with gateway ports.
+
 L3 flavors extension (``l3-flavors``)
 =====================================
 
@@ -135,6 +143,24 @@ Router gateway IP QoS (qos-gateway-ip)
 The ``qos-gateway-ip`` extension adds ``qos_policy_id`` to the
 ``external_gateway_info`` field of routers.
 
+Router enable default route ECMP extension (``enable-default-route-ecmp``)
+==========================================================================
+
+The ``enable-default-route-ecmp`` extension adds a parameter called
+``enable_default_route_ecmp`` to the router resource which can be used to
+enable or disable automatic configuration of ECMP default routes based on the
+default gateways of subnets accessible from a router's gateway ports (see
+the ``external-gateway-multihoming`` extension).
+
+Router enable default route BFD extension (``enable-default-route-bfd``)
+========================================================================
+
+The ``enable-default-route-bfd`` extension adds a parameter called
+``enable_default_route_bfd`` to the router resource which can be used to
+enable or disable automatic configuration of BFD for default routes of a router
+created based on the default gateways of subnets accessible from a router's
+gateway ports (see ``enable-default-route-ecmp`` extension).
+
 List routers
 ============
 
@@ -190,6 +216,7 @@ Response Parameters
    - admin_state_up: admin_state_up
    - status: router-status
    - external_gateway_info: router-external_gateway_info
+   - external_gateways: router-external_gateways
    - revision_number: revision_number
    - routes: router-routes
    - destination: router-destination
@@ -205,6 +232,8 @@ Response Parameters
    - tags: tags
    - conntrack_helpers: router-conntrack_helpers
    - enable_ndp_proxy: router-enable_ndp_proxy
+   - enable_default_route_bfd: router-enable_default_route_bfd
+   - enable_default_route_ecmp: router-enable_default_route_ecmp
 
 Response Example
 ----------------
@@ -251,6 +280,8 @@ Request
    - service_type_id: router-service_type_id-request
    - flavor_id: router-flavor_id-optional
    - enable_ndp_proxy: router-enable_ndp_proxy-request
+   - enable_default_route_bfd: router-enable_default_route_bfd
+   - enable_default_route_ecmp: router-enable_default_route_ecmp
 
 Request Example
 ---------------
@@ -272,6 +303,7 @@ Response Parameters
    - admin_state_up: admin_state_up
    - status: router-status
    - external_gateway_info: router-external_gateway_info
+   - external_gateways: router-external_gateways
    - revision_number: revision_number
    - routes: router-routes
    - destination: router-destination
@@ -287,6 +319,8 @@ Response Parameters
    - tags: tags
    - conntrack_helpers: router-conntrack_helpers
    - enable_ndp_proxy: router-enable_ndp_proxy
+   - enable_default_route_bfd: router-enable_default_route_bfd
+   - enable_default_route_ecmp: router-enable_default_route_ecmp
 
 Response Example
 ----------------
@@ -333,6 +367,7 @@ Response Parameters
    - admin_state_up: admin_state_up
    - status: router-status
    - external_gateway_info: router-external_gateway_info
+   - external_gateways: router-external_gateways
    - revision_number: revision_number
    - routes: router-routes
    - destination: router-destination
@@ -348,6 +383,8 @@ Response Parameters
    - tags: tags
    - conntrack_helpers: router-conntrack_helpers
    - enable_ndp_proxy: router-enable_ndp_proxy
+   - enable_default_route_bfd: router-enable_default_route_bfd
+   - enable_default_route_ecmp: router-enable_default_route_ecmp
 
 Response Example
 ----------------
@@ -385,6 +422,8 @@ Request
    - routes: router-routes-request
    - distributed: router-distributed-request
    - enable_ndp_proxy: router-enable_ndp_proxy-request
+   - enable_default_route_bfd: router-enable_default_route_bfd
+   - enable_default_route_ecmp: router-enable_default_route_ecmp
 
 Request Example
 ---------------
@@ -406,6 +445,7 @@ Response Parameters
    - admin_state_up: admin_state_up
    - status: router-status
    - external_gateway_info: router-external_gateway_info
+   - external_gateways: router-external_gateways
    - revision_number: revision_number
    - routes: router-routes
    - destination: router-destination
@@ -421,6 +461,8 @@ Response Parameters
    - tags: tags
    - conntrack_helpers: router-conntrack_helpers
    - enable_ndp_proxy: router-enable_ndp_proxy
+   - enable_default_route_bfd: router-enable_default_route_bfd
+   - enable_default_route_ecmp: router-enable_default_route_ecmp
 
 Response Example
 ----------------
@@ -766,3 +808,239 @@ Response Example
 
 .. literalinclude:: samples/routers/router-remove-extraroutes-response.json
    :language: javascript
+
+Add external gateways to router
+===============================
+
+.. rest_method::  PUT /v2.0/routers/{router_id}/add_external_gateways
+
+Add external gateways to a router in addition to the ones it already
+has.
+
+Multiple gateways attached to the same network can be added to the
+same router.
+
+The add/update/remove external gateways operations extend the use of
+``router.external_gateway_info`` to manage multiple external gateways.
+The full set of external gateways is exposed in the read-only
+``router.external_gateways`` parameter.  ``router.external_gateways``
+contains a list of ``external_gateway_info`` structures like:
+
+::
+
+    [
+      {"network_id": ...,
+       "external_fixed_ips": [{"ip_address": ..., "subnet_id": ...}, ...],
+       "enable_snat": ...},
+      ...
+    ]
+
+The first item (index 0) of the ``external_gateways`` list is special if a
+router does not have any gateway ports yet:
+
+* It will provide data for the compatibility ``router.external_gateway_info``
+  field of a router;
+
+* This first item sets a router's default route. If ECMP is enabled for
+  default routes inferred from gateway port subnets, then all of those
+  default routes are used for load-sharing;
+
+* The first item is just another extra gateway if the add operation is
+  performed when a router already has one or more gateways.
+
+The order of the the rest of the list (indexes 1, 2, ...) is irrelevant
+and ignored.
+
+The first external gateway can be managed in two
+ways: via ``router.external_gateway_info`` or via
+``add/update/remove_external_gateways``.  The other external gateways
+can only be managed via ``add/update/remove_external_gateways``.
+
+The format of the request body is the same as the format of the read-only
+``router.external_gateways`` parameter, but wrapped as follows:
+
+::
+
+    {"router": {"external_gateways": EXTERNAL-GATEWAY-LIST}}
+
+The response codes and response body are the same as to the update of
+the router.  That is the whole router object is returned including the
+``external_gateway_info`` and ``external_gateways`` parameters which
+represents the result of the operation.
+
+Changes in ``router.external_gateway_info`` are reflected
+in ``router.external_gateways`` and vice versa.  Updating
+``external_gateway_info`` also updates the first element of
+``external_gateways`` and it leaves the rest of ``external_gateways``
+unchanged.  Setting ``external_gateway_info`` to an empty value removes
+a single gateway and one of the extra gateways takes its place instead.
+
+Normal response codes: 200
+
+Error response codes: 400, 401, 404, 412
+
+Request Parameters
+------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - router_id: router_id
+   - external_gateways: router-external_gateways
+
+Request Example
+---------------
+
+.. literalinclude:: samples/routers/router-add-external-gateways-request.json
+   :language: javascript
+
+Response Parameters
+-------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - id: router-id-body
+   - name: router_name
+   - external_gateways: router-external_gateways
+
+Response Example
+----------------
+
+.. literalinclude:: samples/routers/router-add-external-gateways-response.json
+   :language: javascript
+
+Update external gateways of router
+==================================
+
+.. rest_method::  PUT /v2.0/routers/{router_id}/update_external_gateways
+
+Update some external gateways of router.
+
+For general information on the add/update/remove external gateways
+operations see ``add_external_gateways`` above.
+
+The external gateways to be updated are identified by the ``network_ids``
+found in the PUT request.  The ``external_fixed_ips``, ``enable_snat``,
+fields can be updated.  The ``network_id`` field cannot be updated - any
+changes will cause a gateway port to be removed and recreated.
+
+The format of the request body is the same as the format of the read-only
+``router.external_gateways`` parameter, but wrapped as follows:
+
+::
+
+    {"router": {"external_gateways": EXTERNAL-GATEWAY-LIST}}
+
+The ``enable_snat`` field does not have any effect for extra gateways except
+for the first external gateway in the list.
+
+The ``network_id`` field is used to identify a particular gateway port along
+with the ``external_fixed_ips`` field. Specifying just the ``network_id`` field
+is ambiguous: Neutron will attempt to find the matching gateway port but if
+there are multiple matches it will return an error response code.
+
+The ``enable_snat`` field can be omitted from the request. Specifying
+``external_fixed_ips`` will result in matching ports based on those
+fixed IPs. If a gateway port has a subset of the specified fixed IPs,
+then the set of IPs will be updated to match the ones in the request.
+Alternatively, if a gateway port has a superset of fixed IPs from the
+request the IPs will be removed from the gateway port.
+
+The response codes and response body are the same as to the update of
+the router.  That is the whole router object is returned including the
+``external_gateway_info`` and ``external_gateways`` parameters which
+represents the result of the operation.
+
+Please note that updating ``external_gateway_info`` also updates
+the first element of ``external_gateways`` and it leaves the rest of
+``external_gateways`` unchanged.
+
+Normal response codes: 200
+
+Error response codes: 400, 401, 404, 412
+
+Request Parameters
+------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - router_id: router_id
+   - external_gateways: router-external_gateways
+
+Request Example
+---------------
+
+.. literalinclude:: samples/routers/router-update-external-gateways-request.json
+   :language: javascript
+
+Response Parameters
+-------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - id: router-id-body
+   - name: router_name
+   - external_gateways: router-external_gateways
+
+Response Example
+----------------
+
+.. literalinclude:: samples/routers/router-update-external-gateways-response.json
+   :language: javascript
+
+Remove external gateways from router
+====================================
+
+.. rest_method::  PUT /v2.0/routers/{router_id}/remove_external_gateways
+
+Remove some external gateways from router.
+
+For general information on the add/update/remove external gateways
+operations see ``add_external_gateways`` above.
+
+The format of the request body is the same as the format of the read-only
+``router.external_gateways`` parameter, but wrapped as follows:
+
+::
+
+    {"router": {"external_gateways": EXTERNAL-GATEWAY-LIST}}
+
+However the request body can be partial.  Only the ``network_id``
+and ``external_fixed_ips`` fields from the ``external_gateway_info``
+structure is used in order to match the specific gateway ports.
+The ``enable_snat`` key can be present but its value is ignored.
+
+Please note that setting ``external_gateway_info`` to an empty value
+also resets ``external_gateways`` to the empty list.
+
+Normal response codes: 200
+
+Error response codes: 400, 401, 404, 412
+
+Request Parameters
+------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - router_id: router_id
+   - external_gateways: router-external_gateways
+
+Request Example
+---------------
+
+.. literalinclude:: samples/routers/router-remove-external-gateways-request.json
+   :language: javascript
+
+Response Parameters
+-------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - id: router-id-body
+   - name: router_name
+   - external_gateways: router-external_gateways
+
+Response Example
+----------------
+
+.. literalinclude:: samples/routers/router-remove-external-gateways-response.json
+   :language: javascript

api-ref/source/v2/samples/routers/router-add-external-gateways-request.json

@@ -0,0 +1,16 @@
+{
+   "router" : {
+      "external_gateways" : [
+         {
+            "enable_snat" : false,
+            "external_fixed_ips" : [
+               {
+                  "ip_address" : "192.0.2.2",
+                  "subnet_id" : "b189c314-ebb9-11eb-939c-9bde3f3867cb"
+               }
+            ],
+            "network_id" : "8edec774-ebb9-11eb-9b09-371108ef5905"
+         }
+      ]
+   }
+}

api-ref/source/v2/samples/routers/router-add-external-gateways-response.json

@@ -0,0 +1,73 @@
+{
+   "router" : {
+      "admin_state_up" : true,
+      "availability_zone_hints" : [],
+      "availability_zones" : [
+         "nova"
+      ],
+      "created_at" : "2021-06-29T13:33:40Z",
+      "description" : "",
+      "distributed" : false,
+      "enable_default_route_ecmp" : false,
+      "enable_default_route_bfd" : false,
+      "external_gateway_info" : {
+         "enable_snat" : false,
+         "external_fixed_ips" : [
+            {
+               "ip_address" : "172.24.4.144",
+               "subnet_id" : "1ed1c499-a45d-48d0-a567-e83a2364a40e"
+            }
+         ],
+         "network_id" : "52700ca1-1647-46ad-8f86-b9e64eaed820"
+      },
+      "external_gateways" : [
+         {
+            "enable_snat" : false,
+            "external_fixed_ips" : [
+               {
+                  "ip_address" : "172.24.4.144",
+                  "subnet_id" : "1ed1c499-a45d-48d0-a567-e83a2364a40e"
+               }
+            ],
+            "network_id" : "52700ca1-1647-46ad-8f86-b9e64eaed820"
+         },
+         {
+            "enable_snat" : false,
+            "external_fixed_ips" : [
+               {
+                  "ip_address" : "192.0.2.2",
+                  "subnet_id" : "b189c314-ebb9-11eb-939c-9bde3f3867cb"
+               }
+            ],
+            "network_id" : "8edec774-ebb9-11eb-9b09-371108ef5905"
+         }
+      ],
+      "flavor_id" : null,
+      "ha" : false,
+      "id" : "47c32c39-1c09-47de-8d50-ec57a96db5e7",
+      "interfaces_info" : [
+         {
+            "ip_address" : "fd26:d08e:af31::1",
+            "port_id" : "20683e3d-b041-4977-9686-b97db622c76a",
+            "subnet_id" : "2921b809-b60a-4799-ac99-59dacbeb7c3a"
+         },
+         {
+            "ip_address" : "10.0.0.1",
+            "port_id" : "89ab7084-7883-48e6-8281-d498a0cf4c92",
+            "subnet_id" : "3a5bec20-2df1-4d11-b0d5-5481969b91ac"
+         },
+         {
+            "ip_address" : "10.0.5.1",
+            "port_id" : "b04c6f4e-5bc7-43a2-85e9-c28452368532",
+            "subnet_id" : "f96970c4-026a-46f3-9852-f512a56688fe"
+         }
+      ],
+      "name" : "router1",
+      "project_id" : "b66a1cea961f49738fff1210733ec440",
+      "revision_number" : 7,
+      "routes" : [],
+      "status" : "ACTIVE",
+      "tags" : [],
+      "updated_at" : "2021-06-29T13:37:07Z"
+   }
+}

api-ref/source/v2/samples/routers/router-remove-external-gateways-request.json

@@ -0,0 +1,9 @@
+{
+   "router" : {
+      "external_gateways" : [
+         {
+            "network_id" : "8edec774-ebb9-11eb-9b09-371108ef5905"
+         }
+      ]
+   }
+}

api-ref/source/v2/samples/routers/router-remove-external-gateways-response.json

@@ -0,0 +1,63 @@
+{
+   "router" : {
+      "admin_state_up" : true,
+      "availability_zone_hints" : [],
+      "availability_zones" : [
+         "nova"
+      ],
+      "created_at" : "2021-06-29T13:33:40Z",
+      "description" : "",
+      "distributed" : false,
+      "enable_default_route_ecmp" : false,
+      "enable_default_route_bfd" : false,
+      "external_gateway_info" : {
+         "enable_snat" : false,
+         "external_fixed_ips" : [
+            {
+               "ip_address" : "172.24.4.144",
+               "subnet_id" : "1ed1c499-a45d-48d0-a567-e83a2364a40e"
+            }
+         ],
+         "network_id" : "52700ca1-1647-46ad-8f86-b9e64eaed820"
+      },
+      "external_gateways" : [
+         {
+            "enable_snat" : false,
+            "external_fixed_ips" : [
+               {
+                  "ip_address" : "172.24.4.144",
+                  "subnet_id" : "1ed1c499-a45d-48d0-a567-e83a2364a40e"
+               }
+            ],
+            "network_id" : "52700ca1-1647-46ad-8f86-b9e64eaed820"
+         }
+      ],
+      "flavor_id" : null,
+      "ha" : false,
+      "id" : "47c32c39-1c09-47de-8d50-ec57a96db5e7",
+      "interfaces_info" : [
+         {
+            "ip_address" : "fd26:d08e:af31::1",
+            "port_id" : "20683e3d-b041-4977-9686-b97db622c76a",
+            "subnet_id" : "2921b809-b60a-4799-ac99-59dacbeb7c3a"
+         },
+         {
+            "ip_address" : "10.0.0.1",
+            "port_id" : "89ab7084-7883-48e6-8281-d498a0cf4c92",
+            "subnet_id" : "3a5bec20-2df1-4d11-b0d5-5481969b91ac"
+         },
+         {
+            "ip_address" : "10.0.5.1",
+            "port_id" : "b04c6f4e-5bc7-43a2-85e9-c28452368532",
+            "subnet_id" : "f96970c4-026a-46f3-9852-f512a56688fe"
+         }
+      ],
+      "name" : "router1",
+      "project_id" : "b66a1cea961f49738fff1210733ec440",
+      "revision_number" : 7,
+      "routes" : [],
+      "status" : "ACTIVE",
+      "tags" : [],
+      "updated_at" : "2021-06-29T13:37:07Z"
+   }
+}

api-ref/source/v2/samples/routers/router-update-external-gateways-request.json

@@ -0,0 +1,10 @@
+{
+   "router" : {
+      "external_gateways" : [
+         {
+            "enable_snat" : true,
+            "network_id" : "8edec774-ebb9-11eb-9b09-371108ef5905"
+         }
+      ]
+   }
+}

api-ref/source/v2/samples/routers/router-update-external-gateways-response.json

@@ -0,0 +1,75 @@
+{
+   "router" : {
+      "admin_state_up" : true,
+      "availability_zone_hints" : [],
+      "availability_zones" : [
+         "nova"
+      ],
+      "created_at" : "2021-06-29T13:33:40Z",
+      "description" : "",
+      "distributed" : false,
+      "enable_default_route_ecmp" : false,
+      "enable_default_route_bfd" : false,
+      "external_gateway_info" : {
+         "enable_snat" : false,
+         "enable_default_route_ecmp": false,
+         "enable_default_route_bfd": false,
+         "external_fixed_ips" : [
+            {
+               "ip_address" : "172.24.4.144",
+               "subnet_id" : "1ed1c499-a45d-48d0-a567-e83a2364a40e"
+            }
+         ],
+         "network_id" : "52700ca1-1647-46ad-8f86-b9e64eaed820"
+      },
+      "external_gateways" : [
+         {
+            "enable_snat" : false,
+            "external_fixed_ips" : [
+               {
+                  "ip_address" : "172.24.4.144",
+                  "subnet_id" : "1ed1c499-a45d-48d0-a567-e83a2364a40e"
+               }
+            ],
+            "network_id" : "52700ca1-1647-46ad-8f86-b9e64eaed820"
+         },
+         {
+            "enable_snat" : true,
+            "external_fixed_ips" : [
+               {
+                  "ip_address" : "192.0.2.2",
+                  "subnet_id" : "b189c314-ebb9-11eb-939c-9bde3f3867cb"
+               }
+            ],
+            "network_id" : "8edec774-ebb9-11eb-9b09-371108ef5905"
+         }
+      ],
+      "flavor_id" : null,
+      "ha" : false,
+      "id" : "47c32c39-1c09-47de-8d50-ec57a96db5e7",
+      "interfaces_info" : [
+         {
+            "ip_address" : "fd26:d08e:af31::1",
+            "port_id" : "20683e3d-b041-4977-9686-b97db622c76a",
+            "subnet_id" : "2921b809-b60a-4799-ac99-59dacbeb7c3a"
+         },
+         {
+            "ip_address" : "10.0.0.1",
+            "port_id" : "89ab7084-7883-48e6-8281-d498a0cf4c92",
+            "subnet_id" : "3a5bec20-2df1-4d11-b0d5-5481969b91ac"
+         },
+         {
+            "ip_address" : "10.0.5.1",
+            "port_id" : "b04c6f4e-5bc7-43a2-85e9-c28452368532",
+            "subnet_id" : "f96970c4-026a-46f3-9852-f512a56688fe"
+         }
+      ],
+      "name" : "router1",
+      "project_id" : "b66a1cea961f49738fff1210733ec440",
+      "revision_number" : 7,
+      "routes" : [],
+      "status" : "ACTIVE",
+      "tags" : [],
+      "updated_at" : "2021-06-29T13:37:07Z"
+   }
+}

neutron_lib/api/definitions/__init__.py

@@ -62,7 +62,10 @@ from neutron_lib.api.definitions import ip_substring_port_filtering
 from neutron_lib.api.definitions import l2_adjacency
 from neutron_lib.api.definitions import l3
 from neutron_lib.api.definitions import l3_conntrack_helper
+from neutron_lib.api.definitions import l3_enable_default_route_bfd
+from neutron_lib.api.definitions import l3_enable_default_route_ecmp
 from neutron_lib.api.definitions import l3_ext_gw_mode
+from neutron_lib.api.definitions import l3_ext_gw_multihoming
 from neutron_lib.api.definitions import l3_ext_ha_mode
 from neutron_lib.api.definitions import l3_ext_ndp_proxy
 from neutron_lib.api.definitions import l3_flavors
@@ -211,6 +214,9 @@ _ALL_API_DEFINITIONS = {
     l3_ext_ha_mode,
     l3_ext_ndp_proxy,
     l3_flavors,
+    l3_ext_gw_multihoming,
+    l3_enable_default_route_bfd,
+    l3_enable_default_route_ecmp,
     l3_ndp_proxy,
     l3_port_ip_change_not_allowed,
     local_ip,

neutron_lib/api/definitions/base.py

@@ -25,6 +25,7 @@ KNOWN_ATTRIBUTES = (
     'dns_nameservers',
     'enable_dhcp',
     'enable_ndp_proxy',
+    'external_gateways',
     'fixed_ips',
     'gateway_ip',
     'host_routes',
@@ -92,6 +93,8 @@ KNOWN_EXTENSIONS = (
     'dns-domain-ports',
     'dns-integration',
     'dvr',
+    'enable-default-route-ecmp',
+    'enable-default-route-bfd',
     'empty-string-filtering',
     'expose-l3-conntrack-helper',
     'expose-port-forwarding-in-fip',
@@ -118,6 +121,7 @@ KNOWN_EXTENSIONS = (
     'metering',
     'metering_source_and_destination_filters',
     'multi-provider',
+    'external-gateway-multihoming',
     'net-mtu',
     'network-ip-availability',
     'network-segment-range',

neutron_lib/api/definitions/l3_enable_default_route_bfd.py

@@ -0,0 +1,48 @@
+# Copyright 2023 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from neutron_lib.api import converters
+from neutron_lib.api.definitions import l3
+from neutron_lib.api.definitions import l3_ext_gw_multihoming
+from neutron_lib import constants
+
+
+ENABLE_DEFAULT_ROUTE_BFD = 'enable_default_route_bfd'
+
+ALIAS = constants.L3_ENABLE_DEFAULT_ROUTE_BFD_ALIAS
+IS_SHIM_EXTENSION = False
+IS_STANDARD_ATTR_EXTENSION = False
+NAME = 'Enable Default Route BFD'
+API_PREFIX = ''
+DESCRIPTION = 'Enables configurable BFD behavior for default routes'
+UPDATED_TIMESTAMP = '2023-02-27T00:00:00-00:00'
+RESOURCE_NAME = l3.ROUTER
+COLLECTION_NAME = l3.ROUTERS
+RESOURCE_ATTRIBUTE_MAP = {
+    COLLECTION_NAME: {
+        ENABLE_DEFAULT_ROUTE_BFD: {
+            'allow_post': True,
+            'allow_put': True,
+            'default': False,
+            'is_visible': True,
+            'enforce_policy': True,
+            'convert_to': converters.convert_to_boolean_if_not_none,
+        }
+    }
+}
+SUB_RESOURCE_ATTRIBUTE_MAP = {}
+ACTION_MAP = {}
+REQUIRED_EXTENSIONS = [l3.ALIAS, l3_ext_gw_multihoming.ALIAS]
+OPTIONAL_EXTENSIONS = []
+ACTION_STATUS = {}

neutron_lib/api/definitions/l3_enable_default_route_ecmp.py

@@ -0,0 +1,47 @@
+# Copyright 2023 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from neutron_lib.api import converters
+from neutron_lib.api.definitions import l3
+from neutron_lib.api.definitions import l3_ext_gw_multihoming
+from neutron_lib import constants
+
+ENABLE_DEFAULT_ROUTE_ECMP = 'enable_default_route_ecmp'
+
+ALIAS = constants.L3_ENABLE_DEFAULT_ROUTE_ECMP_ALIAS
+IS_SHIM_EXTENSION = False
+IS_STANDARD_ATTR_EXTENSION = False
+NAME = 'Enable Default Route ECMP'
+API_PREFIX = ''
+DESCRIPTION = 'Enables configurable ECMP behavior for default routes'
+UPDATED_TIMESTAMP = '2023-02-27T00:00:00-00:00'
+RESOURCE_NAME = l3.ROUTER
+COLLECTION_NAME = l3.ROUTERS
+RESOURCE_ATTRIBUTE_MAP = {
+    COLLECTION_NAME: {
+        ENABLE_DEFAULT_ROUTE_ECMP: {
+            'allow_post': True,
+            'allow_put': True,
+            'default': False,
+            'is_visible': True,
+            'enforce_policy': True,
+            'convert_to': converters.convert_to_boolean_if_not_none,
+        }
+    }
+}
+SUB_RESOURCE_ATTRIBUTE_MAP = {}
+ACTION_MAP = {}
+REQUIRED_EXTENSIONS = [l3.ALIAS, l3_ext_gw_multihoming.ALIAS]
+OPTIONAL_EXTENSIONS = []
+ACTION_STATUS = {}

neutron_lib/api/definitions/l3_ext_gw_multihoming.py

@@ -0,0 +1,49 @@
+# Copyright 2023 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from neutron_lib.api.definitions import l3
+from neutron_lib.api.definitions import l3_ext_gw_mode
+
+EXTERNAL_GATEWAYS = 'external_gateways'
+
+ALIAS = 'external-gateway-multihoming'
+IS_SHIM_EXTENSION = False
+IS_STANDARD_ATTR_EXTENSION = False
+NAME = 'Neutron L3 External Gateway Multihoming'
+API_PREFIX = ''
+DESCRIPTION = 'Allow multiple external gateway ports per router'
+UPDATED_TIMESTAMP = '2023-01-18T00:00:00-00:00'
+RESOURCE_NAME = l3.ROUTER
+COLLECTION_NAME = l3.ROUTERS
+RESOURCE_ATTRIBUTE_MAP = {
+    COLLECTION_NAME: {
+        EXTERNAL_GATEWAYS: {
+            'allow_post': False,
+            'allow_put': False,
+            'is_visible': True,
+            'default': None,
+            'validate': {'type:external_gw_info_list': None},
+        },
+    },
+}
+SUB_RESOURCE_ATTRIBUTE_MAP = {}
+ACTION_MAP = l3.ACTION_MAP
+ACTION_MAP[l3.ROUTER].update({
+    'add_external_gateways': 'PUT',
+    'update_external_gateways': 'PUT',
+    'remove_external_gateways': 'PUT',
+})
+REQUIRED_EXTENSIONS = [l3.ALIAS, l3_ext_gw_mode.ALIAS]
+OPTIONAL_EXTENSIONS = []
+ACTION_STATUS = {}

neutron_lib/api/definitions/qos_gateway_ip.py

@@ -11,11 +11,9 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-from neutron_lib.api import converters
 from neutron_lib.api.definitions import l3
 from neutron_lib.api.definitions import l3_ext_gw_mode
 from neutron_lib.api.definitions import qos
-from neutron_lib.services.qos import constants as qos_consts
 
 
 ALIAS = 'qos-gateway-ip'
@@ -35,22 +33,7 @@ RESOURCE_ATTRIBUTE_MAP = {
             'is_visible': True,
             'default': None,
             'enforce_policy': True,
-            'validate': {
+            'validate': {'type:external_gw_info': None},
-                'type:dict_or_nodata': {
-                    'network_id': {'type:uuid': None, 'required': True},
-                    'enable_snat': {'type:boolean': None, 'required': False,
-                                    'convert_to':
-                                        converters.convert_to_boolean},
-                    'external_fixed_ips': {
-                        'type:fixed_ips': None,
-                        'required': False
-                    },
-                    qos_consts.QOS_POLICY_ID: {
-                        'type:uuid_or_none': None,
-                        'required': False
-                    }
-                }
-            }
         }
     }
 }

neutron_lib/api/validators/__init__.py

@@ -28,7 +28,7 @@ from neutron_lib._i18n import _
 from neutron_lib import constants
 from neutron_lib import exceptions as n_exc
 from neutron_lib.plugins import directory
-
+from neutron_lib.services.qos import constants as qos_consts
 
 LOG = logging.getLogger(__name__)
 
@@ -1215,6 +1215,42 @@ def validate_ethertype(ethertype, valid_values=None):
         return _(msg) % msg_data
 
 
+def validate_external_gw_info(data, valid_values=None):
+    """Validate data is an external_gateway_info.
+
+    :param data: The data to validate.
+    :param valid_values: Not used!
+    :returns: None if valid, error string otherwise.
+    """
+    # See https://github.com/PyCQA/pylint/issues/850
+    # pylint: disable=import-outside-toplevel,cyclic-import
+    from neutron_lib.api import converters
+    return validate_dict_or_nodata(
+        data,
+        key_specs={
+            'network_id': {'type:uuid': None, 'required': True},
+            'external_fixed_ips': {'type:fixed_ips': None, 'required': False},
+            'enable_snat': {'type:boolean': None, 'required': False,
+                            'convert_to': converters.convert_to_boolean},
+            qos_consts.QOS_POLICY_ID: {
+                'type:uuid_or_none': None,
+                'required': False
+            }
+        }
+    )
+
+
+def validate_external_gw_info_list(data, valid_values=None):
+    """Validate data is a list of external_gateway_info.
+
+    :param data: The data to validate.
+    :param valid_values: Not used!
+    :returns: None if valid, error string otherwise.
+    """
+    if data is not None:
+        return _validate_list_of_items(validate_external_gw_info, data)
+
+
 # Dictionary that maintains a list of validation functions
 validators = {'type:dict': validate_dict,
               'type:dict_or_none': validate_dict_or_none,
@@ -1265,7 +1301,9 @@ validators = {'type:dict': validate_dict,
               'type:service_plugin_type': validate_service_plugin_type,
               'type:list_of_subnets_or_none': validate_subnet_list_or_none,
               'type:list_of_subnet_service_types':
-                  validate_subnet_service_types
+                  validate_subnet_service_types,
+              'type:external_gw_info': validate_external_gw_info,
+              'type:external_gw_info_list': validate_external_gw_info_list,
               }
 
 

neutron_lib/constants.py

@@ -148,6 +148,8 @@ L3_AGENT_SCHEDULER_EXT_ALIAS = 'l3_agent_scheduler'
 DHCP_AGENT_SCHEDULER_EXT_ALIAS = 'dhcp_agent_scheduler'
 L3_DISTRIBUTED_EXT_ALIAS = 'dvr'
 L3_HA_MODE_EXT_ALIAS = 'l3-ha'
+L3_ENABLE_DEFAULT_ROUTE_ECMP_ALIAS = 'enable-default-route-ecmp'
+L3_ENABLE_DEFAULT_ROUTE_BFD_ALIAS = 'enable-default-route-bfd'
 SUBNET_ALLOCATION_EXT_ALIAS = 'subnet_allocation'
 
 # Protocol names and numbers for Security Groups/Firewalls

neutron_lib/exceptions/l3_ext_gw_multihoming.py

@@ -0,0 +1,33 @@
+# Copyright 2023 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from neutron_lib._i18n import _
+from neutron_lib import exceptions
+
+
+class UnableToAddExtraGateways(exceptions.NeutronException):
+    message = _("Unable to add extra gateways to a router %(router_id)s")
+
+
+class UnableToRemoveGateways(exceptions.NeutronException):
+    message = _("Unable to remove extra gateways from a router %(router_id)s")
+
+
+class UnableToMatchGateways(exceptions.NeutronException):
+    message = _("Unable to match a requested gateway port to"
+                " existing gateway ports for router %(router_id)s")
+
+
+class UnableToAddExtraGatewayPort(exceptions.NeutronException):
+    message = _("Unable to add an gateway port to a router %(router_id)s")

neutron_lib/tests/unit/api/definitions/test_l3_enable_default_route_bfd.py

@@ -0,0 +1,24 @@
+# Copyright 2023 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from neutron_lib.api.definitions import l3_enable_default_route_bfd
+from neutron_lib.tests.unit.api.definitions import base
+
+
+class L3EnableDefaultRouteBFDDefinitionTestCase(
+        base.DefinitionBaseTestCase):
+    extension_module = l3_enable_default_route_bfd
+    extension_resources = ()
+    extension_attributes = (
+        l3_enable_default_route_bfd.ENABLE_DEFAULT_ROUTE_BFD,)

neutron_lib/tests/unit/api/definitions/test_l3_enable_default_route_ecmp.py

@@ -0,0 +1,24 @@
+# Copyright 2023 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from neutron_lib.api.definitions import l3_enable_default_route_ecmp
+from neutron_lib.tests.unit.api.definitions import base
+
+
+class L3EnableDefaultRouteECMPDefinitionTestCase(
+        base.DefinitionBaseTestCase):
+    extension_module = l3_enable_default_route_ecmp
+    extension_resources = ()
+    extension_attributes = (
+        l3_enable_default_route_ecmp.ENABLE_DEFAULT_ROUTE_ECMP,)

neutron_lib/tests/unit/api/definitions/test_l3_multi_ext_gw.py

@@ -0,0 +1,21 @@
+# Copyright 2023 Canonical Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from neutron_lib.api.definitions import l3_ext_gw_multihoming
+from neutron_lib.tests.unit.api.definitions import base
+
+
+class L3ExternalGatewayMultihomingDefinitionTestCase(
+        base.DefinitionBaseTestCase):
+    extension_module = l3_ext_gw_multihoming

releasenotes/notes/l3-ext-gw-multihoming-1a7b556c541923cf.yaml

@@ -0,0 +1,14 @@
+---
+features:
+  - |
+    A definition for API extension: ``external-gateway-multihoming``.
+    Allows multiple gateway ports to be configured per router for the L3
+    service plugins implementing this functionality.
+  - |
+    A definition for API extension: ``enable-default-route-ecmp``.
+    Allows ECMP to be enabled or disabled for default routes inferred from
+    subnets of router's gateway ports.
+  - |
+    A definition for API extension: ``enable-default-route-bfd``.
+    Allows BFD to be enabled or disabled for default routes inferred from
+    subnets of router's gateway ports.