Browse Source

Add bonding configuration fields to portgroups

This change:

* adds mode and properties fields to the portgroup objects and
  describes the changes to nova needed to use them;
* makes portgroup.address field not mandatory;
* brings some updates to CLI & OSC CLI;
* makes this spec dependent on attach/detach interfaces
  work;
* contains some minor cosmetic changes for more readability.

Related-bug: #1618754
Change-Id: I7d1c72e80826d077e36e60443aa7573a1c81646a
changes/10/396610/14
Vladyslav Drok 3 years ago
parent
commit
42a2a1833e
1 changed files with 260 additions and 204 deletions
  1. +260
    -204
      specs/approved/ironic-ml2-integration.rst

+ 260
- 204
specs/approved/ironic-ml2-integration.rst View File

@@ -66,8 +66,7 @@ port. In this way, each portgroup can represent a LAG for which all member
switch ports will belong to the same network and to the same network segment.

The node should first be placed on a provisioning network and then onto the
tenant network as specified in the `network-provider spec
<https://blueprints.launchpad.net/ironic/+spec/network-provider>`_. To help
tenant network as specified in the network-provider spec [1]_. To help
facilitate this there will be a variable stored in the binding profile to
record whether a port binding has been requested. This will allow Ironic to
defer binding until Ironic is able to populate the extra information that is
@@ -120,16 +119,41 @@ addition to the base object it will have the following fields:
* name
* node_id
* address
* mode
* properties
* extra
* internal_info
* standalone_ports_supported

The 'address' field represents the MAC address for bonded NICs of the bare
metal server. The 'extra' field can be used to hold any additional information
The ``address`` field represents the MAC address for bonded NICs of the bare
metal server. It is optional, as its value is highly dependent on the instance
OS. In case of using ironic through nova, the ``address`` gets populated by
whatever value is generated by neutron for the VIF's address representing this
portgroup, if there is one. This happens during every VIF attach call.
In case of standalone mode, it can be always ``None``, as it should be
configured on the instance through the configdrive.

The ``mode`` field is used to specify the bond mode. If not provided in the
portgroup creation request, its default value is determined by the
``[DEFAULT]default_portgroup_mode`` configuration option, which in turn has a
default value of ``active-backup``. For a portgroup that was created in an
older API version, this configuration value will be used as the value for that
portgroup's mode. ``active-backup`` is chosen as the default because this mode
does not require any additional configuration on the switch side.

The ``properties`` field is a dictionary that can contain any parameters
needed to configure the portgroup.

For additional information about ``mode`` and ``properties`` fields' contents,
see `linux kernel bonding documentation <https://www.kernel.org/doc/Documentation/networking/bonding.txt>`_.

The ``extra`` field can be used to hold any additional information
that operators or developers want to store in the portgroup.
The 'internal_info' field is used to store internal metadata. This field is

The ``internal_info`` field is used to store internal metadata. This field is
read-only.
The 'standalone_ports_supported' indicates whether ports that are members of

The ``standalone_ports_supported`` indicates whether ports that are members of
this portgroup can be used as stand-alone ports.

The Ironic port object will then have the following fields added to support
@@ -161,47 +185,31 @@ The following port binding related information needs to be passed to Neutron:
| host_id | This should be set to the Ironic node uuid. |
+------------------------+--------------------------------------------------+

A JSON example to describe the structure is:
A JSON example to describe the structure is::

{"port":
{
{
"port": {
<all other fields>,

"vnic_type": "baremetal",

"host_id": <Ironic node UUID>,

"binding:profile": {

"local_link_information": [
{
"switch_id": xxx,

"port_id": xxx,

"switch_info": zzz,

<optional more information>

},

{
"switch_id": xxx,

"port_id": yyy,

"switch_info": zzz,

<optional more information>

} ]

<some more profile fields>

"local_link_information": [
{
"switch_id": xxx,
"port_id": xxx,
"switch_info": zzz,
<optional more information>
},
{
"switch_id": xxx,
"port_id": yyy,
"switch_info": zzz,
<optional more information>
}
]
<some more profile fields>
}

}

}


@@ -249,7 +257,11 @@ The portgroup object is proposed with the following fields and data types:
+----------------------------+-------------------------+
| node_id | int_or_none |
+----------------------------+-------------------------+
| address | str |
| address | str_or_none |
+----------------------------+-------------------------+
| mode | str_or_none |
+----------------------------+-------------------------+
| properties | dict_or_none |
+----------------------------+-------------------------+
| extra | dict_or_none |
+----------------------------+-------------------------+
@@ -262,6 +274,13 @@ The portgroup object is proposed with the following fields and data types:
| updated_at | datetime_or_str_or_none |
+----------------------------+-------------------------+

.. note::
While ``mode`` attribute of the portgroup object has type str_or_none, its
value can not be ``None``, unless the database was changed manually. It
gets populated either during the database migration, or during portgroup
creation (if not specified explicitly). In both cases it is set to the
``[DEFAULT]default_portgroup_mode`` configuration option value.

State Machine Impact
--------------------

@@ -736,11 +755,28 @@ model:
"rel": "bookmark"
}
],
"mode": "802.3ad",
"name": "node1_portgroup1",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"ports": [
{
"href": "http://127.0.0.1:6385/v1/portgroups/
6eb02b44-18a3-4659-8c0b-8d2802581ae4/ports",
"rel": "self"
},
{
"href": "http://127.0.0.1:6385/portgroups/
6eb02b44-18a3-4659-8c0b-8d2802581ae4/ports",
"rel": "bookmark"
}
],
"properties": {
"bond_xmit_hash_policy": "layer3+4",
"bond_miimon": 100
},
"standalone_ports_supported": true,
"updated_at": "2015-05-15T09:04:12.011844+00:00",
"uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4",
"name": "node1_portgroup1"
"uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4"
}

* JSON schema definition of PortgroupCollection:
@@ -792,6 +828,7 @@ model:
"rel": "bookmark"
}
],
"mode": "802.3ad",
"name": "node1_portgroup1",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"ports": [
@@ -806,6 +843,10 @@ model:
"rel": "bookmark"
}
],
"properties": {
"bond_xmit_hash_policy": "layer3+4",
"bond_miimon": 100
},
"standalone_ports_supported": true,
"updated_at": "2016-11-04T17:46:09+00:00",
"uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4"
@@ -831,14 +872,14 @@ by the caller and the response.
::

{
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"local_link_connection": {
"switch_id": "0a:1b:2c:3d:4e:5f",
"port_id": "Ethernet3/1",
"switch_info": "switch1",
},
"pxe_enabled": true
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"local_link_connection": {
"switch_id": "0a:1b:2c:3d:4e:5f",
"port_id": "Ethernet3/1",
"switch_info": "switch1",
},
"pxe_enabled": true
}

* Response 201 with body:
@@ -846,32 +887,32 @@ by the caller and the response.
::

{
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"local_link_connection": {
"switch_id": "0a:1b:2c:3d:4e:5f",
"port_id": "Ethernet3/1",
"switch_info": "switch1",
},
"pxe_enabled": true
"created_at": "2015-05-12T10:00:00.529243+00:00",
"extra": {
},
"links": [
{
"href": "http://localhost:6385/v1/ports/
1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"rel": "self"
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"local_link_connection": {
"switch_id": "0a:1b:2c:3d:4e:5f",
"port_id": "Ethernet3/1",
"switch_info": "switch1",
},
"pxe_enabled": true
"created_at": "2015-05-12T10:00:00.529243+00:00",
"extra": {
},
{
"href": "http://localhost:6385/ports/
1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"rel": "bookmark"
}
],
"updated_at": null,
"uuid": "1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"portgroup_uuid": null,
"links": [
{
"href": "http://localhost:6385/v1/ports/
1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"rel": "self"
},
{
"href": "http://localhost:6385/ports/
1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"rel": "bookmark"
}
],
"updated_at": null,
"uuid": "1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"portgroup_uuid": null,
}

* Example of portgroup create.
@@ -881,10 +922,10 @@ by the caller and the response.
::

{
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"standalone_ports_supported": true,
"name": "node1_portgroup1"
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"standalone_ports_supported": true,
"name": "node1_portgroup1"
}

* Response 201 with body:
@@ -892,28 +933,42 @@ by the caller and the response.
::

{
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"name": "node1_portgroup1"
"created_at": "2015-05-12T10:10:00.529243+00:00",
"extra": {
},
"internal_info": {},
"links": [
{
"href": "http://localhost:6385/v1/portgroups/
6eb02b44-18a3-4659-8c0b-8d2802581ae4",
"rel": "self"
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"name": "node1_portgroup1"
"created_at": "2015-05-12T10:10:00.529243+00:00",
"extra": {
},
{
"href": "http://localhost:6385/portgroups/
6eb02b44-18a3-4659-8c0b-8d2802581ae4",
"rel": "bookmark"
}
],
"standalone_ports_supported": true,
"updated_at": null,
"uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4",
"internal_info": {},
"links": [
{
"href": "http://localhost:6385/v1/portgroups/
6eb02b44-18a3-4659-8c0b-8d2802581ae4",
"rel": "self"
},
{
"href": "http://localhost:6385/portgroups/
6eb02b44-18a3-4659-8c0b-8d2802581ae4",
"rel": "bookmark"
}
],
"mode": null,
"ports": [
{
"href": "http://127.0.0.1:6385/v1/portgroups/
6eb02b44-18a3-4659-8c0b-8d2802581ae4/ports",
"rel": "self"
},
{
"href": "http://127.0.0.1:6385/portgroups/
6eb02b44-18a3-4659-8c0b-8d2802581ae4/ports",
"rel": "bookmark"
}
],
"properties": {},
"standalone_ports_supported": true,
"updated_at": null,
"uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4",
}

* Example of port update.
@@ -932,32 +987,32 @@ by the caller and the response.
::

{
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"local_link_connection": {
"switch_id": "0a:1b:2c:3d:4e:5f",
"port_id": "Ethernet3/1",
"switch_info": "switch1",
},
"pxe_enabled": true
"created_at": "2015-05-12T10:00:00.529243+00:00",
"extra": {
},
"links": [
{
"href": "http://localhost:6385/v1/ports/
1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"rel": "self"
"address": "fe:54:00:77:07:d9",
"node_uuid": "e7a6f1e2-7176-4fe8-b8e9-ed71c77d74dd",
"local_link_connection": {
"switch_id": "0a:1b:2c:3d:4e:5f",
"port_id": "Ethernet3/1",
"switch_info": "switch1",
},
"pxe_enabled": true
"created_at": "2015-05-12T10:00:00.529243+00:00",
"extra": {
},
{
"href": "http://localhost:6385/ports/
1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"rel": "bookmark"
}
],
"updated_at": "2015-05-12T10:20:00.529243+00:00",
"uuid": "1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"portgroup_uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4",
"links": [
{
"href": "http://localhost:6385/v1/ports/
1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"rel": "self"
},
{
"href": "http://localhost:6385/ports/
1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"rel": "bookmark"
}
],
"updated_at": "2015-05-12T10:20:00.529243+00:00",
"uuid": "1004e542-2f9f-4d9b-b8b9-5b719fa6613f",
"portgroup_uuid": "6eb02b44-18a3-4659-8c0b-8d2802581ae4",
}

* Note that the port update API should support updating the portgroup_id
@@ -1021,6 +1076,10 @@ Client (CLI) impact
The python-ironicclient and OSC would need updated to support the new
portgroups APIs.

In the commands below, ``<portgroup>`` means that this placeholder can contain
both portgroup UUID or name. ``<portgroup_uuid>`` can contain only portgroup
UUID.

Example usage of the new methods:

* For ports, the CLI would support port creation with new optional
@@ -1031,50 +1090,54 @@ Example usage of the new methods:
"ironic" CLI:

* ironic port-create -a <address> -n <node> [-e <key=value>]
[--local-link-connection <local_link_connection>]
[--portgroup-uuid <portgroup_uuid>] [--pxe-enabled <pxe_enabled>]

* ironic port-update port_uuid replace portgroup_uuid=<portgroup_uuid>

* ironic port-list [--detail] [--address <mac-address>]
[--portgroup-uuid <portgroup_uuid>]
[--local-link-connection <key=value>]
[--portgroup <portgroup>] [--pxe-enabled <boolean>]

* ironic port-update <port_uuid> add portgroup_uuid=<portgroup_uuid>
--local-link-connection <key=value> --pxe-enabled <boolean>

"openstack baremetal" CLI:

* openstack baremetal port create --node <node>
[--local-link-connection <key=value>]
[--portgroup-uuid <portgroup_uuid>]
[--pxe-enabled <boolean>]
<address>
[--local-link-connection <key=value>] [--port-group <portgroup>]
[--pxe-enabled <boolean>] <address>

* openstack baremetal port set [--portgroup-uuid <portgroup_uuid>]
* openstack baremetal port set [--port-group <portgroup>]
[--local-link-connection <key=value>] [--pxe-enabled <boolean>]
<port>

* openstack baremetal port list --address <mac-address>]
[--node <node>] [--portgroup-uuid <portgroup_uuid>]
* openstack baremetal port list [--address <mac-address>]
[--node <node> | --port-group <portgroup>]


* For portgroups, the CLI would support the following new methods:

"ironic" CLI:

* ironic portgroup-create --node <node> [--name <portgroupname>]
[--address <mac-address>] [-e <key=value>]
* ironic portgroup-create --node <node> [--address <mac-address>]
[--name <portgroupname>] [-e <key=value>]
[--standalone-ports-supported <boolean>] [-m <mode>]
[-p <key=value>]

* ironic portgroup-delete <portgroup_uuid>
* ironic portgroup-delete <portgroup> [<portgroup> ...]

* ironic portgroup-list [--detail] [--node <node>]
[--address <mac-address>]
[--limit <limit>] [--marker <portgroup_uuid] [--sort-key <field>]
* ironic portgroup-list [--detail | --fields <field> [<field> ...]]
[--node <node>] [--address <mac-address>] [--limit <limit>]
[--marker <portgroup_uuid>] [--sort-key <field>]
[--sort-dir <direction>]

* ironic portgroup-show [--address] <id>
* ironic portgroup-port-list
[--detail | --fields <field> [<field> ...]]
[--limit <limit>] [--marker <portgroup_uuid>] [--sort-key <field>]
[--sort-dir <direction>] <portgroup>

* ironic portgroup-show [--address] [--fields <field> [<field> ...]]
<id>

* <id> is the UUID of the portgroup (or MAC address if --address is
specified)
* <id> is the UUID or name of the portgroup (or MAC address if
--address is specified)

* ironic portgroup-update <portgroup_uuid> <op> <path=value>
* ironic portgroup-update <portgroup> <op> <path=value>
[<path=value> ... ]

* <op> is add, remove or replace.
@@ -1089,30 +1152,33 @@ Example usage of the new methods:

"openstack baremetal" CLI:

* openstack baremetal portgroup create --node <uuid> [--name NAME]
* openstack baremetal port group create --node <uuid> [--name <name>]
[--extra <key=value>]
[--support-standalone-ports | --unsupport-standalone-ports]
<address>
[--mode <mode>] [--properties <key=value>]
[--address <mac-address>]

* openstack baremetal portgroup delete <portgroup> [<portgroup> ...]
* openstack baremetal port group delete <portgroup> [<portgroup> ...]

* openstack baremetal portgroup list [--marker <portgroup>]
* openstack baremetal port group list [--marker <portgroup>]
[--address <mac-address>] [--node <node>]
[--sort <key>[:<direction>]]
[--long | --fields <field> [<field> ...]]

* openstack baremetal portgroup show [--address]
[--fields <field> [<field> ...]]
<portgroup>
* openstack baremetal port group show [--address]
[--fields <field> [<field> ...]] <id>

* <id> is the UUID or name of the portgroup (or MAC address if
--address is specified)

* openstack baremetal portgroup set [--address] [--name NAME]
[--node <uuid>] [--extra <key=value>]
* openstack baremetal port group set [--address <mac-address>]
[--name <name>] [--node <node>] [--extra <key=value>]
[--support-standalone-ports | --unsupport-standalone-ports]
[--fields <field> [<field> ...]]
[--mode <mode>] [--properties <key=value>]
<portgroup>

* openstack baremetal portgroup unset [--name] [--extra <key>]
[--node <uuid>] <portgroup>
* openstack baremetal port group unset [--address] [--name]
[--extra <key>] [--properties key] <portgroup>


* To add ports to a portgroup, the portgroup should first
@@ -1149,8 +1215,7 @@ extracting ``vif_port_id`` from the ``extra`` attributes of Ironic ports.
This method should be updated if vifs are bound to portgroups as well as
ports.

The complementary `network-provider spec
<https://blueprints.launchpad.net/ironic/+spec/network-provider>`_ provides
The complementary network-provider spec [1]_ provides
details regarding the workflow of the network flip and the point at which
the binding profile will be passed to Neutron to bind the port.

@@ -1159,31 +1224,17 @@ the binding profile will be passed to Neutron to bind the port.
Nova driver impact
------------------

There will be changes necessary to the Nova driver. Proposed changes are:

* To enable the mapping between Neutron ports and Ironic ports and
portgroups.

The Ironic Nova driver has methods ``macs_for_instance``,
``dhcp_options_for_instance``, ``extra_options_for_instance`` and
``plug_vifs``. Currently Nova puts a network on one port at random - see
`ports cannot be mapped to networks
<https://bugs.launchpad.net/ironic/+bug/1405131>`_. This bug has high
priority and the issue is being addressed. Once addressed, these methods
should determine the number of Neutron ports that are
created as well as the mapping between Neutron and Ironic ports. These
methods should be updated to not only account for Ironic ports but also
Ironic portgroups. The selection process would be:

* Select all Ironic ports that do not belong to Ironic portgroups
(possible if the Ironic port list API returns portgroup_uuid as
standard, as suggested in the above section)

* Select all Ironic portgroups
As this work depends on the attach/detach interface work [2]_, the only thing
that needs to be changed to fully support portgroups is configdrive
generation.

This modified functionality could be implemented using a new config flag in
Nova to allow toggling between the old and the new methods. The flag could
help de-couple the upgrading of Nova and of Ironic.
Nova will call into ironic to get the list of ports of each portgroup that has
a VIF associated with it, along with portgroup ``mode`` and ``properties``
fields (see `Data model impact`_ section), and update the network metadata
with the needed information. When the contents of the ``properties``
dictionary gets passed to the config drive builder in nova, we will ensure
that ``bond_`` prefix is prepended to all key names, so that these keys are
not ignored by cloud-init when reading the config drive.

Ramdisk impact
--------------
@@ -1210,8 +1261,7 @@ Using the binding profile to enable flipping between provisioning and tenant
networks means there will be no support for PXE booting after deploy (i.e.
local disk installation only). How to allow operators to deploy instances
using either net-boot or local boot using the same Ironic conductor should be
discussed in the complementary `network-provider spec
<https://blueprints.launchpad.net/ironic/+spec/network-provider>`_.
discussed in the complementary network-provider spec [1]_.

Scalability impact
------------------
@@ -1236,9 +1286,10 @@ Deployers will need to deploy an ML2 mechanism driver that supports connecting
baremetal resources to Neutron networks.

If using Nova, deployers will need to deploy a version of Nova that supports
this feature. Deployers will need to set a flag in the Nova config file to
turn this new feature on or off, which would be important when upgrading
Nova and Ironic.
this feature.

Deployers may want to set the ``[DEFAULT]default_portgroup_mode`` configuration
option to match their environment. Its default value is ``active-backup``.

Deployers should be aware that automated upgrade or migration for
already-provisioned nodes is not supported. Deployers should follow this
@@ -1247,12 +1298,9 @@ new feature:

* Upgrade the OpenStack services.

* Update the flag in the Nova config file to turn this feature on.

* Move node into the MANAGEABLE state.

* Update node driver field (see `network-provider spec
<https://blueprints.launchpad.net/ironic/+spec/network-provider>`_).
* Update node driver field (see the network-provider spec [1]_).

* Create Ironic portgroups.

@@ -1286,6 +1334,10 @@ Assignee(s)

* sukhdev-8

* vsaienko

* vdrok

Work Items
----------

@@ -1314,8 +1366,10 @@ Work Items
Dependencies
============

Network flip is dependent on `network-provider spec
<https://blueprints.launchpad.net/ironic/+spec/network-provider>`_.
Network flip is dependent on the network-provider spec [1]_.

Nova `changes <https://blueprints.launchpad.net/nova/+spec/ironic-portgroups-support>`_
are dependent on attach/detach interfaces work [2]_.

VLAN provisioning on switch(es) is dependent on ML2 driver functionality
being developed to support this feature.
@@ -1330,8 +1384,7 @@ New tests will need to be written to test the new APIs and database
updates.

Simulation of connecting real hardware to real switches for testing
purposes is described in `network-provider spec
<https://blueprints.launchpad.net/ironic/+spec/network-provider>`_.
purposes is described in the network-provider spec [1]_.


Upgrades and Backwards Compatibility
@@ -1350,13 +1403,16 @@ This feature will be fully documented.
References
==========

.. [1] http://specs.openstack.org/openstack/ironic-specs/specs/6.1/network-provider.html

.. [2] http://specs.openstack.org/openstack/ironic-specs/specs/approved/interface-attach-detach-api.html

Discussions on the topic include:

* https://etherpad.openstack.org/p/YVR-neutron-ironic

* https://etherpad.openstack.org/p/liberty-ironic-network-isolation

* Logs from https://wiki.openstack.org/wiki/Meetings/Ironic-neutron
* https://etherpad.openstack.org/p/network-interface-vifs-configdrive

* The network provider spec enabling the network flip between provisioning
and tenant network: https://review.openstack.org/#/c/187829
* Logs from https://wiki.openstack.org/wiki/Meetings/Ironic-neutron

Loading…
Cancel
Save