22 KiB
- This work is licensed under a Creative Commons Attribution 3.0 Unported
-
License.
http://creativecommons.org/licenses/by/3.0/legalcode
Introspection LLDP reporting
Link Layer Discovery Protocol (LLDP) packets are transmitted periodically by network switches out each switch port in accordance with IEEE Std 802.1AB-2016. The protocol is used to advertise the switch port's capabilities and configuration. The LLDP packets are gathered by Ironic Python Agent (IPA) running on each node and stored per interface in the Ironic Inspector database in the same type, length, value (TLV) format as they are received per the IPA change to add LLDP support. The LLDP data contains switch port VLAN assignments, MTU sizes, link aggregation configuration, etc., offering a rich set of networking information that can be used for planning and troubleshooting a deployment from the point of view of Ironic nodes.
This proposal is for additional Ironic Inspector hooks to parse the raw LLDP TLVs and store the results back to Swift as part of the interface data. The processed LLDP data can be accessed directly or displayed via a client side CLI. This proposal includes the python-ironic-inspector-client CLI commands to display the processed LLDP data. The data should be displayed in a format that will allow a user to quickly see the network switch port configuration and detect mismatches within and between nodes. For example, it should be easy to see whether a particular VLAN is configured on each node's eth0 interface, or detect that all switch ports connected to all nodes are configured to support jumbo frames.
Problem description
Network switch configuration problems can be a major source of problems when doing Openstack deployments and are difficult to detect and diagnose. There may be many network switch ports that multiple baremetal nodes are connected to and the switch port configuration may not match the deployment parameters. The user doing the Openstack deployment may not have access to the network switches to view the port configurations, or may not be familiar with the particular switch user interface.
Some potential mismatches between switch port configuration and the Openstack deployment's node interface settings are:
- VLAN configuration
- Untagged VLAN ID on provisioning network
- MTU sizes
- Link aggregation (aka bonding) configuration
Proposed change
The scope of the proposed change is to parse the LLDP data captured by Ironic Python Agent by Ironic Inspector plugins TLVs as defined in IEEE Std 802.1AB-2016 and display the data in a user friendly format. It's beyond scope to determine if an Openstack deployment matches the switch port configuration, but the CLI commands can be used to verify and validate the deployment configuration.
A new Ironic Inspector hook (aka plugin) will be added to parse all standard LLDP data and store the data per interface. Not all LLDP TLVs as defined in the specification are sent by every network switch. However, it appears to be standard practice that switches that support the standard will implement the basic management, IEEE Std 802.1Q-2014, and IEEE Std 802.3-2012 optional TLVs. The new Ironic Inspector hook must support the following mandatory and optional TLVs:
- Chassis ID TLV (mandatory)
- Port ID TLV (mandatory)
- Basic Management TLV Set (optional)
- Port Description TLV
- System Name TLV
- System Description TLV
- System Capabilities TLV
- Management Address TLV
- IEEE 802.1 Organizationally Specific TLV Set (optional)
- Port VLAN ID TLV
- Port And Protocol VLAN ID TLV
- VLAN Name TLV
- Protocol Identity TLV
- Management VID TLV
- Link Aggregation TLV
- IEEE 802.3 Organizationally Specific TLV Set (optional)
- MAC/Phy Config/Status TLV (includes duplex/speed/autoneg)
- Link Aggregation TLV
- Maximum Frame Size MTU
The LLDP-MED TLV set was developed to support IP telephony and as such isn't relevant for a LAN environment. The LLDP-MED TLVs that are received could be handled by a separate processing hook that will parse and store the decoded TLVs. Processing hooks can be stacked such that the standard LLDP processing hook can be run in addition to the LLDP-MED hook. The development of a LLDP-MED processing hook is out of scope for this effort.
Some network switches send vendor specific TLVs. The definition of these TLVs may not be widely available or consistent across releases. In order to support the parsing of these TLVs, vendor specific hooks could be added to the other LLDP hooks in the future, although vendor-specific hooks are out of scope for this effort.
The hooks can be enabled in the inspector.conf
file. By
default, these LLDP plugins will not be enabled. Background information
on ironic-inspector plugins is here: ironic-inspector
plugins.
The new plugins will reside in ironic-inspector. The new CLI commands will be in python-ironic-inspector-client.
Alternatives
This proposed implementation uses data captured by IPA during
introspection, it is not a real-time LLPD monitoring tool like
lldpd
. The data used could be outside the Time To Live
(TTL) window and therefore considered not valid. Any configuration
changes made at the switch would not be detected until the user runs
introspection again.
The major advantage of this approach is that the data is cached by inspector for all nodes and their interfaces. This implementation is most useful as a tool to point to potential mismatches between the switch and the deployment configuration, rather than as an absolute source of truth about the real-time switch configuration.
Data model impact
Currently Ironic Inspector stores received LLDP data in Swift in raw type/value format for each interface. For example, here is a subset of stored LLDP TLVs showing Chassis ID (Type 1), Port ID (Type 2), System Name (Type 5), and VLANs (Type 127 and OUI for 802.1 - 0x0080c2). Two VLAN TLVs are shown that map to VLANs 100 and 101.
"lldp": [
[
1,
"0464649b32f300"
],
[
2,
"07373231"
[
5,
"737730312d646973742d31622d6231322e72647532"
],
[
127,
"0080c203006407766c616e313030"
],
[
127,
"0080c203006507766c616e313031"
],
...
The proposed Ironic Inspector processing hooks will parse this LLDP
data and update the data store with an lldp_processed
struct per interface containing name/value pairs. This new struct will
be stored under all_interfaces
.
Note that in the raw data there may be multiple TLVs with the same TLV type/subtype. In some cases this is expected, for example there are individual VLAN TLVs for each configured VLAN. In other cases, multiple TLVs for the same type/subtype is unexpected, perhaps due to the switch sending the same TLV twice or IPA receiving them out of order, etc. The unexpected case still needs to be handled.
Depending on the TLV type, the hook will store the data as either a
name/list or name/value binding. The name/value will be for TLVs that
should only have a single value, as with chassis_id
, while
the name/list is for data that can incorporate multiple TLVs with the
same type/subtype, for example VLANs. Data stored in the list each entry
must be unique, there cannot be duplicate list entries.
system_capabilities
and port_capabilites
TLVs
can be handled as a list in the same way as VLANs.
For TLVs that map to a single name/value pair, i.e.
chassis_id
, port_id
,
autonegotiation_enabled
etc. a check must be made to ensure
that duplicate TLV(s) are not processed. In other words, if a name/value
pair for chassis_id
has already been stored it will not be
overwritten.
Example processed content is shown below.
all_interfaces": {"eth0":
{"ip": null, "mac": "a0:36:xx:xx:xx",
"lldp_processed": {
"switch_chassis_id": "64:64:9b:xx:xx:xx",
"switch_port_id": "734",
"switch_system_name": "sw01-bld2",
"switch_port_physical_capabilities" : ['100Base-TX hdx',
'100BASE-TX fdx',
'1000BASE-T fdx'],
"switch_port_mtu" : "9216",
"switch_port_link_aggregation_support": "True",
"switch_port_link_aggregation_enabled": "False",
"switch_port_autonegotiation_support" "True",
"switch_port_autonegotiation_enabled" "True",
"switch_port_vlans": [{"name": "vlan101", "id": 101},
{"name": "vlan102", "id": 102},
{"name": "vlan104", "id": 103}],
...
}
}
}
Each processing hook will add additional named pairs to
lldp_processed
per interface. This allows both standard and
vendor specific hooks to run that can interpret all received LLDP TLVs.
Vendor specific plugins will only process TLVs that correspond to the
particular vendor as identified by the OUI in the Organizationally
Specific TLV (type 127). For example, Juniper uses OUI
0x009069
. Likewise an LLDP-MED hook will only process
Organizationally Specific TLVs with OUI 0x0012bb
. In this
way, individual TLVs are not processed more than once. However clashes
between the processed names used by the standard LLDP plugin and vendor
or LLDP-MED plugins needs to be avoided. For that reason the additional
plugins (beyond the standard plugin) will use the naming format:
<OUI>_<OUIsubtype>
:
where "OUI" is the string corresponding to the vendor OUI and "OUIsubtype" is the vendor specific subtype, e.g.:
"juniper_chassis_id": "0123456789"
Likewise, some examples for the LLDP-MED plugin:
"lldp_med_location_id": "5567892"
"lldp_med_device_type": "Network connectivity"
HTTP API impact
None.
Client (CLI) impact
To display the LLDP collected by Ironic python agent, a new set of
commands under openstack baremetal introspection
is
proposed as follows with example output.
- List interfaces for each node with key LLDP data.
$ openstack baremetal introspection interface list
5f428939-698d-4942-b164-ff645a768e4a
Interface | MAC | Switch VLAN IDs | Switch Chassis | Switch Port |
eth0 | b0... | [101, 102, 103] | 64:64:9b:xx:xx:xx | 554 |
eth1 | b0... | [101, 102, 103] | 64:64:9b:xx:xx:xx | 734 |
eth2 | b0... | [101, 102, 103] | 64:64:9b:xx:xx:xx | 587 |
eth3 | b0... | [101, 102] | 64:64:9b:xx:xx:xx | 772 |
2. Show all LLDP values for an interface. The field names will come directly from the names stored in the processed data.
$ openstack baremetal introspection interface show
5f428939-698d-4942-b164-ff645a768e4a eth0
Field | Value |
node | 5f428939-698d-4942-b164-ff645a768e4a |
interface | eth0 |
interface_mac_address | b0:83:fe:xx:xx:xx |
switch_chassis_id | 64:64:9b:xx:xx:x |
switch_port_id | 554 |
switch_system_name | sw01-dist-1b-b12.rdu2 |
switch_system_capabilities | ['Bridge', 'Router'] |
switch_port_description | host2.lab.eng port 1 (Prov/Trunked VLANs) |
switch_port_autonegotiation_support | True |
switch_port_autonegotiation_enabled | True |
switch_port_physical_capabilities | ['100Base-TX hdx', '100BASE-TX fdx', '1000BASE-T fdx'] |
switch_port_mau_type | Unknown |
switch_port_link_aggregation_support | True |
switch_port_link_aggregation_enabled | False |
switch_port_link_aggregation_id | 0 |
switch_port_mtu | 9216 |
switch_port_untagged_vlan_id | 102 |
switch_port_vlans |
|
- Show interface data filtered by particular VLANs
$ openstack baremetal introspection interface list
5f428939-698d-4942-b164-ff645a768e4a --vlan=103
Interface | MAC | Switch VLAN IDs | Switch Chassis | Switch Port |
eth0 | b0... | [101, 102, 103] | 64:64:9b:xx:xx:xx | 554 |
eth1 | b0... | [101, 102, 103] | 64:64:9b:xx:xx:xx | 734 |
eth2 | b0... | [101, 102, 103] | 64:64:9b:xx:xx:xx | 587 |
4. Show the value of provided field for each node/interface using the field names stored in the processed data and shown via the interface show command.
To show switch port MTU on a node for all interfaces:
$ openstack baremetal introspection interface list
5f428939-698d-4942-b164-ff645a768e4a --fields interface,
switch_port_mtu
Interface | switch_port_mtu |
eth0 | 9216 |
eth1 | 9216 |
eth2 | 1514 |
eth3 | 1514 |
To show the switch port link aggregation (aka bonding) configuration for a node:
$ openstack baremetal introspection interface list
22aadc81-e134-4ff0-ac53-229126e77f62 --fields interface,
switch_port_link_aggregation_enabled
Interface | switch_port_link_aggregation_enabled |
eth0 | False |
eth1 | False |
eth2 | True |
eth3 | True |
To show the switch port native VLAN configuration for a node and interface:
$ openstack baremetal introspection interface list --interface eth0
--fields interface, switch_port_untagged_vlan_id
Interface | switch_port_untagged_vlan_id |
eth0 | 102 |
5. To display the full LLDP processed report for all nodes in json
format the interface list
command can be run for all nodes
using the built-in arguments --long
(to display all fields)
and --format json
(to output in json format), for
example:
$ openstack baremetal introspection interface list
5f428939-698d-4942-b164-ff645a768e4a --long --format json
Ironic python agent impact
LLDP data collection is available in Newton but it must be enabled by
the kernel flag ipa-collect-lldp
.
Performance and scalability impact
Each time the new lldp commands are invoked, Ironic Inspector will be queried to get the LLDP data. Since the data has already been processed by the Inspector hook, there will be little additional processing that needs to be done to display the data.
Security impact
No sensitive or proprietary data will be displayed by these commands. All LLDP data was received as unencrypted UDP data.
These commands may provide a benefit for security audits of the deployment as they will make it possible to ensure that no systems are attached to unintended VLANs, thus reducing the possibility of accidental exposure.
In order for a switch to send LLDP packets, the network administrator must enable LLDP on the ports connected to node interfaces. A user on the Openstack CLI will be able to see everything that is sent in the LLDP packets including VLANs, management IP, switch model, port number, and firmware version. This information may potentially be used to organize attacks against networking equipment. For this reason the System Description TLV, which can include switch model, version, and build info, will be processed but not displayed; the Management Address TLV will be handled the same way. This will reduce the information available while still maintaining enough data for networking related validations.
Deployer impact
As discussed, these new commands may facilitate deployments as they could help detect mismatches between network switch configurations and deployment settings in areas such as VLANs, MTUs, bonding, port speed etc.
By default, the new plugins will be not be enabled. The deployer should set the standard LLDP hook in inspector.conf when in a baremetal environment.
In order to enable data collection in IPA, the deployer should set
the kernel flag ipa-collect-lldp=1
. Examples of setting
kernel parameters can be seen in configuring
PXE.
Developer impact
When the CLI is implemented, vendors will be able to develop vendor-specific plugins to handle vendor LLDP TLVs and expand the functionality.
Implementation
Assignee(s)
- Primary assignee::
Work Items
- Add processing hook to parse standard lldp data and write to data store.
- Integrate OSC commands with python-ironic-inspector-client.
- Add unit tests.
- Test with multiple vendors' network switches.
Dependencies
The API for listing all introspection statuses affects similar commands so would be good to wait until that is complete. https://review.openstack.org/#/c/344921/
Testing
In addition to functional testing, if baremetal CI is available, a test to ensure that LLDP collection is enabled and working would be useful, along with a test of the standard LLDP plugin as defined in the spec.