Browse Source

Minimum bandwidth support (egress)

Minimum bandwidth support (opposed to bandwidth limiting), guarantees
a port minimum bandwidth when it's neighbours are consuming egress
traffic and can be throttled in favor of the guaranteed port.

Change-Id: Ia7a45f812f51d22469f5accf7efb3b515a656b64
Related-Bug: #1560963
Rodolfo Alonso Hernandez 3 years ago
1 changed files with 383 additions and 0 deletions
  1. +383

+ 383
- 0
specs/newton/ml2-qos-minimum-egress-bw-support.rst View File

@@ -0,0 +1,383 @@
This work is licensed under a Creative Commons Attribution 3.0 Unported

QoS minimum egress bandwidth support


Problem Description

The current `QoS API`_ does not provide functionality to define an assured
minimum egress bandwidth per port. This proposal talks about enhancing the
existing QoS API's by adding assured minimum egress bandwidth.

This functionality is currently supported in:

* Open vSwitch: minimum bandwidth assurance is supported, although the traffic
shaping applies only to egress traffic, from the switch point of view. This
egress traffic (from the switch point of view) should not be mistaken with
the egress traffic from the tenant, which is the goal of this new feature.

* Linux Bridge: using traffic control `HTB`_ implementation.

* SR-IOV: it will depends on the NIC driver and the `ip-link`_ version, which
needs to have "min_tx_rate" option; this option was merged in version 3.16.0.

Proposed Change
The proposed change is an update to the QoS API and the drivers mentioned
(Open vSwitch, Linux Bridge and SR-IOV) to support minimum egress bandwidth
assurance. The value of the bandwidth will be stored as an integer value, with
valid values greater than zero. The unit is kbps, the same unit used in

The scope of this feature is to enable the API access to the egress minimum
bandwidth value and to enable this feature in the previous commented drivers.
This feature doesn't provide a method to inform about the backend capacity or
to check the bandwidth availability. These limitations will be documented in
user and developer guides.

Data Model Impact
The model follows the way that the QoS Bandwidth Limiting functionality applies
to QoS policies by adding a QosMinimumBandwidthRule table.

The ``QosMinimumBandwidthRule`` model would look like::

|Attribute |Type |Access |Default |Validation/ |Description |
|Name | | |Value |Conversion | |
|id |string |RO, all |generat_|uuid |identity |
| |(UUID) | |ed | | |
|qos_policy_id |string |RO, all |N/A |uuid |QoSPolicy |
| |(UUID) | | | |reference |
|min_kbps |integer|RW, tenant|N/A |not negative, |Minimum |
| | | | |<= max_kbps |bandwidth |
|direction |enum |RW, tenant|'egress'|'egress' |Traffic |
| | | | | |direction |

If "max_kbps" is None, the validation check for "min_kbps" will not apply.

This QoS rule can be used in combination with QoS bandwidth limit rule, defined
in `Neutron QoS API Models and Extension`_.

Both "qos_policy_id" and "direction" will be linked as a unique constraint set,
because the combination of both parameters must be unique.

This QoS rule will have a "direction" parameter to define the direction of the
traffic. This new feature will implement only egress traffic shaping. "ingress"
traffic direction won't be allowed for the nonce.

Proposed attribute::

'parent': {'collection_name': 'policies',
'member_name': 'policy'},
'parameters': dict(QOS_RULE_COMMON_FIELDS,
**{'min_kbps': {
'allow_post': True, 'allow_put': True,
'convert_to': attr.convert_to_int,
'is_visible': True, 'default': None,
'validate': {'type:non_negative': None,
'type:values': max_kbps}}
**{'direction': {
'allow_post': True, 'allow_put': True,
'is_visible': True, 'default': 'egress',
'validate': {'type:values': common_constants.

Sample REST calls::

GET /v2.0/qos/policies

"policy": {
"name": "TypeOne",
"description": "This policy sets a minimum bandwidth for type one users",
"shared": "False"

GET /v2.0/qos/policies/<policy-uuid>

"policy": {
"tenant_id": "<tenant-id>",
"id": "<id>",
"name": "TypeOne",
"description": "This policy sets a minimum bandwidth for type one users",
"shared": False,
"rules": [{
"id": "<id>",
"policy_id": "<policy-uuid>",
"min_kbps": 10000,
"direction": "egress"

POST /v2.0/qos/policies/<policy-uuid>/minimum_bandwidth_rules/
"minimum_bandwidth_rule": {
"min_kbps": 20000,
"direction": "egress"

"id": "<id>",
"policy_id": "<policy-uuid>",
"min_kbps": 20000,
"direction": "egress"

PUT /v2.0/qos/policies/<policy-uuid>/minimum_bandwidth_rules/<rule-uuid>
"minimum_bandwidth_rule": {
"min_kbps": 10000,
"direction": "egress"

"id": "<id>",
"policy_id": "<policy-uuid>",
"min_egress_kbps": 10000,
"direction": "egress"

Command Line Client Impact

* qos-minimum-bandwidth-rule-create <policy-id> --min-kbps <value> --direction <value>
* qos-minimum-bandwidth-rule-show <rule-id> <policy-id>
* qos-minimum-bandwidth-rule-list <policy-id>
* qos-minimum-bandwidth-rule-update <rule-id> <policy-id> --min-kbps <value>
--direction <value>
* qos-minimum-bandwidth-rule-delete <rule-id> <policy-id>

Security Impact


Notifications Impact


Performance Impact


IPv6 Impact


Other Deployer Impact

Deployers may need to configure the specific QoS driver / ML2 agent extension.

Developer Impact


Community Impact




* Rodolfo Alonso Hernandez
* Miguel Angel Ajo
* Hirofumi Ichihara
* Moshe Levi

Work Items

* Versioned DB objects for the new rule type

* API changes to allow for minimum egress bandwidth modifications

* Client changes to allow minimum egress bandwidth values being set

* QoS Openflow integration within the L2 agent extension for the Open vSwitch
driver to enable the minimum egress bandwidth support:

* Added/modified switch flows to mark the packet processing queue.

* Added/modified QoS rules and processing queues in the integration bridge,
physical bridges and tunnel bridge.

* QoS Traffic Control integration within the L2 agent extension for the Linux
Bridge driver to enable this feature.

* QoS ip-link integration within the L2 agent extension for the SR-IOV driver
to enable this feature. Add a sanity check to detect the version of ip-link
tool; minimum version required is 3.16.0.

Open vSwitch driver
The current Open vSwitch QoS implementation only shapes egress traffic.

Linux Bridge driver
The current Linux Bridge QoS implementation using Traffic Control (TC) only
shapes egress traffic (from the interface point of view). To shape ingress
traffic from the interface point of view (egress from the instance), what is
needed in this feature, the following steps are required:

* Create a `IFB`_.

* Send all ingress traffic to this IFB, mirroring it, using a TC rule. This
traffic will be sent to this IFB as egress traffic.

* Add a qdisc and a class in this IFB, with the QoS filter.

Also the algorithm used, `TBF`_, must be substituted by `HTB`_, a classful
algorithm that allows to set a "rate" parameter, defined as the "maximum rate
this class and all its children are guaranteed".

Future work

* Implement a method to report backend capacity to Neutron. Several methods to
enable this feature can be carried out:

* Use an agent to periodically read the capacity of the backends, using
config files, monitoring systems, etc.
The agent state reports could be used for that purpose. The information
could come from system inspection also enabling an option to override the
details in the agent config file.
The bandwidth mapping should be per physical network, and there's a
peculiarity we must consider: connection to physical networks can happen
by several unbound interfaces (like an SR-IOV card having several PFs
(with it's separate cables) to the switch. That would effectively give
different connection resources to the same net which are exhausted
This is the best option.

* Insert the backend capacity manually, using a static file during the
deployment process.

* Enable an API to manually insert the backend capacity. The API can be used
for testing and development purposes, but this option should be avoided.

* Inform Nova Scheduler about the total backend capacity and the QoS minimum
egress bandwidth rules, to improve the Scheduler decision. This feature is
being addressed by `[RFE] Strict minimum bandwidth support (egress)`_.





* Creating minimum egress bandwidth values
* Updating minimum egress bandwidth values
* Deleting minimum egress bandwidth values
* Listing minimum egress bandwidth values
* Showing a minimum egress bandwidth value

Functional Tests

Functional tests will be used to verify system interactions:

* Setting minimun egress bandwidth values
* Updating minimun egress bandwidth values
* Deleting minimun egress bandwidth values
* Listing minimun egress bandwidth values

Fullstack Tests

* Setting a QoS policy for minimum egress bandwidth on the port of the first
instance from the API, making a query to the backend and check the
correctness of the stored values.
* Updating QoS policy and checking again the values.
* Deleting QoS policy and verifying all the values related to the rule are

These are no benchmark tests.

Documentation Impact

User Documentation

Existing `Networking Guide`_ will be updated for this feature.

Existing `CLI guide`_ will be updated for this feature.

Developer Documentation

Existing `QoS devref document`_ will be updated for this feature.

API Documentation

Existing `QoS API documentation`_ will be updated for this feature.

.. target-notes::

.. _`QoS API`:
.. _`HTB`:
.. _`ip-link`:
.. _`Neutron QoS API Models and Extension`:
.. _`IFB`:
.. _`TBF`:
.. _`[RFE] Strict minimum bandwidth support (egress)`:
.. _`Networking Guide`:
.. _`CLI guide`:
.. _`QoS devref document`:
.. _`QoS API documentation`: