Minimum bandwidth support (egress)
Minimum bandwidth support (opposed to bandwidth limiting), guarantees a port minimum bandwidth when it's neighbours are consuming egress traffic and can be throttled in favor of the guaranteed port. Change-Id: Ia7a45f812f51d22469f5accf7efb3b515a656b64 Related-Bug: #1560963
This commit is contained in:
parent
4c81e54ad2
commit
c5f9d0b0d4
|
@ -0,0 +1,383 @@
|
||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License.
|
||||||
|
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
====================================
|
||||||
|
QoS minimum egress bandwidth support
|
||||||
|
====================================
|
||||||
|
|
||||||
|
RFE: https://bugs.launchpad.net/neutron/+bug/1560963
|
||||||
|
|
||||||
|
Problem Description
|
||||||
|
===================
|
||||||
|
|
||||||
|
The current `QoS API`_ does not provide functionality to define an assured
|
||||||
|
minimum egress bandwidth per port. This proposal talks about enhancing the
|
||||||
|
existing QoS API's by adding assured minimum egress bandwidth.
|
||||||
|
|
||||||
|
This functionality is currently supported in:
|
||||||
|
|
||||||
|
* Open vSwitch: minimum bandwidth assurance is supported, although the traffic
|
||||||
|
shaping applies only to egress traffic, from the switch point of view. This
|
||||||
|
egress traffic (from the switch point of view) should not be mistaken with
|
||||||
|
the egress traffic from the tenant, which is the goal of this new feature.
|
||||||
|
|
||||||
|
* Linux Bridge: using traffic control `HTB`_ implementation.
|
||||||
|
|
||||||
|
* SR-IOV: it will depends on the NIC driver and the `ip-link`_ version, which
|
||||||
|
needs to have "min_tx_rate" option; this option was merged in version 3.16.0.
|
||||||
|
|
||||||
|
Proposed Change
|
||||||
|
===============
|
||||||
|
The proposed change is an update to the QoS API and the drivers mentioned
|
||||||
|
(Open vSwitch, Linux Bridge and SR-IOV) to support minimum egress bandwidth
|
||||||
|
assurance. The value of the bandwidth will be stored as an integer value, with
|
||||||
|
valid values greater than zero. The unit is kbps, the same unit used in
|
||||||
|
``QosBandwidthLimitRule``.
|
||||||
|
|
||||||
|
The scope of this feature is to enable the API access to the egress minimum
|
||||||
|
bandwidth value and to enable this feature in the previous commented drivers.
|
||||||
|
This feature doesn't provide a method to inform about the backend capacity or
|
||||||
|
to check the bandwidth availability. These limitations will be documented in
|
||||||
|
user and developer guides.
|
||||||
|
|
||||||
|
Data Model Impact
|
||||||
|
-----------------
|
||||||
|
The model follows the way that the QoS Bandwidth Limiting functionality applies
|
||||||
|
to QoS policies by adding a QosMinimumBandwidthRule table.
|
||||||
|
|
||||||
|
The ``QosMinimumBandwidthRule`` model would look like::
|
||||||
|
|
||||||
|
+--------------+-------+----------+--------+-----------------+------------+
|
||||||
|
|Attribute |Type |Access |Default |Validation/ |Description |
|
||||||
|
|Name | | |Value |Conversion | |
|
||||||
|
+==============+=======+==========+========+=================+============+
|
||||||
|
|id |string |RO, all |generat_|uuid |identity |
|
||||||
|
| |(UUID) | |ed | | |
|
||||||
|
+--------------+-------+----------+--------+-----------------+------------+
|
||||||
|
|qos_policy_id |string |RO, all |N/A |uuid |QoSPolicy |
|
||||||
|
| |(UUID) | | | |reference |
|
||||||
|
+--------------+-------+----------+--------+-----------------+------------+
|
||||||
|
|min_kbps |integer|RW, tenant|N/A |not negative, |Minimum |
|
||||||
|
| | | | |<= max_kbps |bandwidth |
|
||||||
|
+--------------+-------+----------+--------+-----------------+------------+
|
||||||
|
|direction |enum |RW, tenant|'egress'|'egress' |Traffic |
|
||||||
|
| | | | | |direction |
|
||||||
|
+--------------+-------+----------+--------+-----------------+------------+
|
||||||
|
|
||||||
|
If "max_kbps" is None, the validation check for "min_kbps" will not apply.
|
||||||
|
|
||||||
|
This QoS rule can be used in combination with QoS bandwidth limit rule, defined
|
||||||
|
in `Neutron QoS API Models and Extension`_.
|
||||||
|
|
||||||
|
Both "qos_policy_id" and "direction" will be linked as a unique constraint set,
|
||||||
|
because the combination of both parameters must be unique.
|
||||||
|
|
||||||
|
This QoS rule will have a "direction" parameter to define the direction of the
|
||||||
|
traffic. This new feature will implement only egress traffic shaping. "ingress"
|
||||||
|
traffic direction won't be allowed for the nonce.
|
||||||
|
|
||||||
|
REST API Impact
|
||||||
|
---------------
|
||||||
|
Proposed attribute::
|
||||||
|
|
||||||
|
SUB_RESOURCE_ATTRIBUTE_MAP = {
|
||||||
|
'minimum_bandwidth_rules':{
|
||||||
|
'parent': {'collection_name': 'policies',
|
||||||
|
'member_name': 'policy'},
|
||||||
|
'parameters': dict(QOS_RULE_COMMON_FIELDS,
|
||||||
|
**{'min_kbps': {
|
||||||
|
'allow_post': True, 'allow_put': True,
|
||||||
|
'convert_to': attr.convert_to_int,
|
||||||
|
'is_visible': True, 'default': None,
|
||||||
|
'validate': {'type:non_negative': None,
|
||||||
|
'type:values': max_kbps}}
|
||||||
|
}),
|
||||||
|
**{'direction': {
|
||||||
|
'allow_post': True, 'allow_put': True,
|
||||||
|
'is_visible': True, 'default': 'egress',
|
||||||
|
'validate': {'type:values': common_constants.
|
||||||
|
EGRESS_DIRECTION}}
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
Sample REST calls::
|
||||||
|
|
||||||
|
GET /v2.0/qos/policies
|
||||||
|
|
||||||
|
Response:
|
||||||
|
{
|
||||||
|
"policy": {
|
||||||
|
"name": "TypeOne",
|
||||||
|
"description": "This policy sets a minimum bandwidth for type one users",
|
||||||
|
"shared": "False"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
GET /v2.0/qos/policies/<policy-uuid>
|
||||||
|
|
||||||
|
Response:
|
||||||
|
{
|
||||||
|
"policy": {
|
||||||
|
"tenant_id": "<tenant-id>",
|
||||||
|
"id": "<id>",
|
||||||
|
"name": "TypeOne",
|
||||||
|
"description": "This policy sets a minimum bandwidth for type one users",
|
||||||
|
"shared": False,
|
||||||
|
"rules": [{
|
||||||
|
"id": "<id>",
|
||||||
|
"policy_id": "<policy-uuid>",
|
||||||
|
"rule_type": neutron.services.qos.RULE_TYPE_MINIMUM_BANDWIDTH
|
||||||
|
"min_kbps": 10000,
|
||||||
|
"direction": "egress"
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
POST /v2.0/qos/policies/<policy-uuid>/minimum_bandwidth_rules/
|
||||||
|
{
|
||||||
|
"minimum_bandwidth_rule": {
|
||||||
|
"min_kbps": 20000,
|
||||||
|
"direction": "egress"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Response:
|
||||||
|
{
|
||||||
|
"minimum_bandwidth_rule":{
|
||||||
|
"id": "<id>",
|
||||||
|
"policy_id": "<policy-uuid>",
|
||||||
|
"min_kbps": 20000,
|
||||||
|
"direction": "egress"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
PUT /v2.0/qos/policies/<policy-uuid>/minimum_bandwidth_rules/<rule-uuid>
|
||||||
|
{
|
||||||
|
"minimum_bandwidth_rule": {
|
||||||
|
"min_kbps": 10000,
|
||||||
|
"direction": "egress"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Response:
|
||||||
|
{
|
||||||
|
"minimum_bandwidth_rule":{
|
||||||
|
"id": "<id>",
|
||||||
|
"policy_id": "<policy-uuid>",
|
||||||
|
"min_egress_kbps": 10000,
|
||||||
|
"direction": "egress"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Command Line Client Impact
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
* qos-minimum-bandwidth-rule-create <policy-id> --min-kbps <value> --direction <value>
|
||||||
|
* qos-minimum-bandwidth-rule-show <rule-id> <policy-id>
|
||||||
|
* qos-minimum-bandwidth-rule-list <policy-id>
|
||||||
|
* qos-minimum-bandwidth-rule-update <rule-id> <policy-id> --min-kbps <value>
|
||||||
|
--direction <value>
|
||||||
|
* qos-minimum-bandwidth-rule-delete <rule-id> <policy-id>
|
||||||
|
|
||||||
|
Security Impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Notifications Impact
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
IPv6 Impact
|
||||||
|
-----------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Other Deployer Impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
Deployers may need to configure the specific QoS driver / ML2 agent extension.
|
||||||
|
|
||||||
|
Developer Impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Community Impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
* Rodolfo Alonso Hernandez
|
||||||
|
* Miguel Angel Ajo
|
||||||
|
* Hirofumi Ichihara
|
||||||
|
* Moshe Levi
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
* Versioned DB objects for the new rule type
|
||||||
|
|
||||||
|
* API changes to allow for minimum egress bandwidth modifications
|
||||||
|
|
||||||
|
* Client changes to allow minimum egress bandwidth values being set
|
||||||
|
|
||||||
|
* QoS Openflow integration within the L2 agent extension for the Open vSwitch
|
||||||
|
driver to enable the minimum egress bandwidth support:
|
||||||
|
|
||||||
|
* Added/modified switch flows to mark the packet processing queue.
|
||||||
|
|
||||||
|
* Added/modified QoS rules and processing queues in the integration bridge,
|
||||||
|
physical bridges and tunnel bridge.
|
||||||
|
|
||||||
|
* QoS Traffic Control integration within the L2 agent extension for the Linux
|
||||||
|
Bridge driver to enable this feature.
|
||||||
|
|
||||||
|
* QoS ip-link integration within the L2 agent extension for the SR-IOV driver
|
||||||
|
to enable this feature. Add a sanity check to detect the version of ip-link
|
||||||
|
tool; minimum version required is 3.16.0.
|
||||||
|
|
||||||
|
Open vSwitch driver
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
The current Open vSwitch QoS implementation only shapes egress traffic.
|
||||||
|
|
||||||
|
Linux Bridge driver
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
The current Linux Bridge QoS implementation using Traffic Control (TC) only
|
||||||
|
shapes egress traffic (from the interface point of view). To shape ingress
|
||||||
|
traffic from the interface point of view (egress from the instance), what is
|
||||||
|
needed in this feature, the following steps are required:
|
||||||
|
|
||||||
|
* Create a `IFB`_.
|
||||||
|
|
||||||
|
* Send all ingress traffic to this IFB, mirroring it, using a TC rule. This
|
||||||
|
traffic will be sent to this IFB as egress traffic.
|
||||||
|
|
||||||
|
* Add a qdisc and a class in this IFB, with the QoS filter.
|
||||||
|
|
||||||
|
Also the algorithm used, `TBF`_, must be substituted by `HTB`_, a classful
|
||||||
|
algorithm that allows to set a "rate" parameter, defined as the "maximum rate
|
||||||
|
this class and all its children are guaranteed".
|
||||||
|
|
||||||
|
Future work
|
||||||
|
-----------
|
||||||
|
|
||||||
|
* Implement a method to report backend capacity to Neutron. Several methods to
|
||||||
|
enable this feature can be carried out:
|
||||||
|
|
||||||
|
* Use an agent to periodically read the capacity of the backends, using
|
||||||
|
config files, monitoring systems, etc.
|
||||||
|
The agent state reports could be used for that purpose. The information
|
||||||
|
could come from system inspection also enabling an option to override the
|
||||||
|
details in the agent config file.
|
||||||
|
The bandwidth mapping should be per physical network, and there's a
|
||||||
|
peculiarity we must consider: connection to physical networks can happen
|
||||||
|
by several unbound interfaces (like an SR-IOV card having several PFs
|
||||||
|
(with it's separate cables) to the switch. That would effectively give
|
||||||
|
different connection resources to the same net which are exhausted
|
||||||
|
separately.
|
||||||
|
This is the best option.
|
||||||
|
|
||||||
|
* Insert the backend capacity manually, using a static file during the
|
||||||
|
deployment process.
|
||||||
|
|
||||||
|
* Enable an API to manually insert the backend capacity. The API can be used
|
||||||
|
for testing and development purposes, but this option should be avoided.
|
||||||
|
|
||||||
|
* Inform Nova Scheduler about the total backend capacity and the QoS minimum
|
||||||
|
egress bandwidth rules, to improve the Scheduler decision. This feature is
|
||||||
|
being addressed by `[RFE] Strict minimum bandwidth support (egress)`_.
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
API-tests
|
||||||
|
---------
|
||||||
|
|
||||||
|
* Creating minimum egress bandwidth values
|
||||||
|
* Updating minimum egress bandwidth values
|
||||||
|
* Deleting minimum egress bandwidth values
|
||||||
|
* Listing minimum egress bandwidth values
|
||||||
|
* Showing a minimum egress bandwidth value
|
||||||
|
|
||||||
|
Functional Tests
|
||||||
|
----------------
|
||||||
|
|
||||||
|
Functional tests will be used to verify system interactions:
|
||||||
|
|
||||||
|
* Setting minimun egress bandwidth values
|
||||||
|
* Updating minimun egress bandwidth values
|
||||||
|
* Deleting minimun egress bandwidth values
|
||||||
|
* Listing minimun egress bandwidth values
|
||||||
|
|
||||||
|
Fullstack Tests
|
||||||
|
---------------
|
||||||
|
|
||||||
|
* Setting a QoS policy for minimum egress bandwidth on the port of the first
|
||||||
|
instance from the API, making a query to the backend and check the
|
||||||
|
correctness of the stored values.
|
||||||
|
* Updating QoS policy and checking again the values.
|
||||||
|
* Deleting QoS policy and verifying all the values related to the rule are
|
||||||
|
deleted.
|
||||||
|
|
||||||
|
These are no benchmark tests.
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
User Documentation
|
||||||
|
------------------
|
||||||
|
|
||||||
|
Existing `Networking Guide`_ will be updated for this feature.
|
||||||
|
|
||||||
|
Existing `CLI guide`_ will be updated for this feature.
|
||||||
|
|
||||||
|
Developer Documentation
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
Existing `QoS devref document`_ will be updated for this feature.
|
||||||
|
|
||||||
|
|
||||||
|
API Documentation
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
Existing `QoS API documentation`_ will be updated for this feature.
|
||||||
|
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
.. target-notes::
|
||||||
|
|
||||||
|
.. _`QoS API`: https://review.openstack.org/#/c/88599/
|
||||||
|
.. _`HTB`: http://linux.die.net/man/8/tc-htb
|
||||||
|
.. _`ip-link`: http://manpages.ubuntu.com/manpages/xenial/en/man8/ip-link.8.html
|
||||||
|
.. _`Neutron QoS API Models and Extension`: http://specs.openstack.org/openstack/neutron-specs/specs/liberty/qos-api-extension.html
|
||||||
|
.. _`IFB`: http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb
|
||||||
|
.. _`TBF`: http://linux.die.net/man/8/tc-tbf
|
||||||
|
.. _`[RFE] Strict minimum bandwidth support (egress)`: https://bugs.launchpad.net/neutron/+bug/1578989
|
||||||
|
.. _`Networking Guide`: https://github.com/openstack/openstack-manuals/blob/master/doc/networking-guide/source/adv-config-qos.rst
|
||||||
|
.. _`CLI guide`: https://github.com/openstack/openstack-manuals/blob/master/doc/cli-reference/source/neutron.rst
|
||||||
|
.. _`QoS devref document`: https://github.com/openstack/neutron/blob/master/doc/source/devref/quality_of_service.rst
|
||||||
|
.. _`QoS API documentation`: https://review.openstack.org/#/c/226834/
|
Loading…
Reference in New Issue