Re-add D001 test and fix all documents accordingly
Re-added the D001 test (line too long) and fixed all documents that had long lines. Change-Id: I2cb3c0807c1668cb48022d380ebbe95d12215210
This commit is contained in:
parent
db0213f3c1
commit
a85d2ec86b
|
@ -24,7 +24,8 @@ Models
|
|||
|
||||
Dragonflow, as many other projects interfacing with a database, uses model
|
||||
layer to allow uniform and easy access to data stored in the (north-bound)
|
||||
database. The current model framework is a fruit of the :doc:`../specs/nb_api_refactor`
|
||||
database. The current model framework is a fruit of the
|
||||
:doc:`../specs/nb_api_refactor`
|
||||
|
||||
Creating new models
|
||||
-------------------
|
||||
|
@ -45,14 +46,16 @@ with `construct_nb_db_model` decorator. Below we'll introduce an example:
|
|||
|
||||
The above example defines a new `Movie` model, that contains 5 fields:
|
||||
|
||||
#. `id` - Object identifier, derived form ModelBase, present in all model objects.
|
||||
#. `id` - Object identifier, derived form ModelBase, present in all model
|
||||
objects.
|
||||
#. `title` - A string containing the movie title, marked as mandatory.
|
||||
#. `year` - A year movie was published.
|
||||
#. `director` - A reference field to an object of director type (will be covered later).
|
||||
#. `director` - A reference field to an object of director type (will be
|
||||
covered later).
|
||||
#. `awards` - A list of all the awards the movie received.
|
||||
|
||||
Class definition also contains `table_name` field that stores the name of the table
|
||||
our model is stored in the north-bound database.
|
||||
Class definition also contains `table_name` field that stores the name of the
|
||||
table our model is stored in the north-bound database.
|
||||
|
||||
|
||||
Initializing this object is done by passing the values as keyword arguments.
|
||||
|
@ -157,7 +160,8 @@ For each event, 2 class methods are defined:
|
|||
|
||||
* `register_{event_name}(callback)` - adds callback to be invoked each time
|
||||
event is emitted.
|
||||
* `unregister_{event_name}(callback)` - removes the callback from being called.
|
||||
* `unregister_{event_name}(callback)` - removes the callback from being
|
||||
called.
|
||||
|
||||
Additionally, an instance method named `emit_{event_name}(*args, **kwargs)` is
|
||||
added.
|
||||
|
@ -178,7 +182,8 @@ would be translated to a sequence of
|
|||
|
||||
The convention of parameters is specific to each event.
|
||||
|
||||
The register calls can be also used as decorators for some extra syntactic sugar
|
||||
The register calls can be also used as decorators for some extra syntactic
|
||||
sugar
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
@ -191,11 +196,11 @@ Indexes
|
|||
|
||||
To allow easy retrieval and lookup of in memory objects we use DbStore module
|
||||
to fetch by IDs and other properties, the new DbStore takes note of model's
|
||||
indexes and creates lookups to allow faster retrieval. Indexes, similar to events
|
||||
are passed in `indexes=` parameter of construct_nb_db_model decorator and
|
||||
specified as a dictionary where the key is the index name and the value is the
|
||||
field indexed by (or a tuple of fields, if the index is multi-key). For example
|
||||
if we'd like to add index by year we can define it as:
|
||||
indexes and creates lookups to allow faster retrieval. Indexes, similar to
|
||||
events are passed in `indexes=` parameter of construct_nb_db_model decorator
|
||||
and specified as a dictionary where the key is the index name and the value is
|
||||
the field indexed by (or a tuple of fields, if the index is multi-key). For
|
||||
example if we'd like to add index by year we can define it as:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
|
|
@ -38,9 +38,9 @@ The controller set the flow metadata in the redirection rules
|
|||
to the local port unique key as a hint to allow fast port info lookup
|
||||
for the reactive DHCP packets handle by the DHCP application.
|
||||
|
||||
The local DHCP application handle the redirected DHCP packets and answer as a DHCP
|
||||
server. DHCP traffic is handled directly at the compute node and never goes on
|
||||
the network.
|
||||
The local DHCP application handle the redirected DHCP packets and answer as a
|
||||
DHCP server. DHCP traffic is handled directly at the compute node and never
|
||||
goes on the network.
|
||||
|
||||
The following diagrams demonstrate this process:
|
||||
|
||||
|
|
|
@ -59,7 +59,8 @@ Dragonflow applications that communicate with the local OpenVSwitch.
|
|||
The DB is being populated by Dragonflow Neutron plugin that converts neutron
|
||||
API to our model.
|
||||
|
||||
The following sections each describe a specific topic/functionality in Dragonflow
|
||||
The following sections each describe a specific topic/functionality in
|
||||
Dragonflow
|
||||
|
||||
Dragonflow Supported Features
|
||||
=============================
|
||||
|
|
|
@ -16,8 +16,8 @@ Running the image
|
|||
|
||||
Preparation work
|
||||
~~~~~~~~~~~~~~~~
|
||||
* Create a network to be used by the containers, use any subnet you find fit, the subnet here is
|
||||
just an example.
|
||||
* Create a network to be used by the containers, use any subnet you find fit,
|
||||
the subnet here is just an example.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
@ -68,12 +68,13 @@ to allow this via `selinux`.
|
|||
|
||||
docker inspect --format "{{ .NetworkSettings.Networks.${DRAGONFLOW_NET_NAME}.IPAddress }}" dragonflow
|
||||
|
||||
There are two configuration files that Dragonflow needs, and creates automatically if they do not
|
||||
exist:
|
||||
There are two configuration files that Dragonflow needs, and creates
|
||||
automatically if they do not exist:
|
||||
|
||||
* `/etc/dragonflow/dragonflow.ini`
|
||||
|
||||
* `/etc/dragonflow//etc/dragonflow/dragonflow_datapath_layout.yaml`
|
||||
|
||||
If these files exist, they are used as-is, and are not overwritten. You can add these files using
|
||||
e.g. `-v local-dragonflow-conf.ini:/etc/dragonflow/dragonflow.ini`.
|
||||
If these files exist, they are used as-is, and are not overwritten. You can add
|
||||
these files using e.g.
|
||||
`-v local-dragonflow-conf.ini:/etc/dragonflow/dragonflow.ini`.
|
||||
|
|
|
@ -16,37 +16,42 @@
|
|||
Guru Meditation Reports
|
||||
=======================
|
||||
|
||||
Dragonflow contains a mechanism whereby developers and system administrators can generate a report about
|
||||
the state of a running Dragonflow executable.
|
||||
Dragonflow contains a mechanism whereby developers and system administrators
|
||||
can generate a report about the state of a running Dragonflow executable.
|
||||
This report is called a *Guru Meditation Report* (*GMR* for short).
|
||||
|
||||
Generating a GMR
|
||||
----------------
|
||||
|
||||
A *GMR* can be generated by sending the *USR2* signal to any Dragonflow process with support (see below).
|
||||
A *GMR* can be generated by sending the *USR2* signal to any Dragonflow process
|
||||
with support (see below).
|
||||
The *GMR* will then be outputted standard error for that particular process.
|
||||
|
||||
For example, suppose that ``df-local-controller`` has process id ``2525``, and was run with
|
||||
``2>/var/log/dragonflow/df-controller.log``. Then, ``kill -USR2 2525`` will trigger the Guru Meditation
|
||||
report to be printed to ``/var/log/dragonflow/df-controller.log``.
|
||||
For example, suppose that ``df-local-controller`` has process id ``2525``, and
|
||||
was run with ``2>/var/log/dragonflow/df-controller.log``. Then,
|
||||
``kill -USR2 2525`` will trigger the Guru Meditation report to be printed to
|
||||
``/var/log/dragonflow/df-controller.log``.
|
||||
|
||||
Structure of a GMR
|
||||
------------------
|
||||
|
||||
The *GMR* is designed to be extensible; any particular executable may add its own sections. However,
|
||||
the base *GMR* consists of several sections:
|
||||
The *GMR* is designed to be extensible; any particular executable may add its
|
||||
own sections. However, the base *GMR* consists of several sections:
|
||||
|
||||
Package
|
||||
Shows information about the package to which this process belongs, including version information
|
||||
Shows information about the package to which this process belongs, including
|
||||
version information
|
||||
|
||||
Threads
|
||||
Shows stack traces and thread ids for each of the threads within this process
|
||||
|
||||
Green Threads
|
||||
Shows stack traces for each of the green threads within this process (green threads don't have thread ids)
|
||||
Shows stack traces for each of the green threads within this process (green
|
||||
threads don't have thread ids)
|
||||
|
||||
Configuration
|
||||
Lists all the configuration options currently accessible via the CONF object for the current process
|
||||
Lists all the configuration options currently accessible via the CONF object
|
||||
for the current process
|
||||
|
||||
Adding Support for GMRs to New Executables
|
||||
------------------------------------------
|
||||
|
@ -67,7 +72,8 @@ Then, register any additional sections (optional):
|
|||
TextGuruMeditation.register_section('Some Special Section',
|
||||
some_section_generator)
|
||||
|
||||
Finally (under main), before running the "main loop" of the executable, register the *GMR* hook:
|
||||
Finally (under main), before running the "main loop" of the executable,
|
||||
register the *GMR* hook:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
|
@ -76,5 +82,6 @@ Finally (under main), before running the "main loop" of the executable, register
|
|||
Extending the GMR
|
||||
-----------------
|
||||
|
||||
As mentioned above, additional sections can be added to the GMR for a particular executable.
|
||||
As mentioned above, additional sections can be added to the GMR for a
|
||||
particular executable.
|
||||
For more information, see the inline documentation under :mod:`oslo.reports`
|
||||
|
|
|
@ -21,13 +21,13 @@ Copy one of the following as your local.conf to your devstack folder
|
|||
Automated setup using Vagrant
|
||||
=============================
|
||||
|
||||
This will create a 3 node devstack (controller + two computes), where Dragonflow is used as
|
||||
the Open vSwitch backend.
|
||||
This will create a 3 node devstack (controller + two computes), where
|
||||
Dragonflow is used as the Open vSwitch backend.
|
||||
|
||||
Vagrant allows to configure the provider on which the virtual machines are
|
||||
created. Virtualbox is the default provider used to launch the VM's on a
|
||||
developer computer, but other providers can be used: libvirt, VMWare, AWS, OpenStack,
|
||||
containers stuff, ...
|
||||
developer computer, but other providers can be used: libvirt, VMWare, AWS,
|
||||
OpenStack, containers stuff, ...
|
||||
|
||||
Quick Start
|
||||
-----------
|
||||
|
@ -49,7 +49,8 @@ Quick Start
|
|||
vagrant plugin install vagrant-cachier
|
||||
vagrant plugin install vagrant-vbguest
|
||||
|
||||
4. | For full install with a controller node and 2 compute nodes follow step 4.1;
|
||||
4. | For full install with a controller node and 2 compute nodes follow step
|
||||
4.1;
|
||||
| For a minimal install with All-In-One setup, follow step 4.2
|
||||
|
||||
4.1. Adjust the settings in `vagrant/provisioning/dragonflow.conf.yml` if
|
||||
|
|
|
@ -21,9 +21,11 @@ Dragonflow mainly has several components:
|
|||
#. Dragonflow neutron plugins (set up in neutron-server configuration)
|
||||
#. Dragonflow local controller running on each compute node
|
||||
#. Dragonflow metadata service running on each compute node
|
||||
#. Dragonflow publisher service running aside neutron server (if zeromq pub/sub driver is enabled)
|
||||
#. Dragonflow publisher service running aside neutron server (if zeromq pub/sub
|
||||
driver is enabled)
|
||||
#. Dragonflow l3 agent running on each network node
|
||||
#. Dragonflow northbound database (depends on which database you set up in dragonflow configuration)
|
||||
#. Dragonflow northbound database (depends on which database you set up in
|
||||
dragonflow configuration)
|
||||
|
||||
Source Code
|
||||
-----------
|
||||
|
@ -114,8 +116,8 @@ Basic Configurations
|
|||
Northbound Database
|
||||
-------------------
|
||||
|
||||
Dragonflow supports etcd, redis, zookeeper and ramcloud. You need to deploy one of them
|
||||
in your environment and expose the necessary TCP port.
|
||||
Dragonflow supports etcd, redis, zookeeper and ramcloud. You need to deploy one
|
||||
of them in your environment and expose the necessary TCP port.
|
||||
|
||||
Next you need to change the configuration, for example, etcd:
|
||||
|
||||
|
@ -130,7 +132,8 @@ Next you need to change the configuration, for example, etcd:
|
|||
Pub/Sub Driver
|
||||
--------------
|
||||
|
||||
Dragonflow supports etcd, redis and zeromq. You need to change the configuration, for example, etcd:
|
||||
Dragonflow supports etcd, redis and zeromq. You need to change the
|
||||
configuration, for example, etcd:
|
||||
|
||||
/etc/neutron/dragonflow.ini:
|
||||
|
||||
|
|
|
@ -6,23 +6,26 @@ Instead of implementing a proprietary DB solution for Dragonflow or picking
|
|||
one open source framework over the other, we designed the DB layer in
|
||||
Dragonflow to be pluggable.
|
||||
|
||||
The DB framework is the mechanism to sync network policy and topology between the CMS and the
|
||||
local controllers and hence control the performance, latency and scale of the environments
|
||||
Dragonflow is deployed in.
|
||||
The DB framework is the mechanism to sync network policy and topology between
|
||||
the CMS and the local controllers and hence control the performance, latency
|
||||
and scale of the environments Dragonflow is deployed in.
|
||||
|
||||
This allows the operator/admin the flexibility of choosing and changing between DB
|
||||
solutions to best fit his/her setup.
|
||||
It also allows, with very minimal integration, a way to leverage the well tested and mature
|
||||
feature set of these DB frameworks (clustering, HA, security, consistency, low latency and more..)
|
||||
This allows the operator/admin the flexibility of choosing and changing between
|
||||
DB solutions to best fit his/her setup.
|
||||
It also allows, with very minimal integration, a way to leverage the well
|
||||
tested and mature feature set of these DB frameworks (clustering, HA, security,
|
||||
consistency, low latency and more..)
|
||||
|
||||
This also allows the operator/admin to pick the correct balance between performance and
|
||||
latency requirements of their setup and the resource overhead of the DB framework.
|
||||
This also allows the operator/admin to pick the correct balance between
|
||||
performance and latency requirements of their setup and the resource overhead
|
||||
of the DB framework.
|
||||
|
||||
Adding support for another DB framework is an easy process, all you need is to implement
|
||||
the DB driver API class and add an installation script for the DB framework server and client.
|
||||
Adding support for another DB framework is an easy process, all you need is to
|
||||
implement the DB driver API class and add an installation script for the DB
|
||||
framework server and client.
|
||||
|
||||
The following diagram depicts the pluggable DB architecture in Dragonflow and the
|
||||
currently supported DB frameworks:
|
||||
The following diagram depicts the pluggable DB architecture in Dragonflow and
|
||||
the currently supported DB frameworks:
|
||||
|
||||
.. image:: ../images/db1.jpg
|
||||
:alt: Pluggable DB architecture
|
||||
|
@ -43,62 +46,65 @@ to the DB driver which is generic.
|
|||
|
||||
This class should be used by all Dragonflow users that need to interact
|
||||
with the DB (write/read).
|
||||
For example: Dragonflow Neutron plugin, the Dragonflow local controller, external applications.
|
||||
For example: Dragonflow Neutron plugin, the Dragonflow local controller,
|
||||
external applications.
|
||||
|
||||
This component was added for one main reason:
|
||||
We didn't want to expose the DB driver to the internal data schema/model of
|
||||
Dragonflow.
|
||||
We didn't want that every new feature in Dragonflow will trigger changes in the various
|
||||
different DB drivers.
|
||||
We didn't want that every new feature in Dragonflow will trigger changes in the
|
||||
various different DB drivers.
|
||||
|
||||
This component has an interface to add/set/delete elements in our model (like logical
|
||||
switches, logical routers and so on) and translate these APIs to a simple, generic
|
||||
key/value operations that are done by the DB driver.
|
||||
This component has an interface to add/set/delete elements in our model (like
|
||||
logical switches, logical routers and so on) and translate these APIs to a
|
||||
simple, generic key/value operations that are done by the DB driver.
|
||||
|
||||
This component also define the Dragonflow data model objects and which fields each
|
||||
one of the logical elements has.
|
||||
This component also define the Dragonflow data model objects and which fields
|
||||
each one of the logical elements has.
|
||||
|
||||
The N/B DB Adapter has a reference to a DB Driver instance which is used to interact
|
||||
with the DB framework.
|
||||
We have identified that different DB frameworks might have different features and
|
||||
capabilities, this layer is in charge of understanding the features exposed by the driver
|
||||
and using them if possible.
|
||||
The N/B DB Adapter has a reference to a DB Driver instance which is used to
|
||||
interact with the DB framework.
|
||||
We have identified that different DB frameworks might have different features
|
||||
and capabilities, this layer is in charge of understanding the features exposed
|
||||
by the driver and using them if possible.
|
||||
|
||||
|
||||
DB Driver API
|
||||
-------------
|
||||
DB Driver is an interface class that list the methods needed to be implemented
|
||||
in order to connect a certain DB framework to work with Dragonflow as a backend.
|
||||
in order to connect a certain DB framework to work with Dragonflow as a
|
||||
backend.
|
||||
|
||||
The DB driver is a very minimalistic interface that uses a simple key/value
|
||||
approach and can fit to almost all DB frameworks.
|
||||
|
||||
In order for Dragonflow to be able to leverage "advance" features of the DB,
|
||||
the driver has a way to indicate if a specific feature is implemented or not, and if
|
||||
it is, provide an API to consume it.
|
||||
the driver has a way to indicate if a specific feature is implemented or not,
|
||||
and if it is, provide an API to consume it.
|
||||
|
||||
Using this method, the applicative DB adapter can choose the best way to manage
|
||||
the way it interact with the DB.
|
||||
|
||||
For example: the driver can state if it support publish-subscribe on its tables,
|
||||
If it does, the local controller will register a callback method to the driver to
|
||||
receive any DB notifications and instead of polling the DB for changes, wait for the
|
||||
driver to send them.
|
||||
For example: the driver can state if it support publish-subscribe on its
|
||||
tables, If it does, the local controller will register a callback method to the
|
||||
driver to receive any DB notifications and instead of polling the DB for
|
||||
changes, wait for the driver to send them.
|
||||
|
||||
If the driver doesn't support publish-subscribe, the controller will keep polling the
|
||||
DB framework looking for changes.
|
||||
If the driver doesn't support publish-subscribe, the controller will keep
|
||||
polling the DB framework looking for changes.
|
||||
|
||||
|
||||
Modes of DB
|
||||
===========
|
||||
There are three different modes for the interaction between Dragonflow and the DB.
|
||||
There are three different modes for the interaction between Dragonflow and the
|
||||
DB.
|
||||
|
||||
Full Proactive
|
||||
--------------
|
||||
In this mode, all the DB data (policy and topology) is synced with all the local
|
||||
Dragonflow controllers (each compute node).
|
||||
Dragonflow saves in a local in-memory cache all the data that was synced from the
|
||||
DB in order to do fast lookups.
|
||||
In this mode, all the DB data (policy and topology) is synced with all the
|
||||
local Dragonflow controllers (each compute node).
|
||||
Dragonflow saves in a local in-memory cache all the data that was synced from
|
||||
the DB in order to do fast lookups.
|
||||
|
||||
Selective Proactive
|
||||
-------------------
|
||||
|
@ -108,10 +114,10 @@ Selective Proactive
|
|||
:height: 525
|
||||
:align: center
|
||||
|
||||
We have identified that in virtualized environments today with tenant isolation, full
|
||||
proactive mode is not really needed.
|
||||
We only need to synchronize each compute node (local-controller) with the relevant
|
||||
data depending on the local ports of this compute node.
|
||||
We have identified that in virtualized environments today with tenant
|
||||
isolation, full proactive mode is not really needed.
|
||||
We only need to synchronize each compute node (local-controller) with the
|
||||
relevant data depending on the local ports of this compute node.
|
||||
This mode is called selective proactive.
|
||||
|
||||
The following diagram depicts why this is needed:
|
||||
|
@ -122,15 +128,15 @@ The following diagram depicts why this is needed:
|
|||
:height: 525
|
||||
:align: center
|
||||
|
||||
We can see from the diagram that each compute node has VMs from one network, and in the
|
||||
topology we can see that the networks are isolated, meaning VMs from one network can not
|
||||
communicate with VMs from another.
|
||||
We can see from the diagram that each compute node has VMs from one network,
|
||||
and in the topology we can see that the networks are isolated, meaning VMs
|
||||
from one network can not communicate with VMs from another.
|
||||
|
||||
It is obvious than that each compute node only needs to get the topology and policy
|
||||
of the network and VMs that are local.
|
||||
(If there was a router connecting between these two networks, this statement was no
|
||||
longer correct, but we kept it simple in order to demonstrate that in setups today there
|
||||
are many isolated topologies)
|
||||
It is obvious than that each compute node only needs to get the topology and
|
||||
policy of the network and VMs that are local.
|
||||
(If there was a router connecting between these two networks, this statement
|
||||
was no longer correct, but we kept it simple in order to demonstrate that in
|
||||
setups today there are many isolated topologies)
|
||||
|
||||
Reactive
|
||||
--------
|
||||
|
|
|
@ -51,8 +51,9 @@ understand the API.
|
|||
__ _PUB_SUB_API
|
||||
|
||||
For both network and IPC based communication, a driver has to implement
|
||||
``dragonflow.db.pub_sub_api.PubSubApi`` (`Link`__). In both cases, ``get_publisher`` and
|
||||
``get_subscriber`` return a ``dragonflow.db.pub_sub_api.PublisherApi`` and a
|
||||
``dragonflow.db.pub_sub_api.PubSubApi`` (`Link`__). In both cases,
|
||||
``get_publisher`` and ``get_subscriber`` return a
|
||||
``dragonflow.db.pub_sub_api.PublisherApi`` and a
|
||||
``dragonflow.db.pub_sub_api.SubscriberApi``, respectively.
|
||||
|
||||
__ _PUB_SUB_API
|
||||
|
@ -116,9 +117,9 @@ the network implementation connects over the transport protocol provided via
|
|||
*publisher_transport*.
|
||||
|
||||
In the case of the publisher, the difference is both in the implementation of
|
||||
``initialize``, ``_connect``, and ``send_event``. The difference in connect is for
|
||||
the same reasons as the subscribers. The difference in ``initialize`` is since
|
||||
the multi-proc subscriber uses the lazy initialization pattern. This also
|
||||
``initialize``, ``_connect``, and ``send_event``. The difference in connect is
|
||||
for the same reasons as the subscribers. The difference in ``initialize`` is
|
||||
since the multi-proc subscriber uses the lazy initialization pattern. This also
|
||||
accounts for the difference in ``send_event``.
|
||||
|
||||
==========
|
||||
|
|
|
@ -19,7 +19,9 @@ Quick Installation
|
|||
|
||||
`DevStack Multi Node Configuration <https://github.com/openstack/dragonflow/tree/master/doc/source/multi-node-conf>`_
|
||||
|
||||
3) Edit local.conf according to your configuration, See `Detailed Installation`_ for more details, or the Devstack_ configuration manual
|
||||
3) Edit local.conf according to your configuration,
|
||||
See `Detailed Installation`_ for more details,
|
||||
or the Devstack_ configuration manual
|
||||
|
||||
`Devstack <https://docs.openstack.org/devstack/latest/configuration.html>`_
|
||||
|
||||
|
@ -105,7 +107,8 @@ Important parameters that needs to be set in ``local.conf`` :
|
|||
RABBIT_HOST <- Management IP address of the controller node
|
||||
GLANCE_HOSTPORT <- Management IP address of the controller node (Leave the port as-is)
|
||||
|
||||
You can find example configuration files in the multi-node-conf or the single-node-conf directories.
|
||||
You can find example configuration files in the multi-node-conf or the
|
||||
single-node-conf directories.
|
||||
|
||||
|
||||
==========================================
|
||||
|
|
|
@ -2,7 +2,8 @@
|
|||
Contributors & Reviewers Guide
|
||||
==============================
|
||||
|
||||
In this document, we try to guide contributors to know what should be included in the patch.
|
||||
In this document, we try to guide contributors to know what should be included
|
||||
in the patch.
|
||||
This guide is also helpful for reviewers covering what to look for when
|
||||
accepting a patch for Dragonflow.
|
||||
|
||||
|
@ -35,8 +36,9 @@ The following items are expected for every patch:
|
|||
discretion whether the change is truly a Trivial Fix.
|
||||
|
||||
# Release Notes:
|
||||
For NB API changes, configuration changes, new drivers and new application relevant
|
||||
release note should be added. It is recommended to use reno, see TBD.
|
||||
For NB API changes, configuration changes, new drivers and new application
|
||||
relevant release note should be added. It is recommended to use reno, see
|
||||
TBD.
|
||||
|
||||
|
||||
Spec & DevRef
|
||||
|
@ -56,16 +58,16 @@ cover the southbound implementation, including the rationale. The general
|
|||
guideline should be - if a new contributor reads this document, they should
|
||||
be able to understand the code of the application.
|
||||
|
||||
The difference between a spec and a devref is difficult to formalize. In essence,
|
||||
the spec should give a high-level design, while the dev-ref should give a low-level
|
||||
design of the feature. The guiding thought is that the spec should remain unchange
|
||||
unless there is a massive feature overhaul, but the dev-ref may change due to
|
||||
bug fixes, since it covers the low-level specifics.
|
||||
The difference between a spec and a devref is difficult to formalize. In
|
||||
essence, the spec should give a high-level design, while the dev-ref should
|
||||
give a low-level design of the feature. The guiding thought is that the spec
|
||||
should remain unchange unless there is a massive feature overhaul, but the
|
||||
dev-ref may change due to bug fixes, since it covers the low-level specifics.
|
||||
|
||||
Note that when writing the dev-ref, that the code is also available. Rather than
|
||||
explain the code, try to explain what the code is supposed to do, what is the
|
||||
end result supposed to look like, and most importantly, why the code looks that
|
||||
way.
|
||||
Note that when writing the dev-ref, that the code is also available. Rather
|
||||
than explain the code, try to explain what the code is supposed to do, what is
|
||||
the end result supposed to look like, and most importantly, why the code looks
|
||||
that way.
|
||||
|
||||
Specs are usually reviewed and accepted before the implementation begins.
|
||||
Dev-refs are usually reviewed and accepted as part of the implementation or
|
||||
|
@ -76,12 +78,13 @@ Bugs & Blueprints
|
|||
|
||||
For any issue with existing implementation, a bug report is expected.
|
||||
|
||||
For any new feature request or existing feature enhancement bug report with [RFE] tag is expected.
|
||||
For any new feature request or existing feature enhancement bug report with
|
||||
[RFE] tag is expected.
|
||||
Blueprint creation is not required.
|
||||
|
||||
Bug report should have descriptive title and detailed description. It is not
|
||||
a trivial task to submit a good bug-report, so we try to outline some guidelines
|
||||
that may help:
|
||||
a trivial task to submit a good bug-report, so we try to outline some
|
||||
guidelines that may help:
|
||||
|
||||
* First explain the functionality issue
|
||||
We have seen many bug reports which were just a stack-trace dump, with no
|
||||
|
@ -91,8 +94,9 @@ that may help:
|
|||
just mis-understood the feature.
|
||||
|
||||
* Explain how to reproduce
|
||||
It is very difficult to mark a bug as solved, if we don't know how you reached
|
||||
it. Reproduction steps go a long way to make a bug clear and easy to tackle.
|
||||
It is very difficult to mark a bug as solved, if we don't know how you
|
||||
reached it. Reproduction steps go a long way to make a bug clear and easy to
|
||||
tackle.
|
||||
It is also very helpful to have a copy of the deployment configurations, e.g.
|
||||
a config file or (in the case of devstack) a local.conf file.
|
||||
|
||||
|
|
|
@ -26,8 +26,8 @@ Dragonflow, we will only support allowed address pairs using IP addresses (not
|
|||
IP address prefixes) in the same subnet of the port's fixed IP.
|
||||
|
||||
In current implementation, security modules like port security and security
|
||||
group will require that packets sent/received from a VM port which must have the
|
||||
fixed IP/MAC address of this VM port. Besides, L2 and L3 transmission will
|
||||
group will require that packets sent/received from a VM port which must have
|
||||
the fixed IP/MAC address of this VM port. Besides, L2 and L3 transmission will
|
||||
forward packets only according to those fixed addresses. Those modules should
|
||||
make some changes to support allowed address pairs.
|
||||
|
||||
|
|
|
@ -43,51 +43,55 @@ Besides performance, it also has many noticeable advantages such as
|
|||
data cluster is fully implemented without support of external applications.
|
||||
It also provides an operation portal for daily maintenance.
|
||||
|
||||
Currently, we implement control plane of clustering for Redis inside Dragonflow,
|
||||
which is actually beyond the scope of Dragonflow project. The reason why we
|
||||
implement db-api layer is that we do not want to maintain the details of data
|
||||
backend as it is not the responsibility of Dragonflow project.
|
||||
Currently, we implement control plane of clustering for Redis inside
|
||||
Dragonflow, which is actually beyond the scope of Dragonflow project. The
|
||||
reason why we implement db-api layer is that we do not want to maintain the
|
||||
details of data backend as it is not the responsibility of Dragonflow project.
|
||||
|
||||
The disadvantage of Cassandra is that it needs external mechanism for PUB/SUB,
|
||||
for example, Zookeeper or ZeroMQ. The latter has been implemented in Dragonflow,
|
||||
so it is usable for now.
|
||||
for example, Zookeeper or ZeroMQ. The latter has been implemented in
|
||||
Dragonflow, so it is usable for now.
|
||||
|
||||
It is noted that Cassandra is run over JVM.
|
||||
|
||||
Highlights
|
||||
----------
|
||||
|
||||
In this section I will highlight some internal mechanisms of Cassandra that will
|
||||
greatly help Dragonflow scale out and put into production.
|
||||
In this section I will highlight some internal mechanisms of Cassandra that
|
||||
will greatly help Dragonflow scale out and put into production.
|
||||
|
||||
#. You can adjust ReplicationFactor to have multiple replications across data centers.
|
||||
#. You can adjust ConsistencyLevel to use different algorithms, like Quorum.
|
||||
#. Every node in the cluster is identical. No Master or Slave roles.
|
||||
#. The data written to Cassandra node is going to append-only CommitLog first and
|
||||
fsync to disk next. You also can adjust the policy of fsync. It guarantees the durability.
|
||||
#. You can adjust ReplicationFactor to have multiple replications across data
|
||||
centers.
|
||||
#. You can adjust ConsistencyLevel to use different algorithms, like Quorum.
|
||||
#. Every node in the cluster is identical. No Master or Slave roles.
|
||||
#. The data written to Cassandra node is going to append-only CommitLog first
|
||||
and fsync to disk next. You also can adjust the policy of fsync. It
|
||||
guarantees the durability.
|
||||
|
||||
High Availability
|
||||
-----------------
|
||||
|
||||
You just need to specify a set of nodes in configuration, *remote_db_hosts* in [df] section.
|
||||
The nodes will automatically form a Quorum-like cluster with replications and consistency
|
||||
you specify in Cassandra configuration.
|
||||
You just need to specify a set of nodes in configuration, *remote_db_hosts* in
|
||||
[df] section.
|
||||
The nodes will automatically form a Quorum-like cluster with replications and
|
||||
consistency you specify in Cassandra configuration.
|
||||
|
||||
JVM in Production
|
||||
-----------------
|
||||
|
||||
Although this section is beyond the scope of Dragonflow, the following links are provided
|
||||
by Cassandra official to guide users on tuning Cassandra and JVM.
|
||||
Although this section is beyond the scope of Dragonflow, the following links
|
||||
are provided by Cassandra official to guide users on tuning Cassandra and JVM.
|
||||
|
||||
#. https://docs.datastax.com/en/landing_page/doc/landing_page/recommendedSettingsLinux.html
|
||||
#. https://docs.datastax.com/en/cassandra/3.x/cassandra/operations/opsTuneJVM.html
|
||||
|
||||
It is observed that the operations on data store in Dragonflow is read intensive according to
|
||||
monitoring in the production. This is actually not the Dragonflow's characteristic but the
|
||||
Neutron's. Most of the operations on data store in Neutron are *high concurrent read*.
|
||||
It is observed that the operations on data store in Dragonflow is read
|
||||
intensive according to monitoring in the production. This is actually not the
|
||||
Dragonflow's characteristic but the Neutron's. Most of the operations on data
|
||||
store in Neutron are *high concurrent read*.
|
||||
|
||||
Here is another link [#]_ that provides hints on how to optimize JVM in Cassandra for
|
||||
read heavy workloads.
|
||||
Here is another link [#]_ that provides hints on how to optimize JVM in
|
||||
Cassandra for read heavy workloads.
|
||||
|
||||
.. [#] https://www.planetcassandra.org/blog/cassandra-tuning-the-jvm-for-read-heavy-workloads/
|
||||
|
||||
|
|
|
@ -33,7 +33,8 @@ Firstly, we use ``tox -e genconfig`` to generate all the conf files.
|
|||
If tox is not prepared, we introduce ./tools/generate_config_file_samples.sh
|
||||
instead.
|
||||
|
||||
Secondly, we use etc/oslo-config-generator/dragonflow.ini to manage oslo options.
|
||||
Secondly, we use etc/oslo-config-generator/dragonflow.ini to manage oslo
|
||||
options.
|
||||
For example::
|
||||
[DEFAULT]
|
||||
output_file = etc/dragonflow.ini.sample
|
||||
|
@ -41,8 +42,8 @@ wrap_width = 79
|
|||
namespace = dragonflow
|
||||
namespace = oslo.log
|
||||
|
||||
Finally, we implements dragonflow/opts.py to include all the references of options
|
||||
from different modules of dragonflow.
|
||||
Finally, we implements dragonflow/opts.py to include all the references of
|
||||
options from different modules of dragonflow.
|
||||
|
||||
References
|
||||
==========
|
||||
|
|
|
@ -8,41 +8,49 @@
|
|||
Dragonflow control plane test
|
||||
=============================
|
||||
|
||||
New Dragonflow version has asynchronous architecture. On each object creation command,
|
||||
happens the following:
|
||||
New Dragonflow version has asynchronous architecture. On each object creation
|
||||
command, happens the following:
|
||||
|
||||
In Neutron:
|
||||
1. Neutron receives an API call to create an object (network, subnet, port, etc...)
|
||||
2. Neutron calls Dragonflow plugin with just created object
|
||||
3. Dragonflow plugin saves a new record to NoSQL database
|
||||
4. Dragonflow Plugin sends a message with the new object id using the MQ Pub/Sub
|
||||
|
||||
1. Neutron receives an API call to create an object (network, subnet, port,
|
||||
etc...)
|
||||
2. Neutron calls Dragonflow plugin with just created object
|
||||
3. Dragonflow plugin saves a new record to NoSQL database
|
||||
4. Dragonflow Plugin sends a message with the new object id using the MQ
|
||||
Pub/Sub
|
||||
|
||||
In Dragonflow controller
|
||||
1. DF controller receives a message from MQ Pub/Sub with an object id
|
||||
2. DF controller fetches full object records from NoSQL db
|
||||
3. DF controller, if required, creates necessary OpenFlow rules.
|
||||
|
||||
1. DF controller receives a message from MQ Pub/Sub with an object id
|
||||
2. DF controller fetches full object records from NoSQL db
|
||||
3. DF controller, if required, creates necessary OpenFlow rules.
|
||||
|
||||
Now a new object is basically transferred to the actual OpenFlow rules.
|
||||
|
||||
In the control plane test we will basically test all components of the Dragonflow
|
||||
architecture.
|
||||
In the control plane test we will basically test all components of the
|
||||
Dragonflow architecture.
|
||||
|
||||
This test plan consists of the following sections:
|
||||
1. NoSQL Database Test Plan
|
||||
2. Message Publish Subscribe Test Plan
|
||||
3. Neutron Objects Set-Up Time
|
||||
4. Stress Tests
|
||||
|
||||
1. NoSQL Database Test Plan
|
||||
2. Message Publish Subscribe Test Plan
|
||||
3. Neutron Objects Set-Up Time
|
||||
4. Stress Tests
|
||||
|
||||
Out of scope:
|
||||
-------------
|
||||
1. Evaluation of network throughput in case the system has hundreds of security rules.
|
||||
We check only time required to deploy these rules in this project.
|
||||
1. Evaluation of network throughput in case the system has hundreds of
|
||||
security rules.
|
||||
|
||||
We check only time required to deploy these rules in this project.
|
||||
|
||||
Our goals:
|
||||
----------
|
||||
In this project we would like to see that the time required to add new configuration
|
||||
element to the system will not exceed XXX milliseconds and we do not have a noticeable
|
||||
degradation when the system has more configuration entities.
|
||||
In this project we would like to see that the time required to add new
|
||||
configuration element to the system will not exceed XXX milliseconds and we do
|
||||
not have a noticeable degradation when the system has more configuration
|
||||
entities.
|
||||
|
||||
|
||||
1. NoSQL Database Test Plan
|
||||
|
@ -51,21 +59,21 @@ degradation when the system has more configuration entities.
|
|||
Dragonflow can work with a number of NoSQL databases. Each database has t's own
|
||||
advantages and disadvantages.
|
||||
|
||||
At the time of writing this document, Dragonflow supports the following key / value
|
||||
databases:
|
||||
At the time of writing this document, Dragonflow supports the following key /
|
||||
value databases:
|
||||
|
||||
1. Etcd
|
||||
2. Redis
|
||||
3. Ramcloud
|
||||
4. Zookeeper
|
||||
1. Etcd
|
||||
2. Redis
|
||||
3. Ramcloud
|
||||
4. Zookeeper
|
||||
|
||||
Some of the databases above support clustering. We will perform tests against
|
||||
database server configured in single and multiple node.
|
||||
|
||||
The system will be tested against the following configuration:
|
||||
|
||||
1. All in one server together with NoSQL server
|
||||
2. Two or more servers required to enable NoSQL clustering configuration.
|
||||
1. All in one server together with NoSQL server
|
||||
2. Two or more servers required to enable NoSQL clustering configuration.
|
||||
|
||||
Some of the results will be compared with MySQL. See bellow.
|
||||
|
||||
|
@ -73,7 +81,8 @@ NoSQL random access test scenario
|
|||
=================================
|
||||
|
||||
The test will create 1000 database objects and performs random access test.
|
||||
We can optionally compare the results against MySQL by reading values from Neutron DB.
|
||||
We can optionally compare the results against MySQL by reading values from
|
||||
Neutron DB.
|
||||
These tests will mimic the case when Dragonflow receives notification when an
|
||||
object is created and it needs to load object records from the database.
|
||||
|
||||
|
@ -95,7 +104,7 @@ TODO: create a script here
|
|||
|
||||
TODO: Create a script that records db process activity
|
||||
|
||||
3. Generate report
|
||||
4. Generate report
|
||||
------------------
|
||||
|
||||
Generate comparison table.
|
||||
|
@ -103,13 +112,15 @@ Generate comparison table.
|
|||
|
||||
2. Message Publish Subscribe Test Plan
|
||||
======================================
|
||||
Message publish/subscribe test plan is intended to test Dragonflow PUB/SUB mechanism.
|
||||
Message publish/subscribe test plan is intended to test Dragonflow PUB/SUB
|
||||
mechanism.
|
||||
This test measures the aggregate throughput of a MQ layer.
|
||||
|
||||
This test will cover the following:
|
||||
1. ZeroMQ Pub/Sub
|
||||
2. Redis Pub/Sub
|
||||
3. Polling for DB changes
|
||||
|
||||
1. ZeroMQ Pub/Sub
|
||||
2. Redis Pub/Sub
|
||||
3. Polling for DB changes
|
||||
|
||||
|
||||
3. Neutron Objects Set-Up Time
|
||||
|
@ -117,24 +128,27 @@ This test will cover the following:
|
|||
|
||||
New Dragonflow version has asynchronous architecture. As covered before
|
||||
it has 2 main parts:
|
||||
1. Neutron API call handling with Dragonflow Plugin and pushing a notification
|
||||
to messaging queue
|
||||
2. Actual processing on the API call in Dragonflow controller
|
||||
|
||||
1. Neutron API call handling with Dragonflow Plugin and pushing a
|
||||
notification to messaging queue
|
||||
2. Actual processing on the API call in Dragonflow controller
|
||||
|
||||
In our test we will tests the following:
|
||||
1. Test time for Neutron to handle API call and push notification to messaging queue
|
||||
2. The same as above with actual time required to handle object creation/modification
|
||||
request in Dragonflow controller.
|
||||
|
||||
1. Test time for Neutron to handle API call and push notification to
|
||||
messaging queue
|
||||
2. The same as above with actual time required to handle object
|
||||
creation/modification request in Dragonflow controller.
|
||||
|
||||
We will test the following Neutron configuration entities
|
||||
---------------------------------------------------------
|
||||
1. Network
|
||||
2. Subnetwork
|
||||
3. Router
|
||||
4. Security rules
|
||||
5. Security groups
|
||||
6. Network ports
|
||||
7. Floating ips
|
||||
1. Network
|
||||
2. Subnetwork
|
||||
3. Router
|
||||
4. Security rules
|
||||
5. Security groups
|
||||
6. Network ports
|
||||
7. Floating ips
|
||||
|
||||
Basic test at zero state
|
||||
------------------------
|
||||
|
@ -142,21 +156,21 @@ We will calculate time to create multiple objects when system is at zero state.
|
|||
We define zero state as a state where we have a system with default rules only.
|
||||
We will do the following tests:
|
||||
|
||||
1. 1 object created/updated
|
||||
2. 5 objects created/updated
|
||||
3. 10 objects created/updated
|
||||
4. 20 objects created/updated
|
||||
5. 50 objects created/updated
|
||||
6. 100 objects created/updated
|
||||
7. 200 objects created/updated
|
||||
8. 300 objects created/updated
|
||||
9. 400 objects created/updated
|
||||
10. 500 objects created/updated
|
||||
11. 600 objects created/updated
|
||||
12. 700 objects created/updated
|
||||
13. 800 objects created/updated
|
||||
14. 900 objects created/updated
|
||||
15. 1000 objects created/updated
|
||||
1. 1 object created/updated
|
||||
2. 5 objects created/updated
|
||||
3. 10 objects created/updated
|
||||
4. 20 objects created/updated
|
||||
5. 50 objects created/updated
|
||||
6. 100 objects created/updated
|
||||
7. 200 objects created/updated
|
||||
8. 300 objects created/updated
|
||||
9. 400 objects created/updated
|
||||
10. 500 objects created/updated
|
||||
11. 600 objects created/updated
|
||||
12. 700 objects created/updated
|
||||
13. 800 objects created/updated
|
||||
14. 900 objects created/updated
|
||||
15. 1000 objects created/updated
|
||||
|
||||
|
||||
Multiple tenants
|
||||
|
@ -173,9 +187,10 @@ measure time to add a new object to the system that is actively used.
|
|||
|
||||
What we are going to test
|
||||
-------------------------
|
||||
1. Check that objects are created are valid and correct Openflow rules are created
|
||||
1. We will measure time to create one or group of objects
|
||||
2. We will measure CPU usage
|
||||
1. Check that objects are created are valid and correct Openflow rules are
|
||||
created
|
||||
2. We will measure time to create one or group of objects
|
||||
3. We will measure CPU usage
|
||||
|
||||
Now we will be able to perform regression tests and compare results with
|
||||
new and old Dragonflow versions. In addition, we can run similar tests
|
||||
|
@ -193,22 +208,24 @@ box and how much time it takes to deploy all of them. In addition, we want
|
|||
to check that all of the VMs got an IP address.
|
||||
|
||||
Test scenarios for single server installation:
|
||||
1. 1000 updates on one subnet (enable / disable DHCP)
|
||||
1. 1 Router with 1000 Subnetworks
|
||||
2. 1000 Routers - 1000 Subnetwork (1 subnetwork in 1 router)
|
||||
3. 100 Routers - 500 subnets (5 subnets per router)
|
||||
4. 1000 Security rules for 1 VM
|
||||
5. 1000 Security rules for 10 VMs
|
||||
6. Launch 200 VMs
|
||||
7. Set up 1000 Security rules in 1 Security group
|
||||
8. Etc...
|
||||
|
||||
1. 1000 updates on one subnet (enable / disable DHCP)
|
||||
2. 1 Router with 1000 Subnetworks
|
||||
3. 1000 Routers - 1000 Subnetwork (1 subnetwork in 1 router)
|
||||
4. 100 Routers - 500 subnets (5 subnets per router)
|
||||
5. 1000 Security rules for 1 VM
|
||||
6. 1000 Security rules for 10 VMs
|
||||
7. Launch 200 VMs
|
||||
8. Set up 1000 Security rules in 1 Security group
|
||||
9. Etc...
|
||||
|
||||
|
||||
Additional action items:
|
||||
|
||||
"There is also a control plane performance issue when we try to catch on the spec of
|
||||
typical AWS limit (200 subnets per router). When a router with 200 subnets is
|
||||
scheduled on a new host, a 30s delay is watched when all data plane setup is finished."
|
||||
"There is also a control plane performance issue when we try to catch on the
|
||||
spec of typical AWS limit (200 subnets per router). When a router with 200
|
||||
subnets is scheduled on a new host, a 30s delay is watched when all data plane
|
||||
setup is finished."
|
||||
|
||||
"Create max Subnet on a router or for a tenant test create 1000 SG etc"
|
||||
|
||||
|
|
|
@ -12,8 +12,9 @@ For forwarding packets toward the dhcp-port its needs to install flow for
|
|||
each port in the subnet.
|
||||
|
||||
This behaviour causes complexity in the code and create a performance penalty,
|
||||
that because every change in subnet (updated/add/delete/enabled-dhcp/disabled-dhcp)
|
||||
raises the need to do diff with the previous state, and install/remove all relevant flows.
|
||||
that because every change in subnet
|
||||
(updated/add/delete/enabled-dhcp/disabled-dhcp) raises the need to do diff with
|
||||
the previous state, and install/remove all relevant flows.
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
@ -29,8 +30,9 @@ plugin for using it's ip-address-managemnt.
|
|||
Neutron data-model impact
|
||||
-------------------------
|
||||
Instead of one dhcp logical port subnet, there will be
|
||||
one dhcp-port per network.That port will be connected to all enabled-dhcp-subnet,
|
||||
and will hold an ip for each subnet (The port owner will remain "neutron:dhcp").
|
||||
one dhcp-port per network.That port will be connected to all
|
||||
enabled-dhcp-subnet, and will hold an ip for each subnet (The port owner will
|
||||
remain "neutron:dhcp").
|
||||
|
||||
Dragonflow data-model impact
|
||||
----------------------------
|
||||
|
|
|
@ -83,7 +83,8 @@ table will be added to the Dragonflow ingress and egress pipelines.
|
|||
|
||||
To decouple ports and firewall rules, the firewall group port table only handle
|
||||
ports that has been added to a firewall group, and the firewall rule table maps
|
||||
the firewall rules to Openflow flow entries without caring adding/removing ports.
|
||||
the firewall rules to Openflow flow entries without caring adding/removing
|
||||
ports.
|
||||
When a port is added or deleted from a firewall group, only the firewall group
|
||||
port table is updated and the traffic from/to other ports in the firewall group
|
||||
will not be affected during the updating.
|
||||
|
@ -160,12 +161,12 @@ The priority of the flow entry in firewall rule table is corresponding to the
|
|||
order of firewall rules. The rules come first have the higher priority.
|
||||
|
||||
To support inserting firewall rules, we use a big number as the priority when
|
||||
firewall group is created and leave a big gap between rules. For example, a rule
|
||||
is inserted between rule1 with priority A and rule2 with priority B in an
|
||||
firewall group is created and leave a big gap between rules. For example, a
|
||||
rule is inserted between rule1 with priority A and rule2 with priority B in an
|
||||
existing firewall policy, the firewall APP will check if there is a number
|
||||
between A and B available. If yes, install the flow with this number as the
|
||||
priority; if no, re-organize the priorities of all the flows, and then reinstall
|
||||
them.
|
||||
priority; if no, re-organize the priorities of all the flows, and then
|
||||
reinstall them.
|
||||
|
||||
Ingress:
|
||||
|
||||
|
@ -211,9 +212,9 @@ It is similar to the Egress pipeline:
|
|||
NB Data Model Impact
|
||||
--------------------
|
||||
|
||||
Three tables will be added to the Dragonflow Northbound DB, firewall group table,
|
||||
firewall policy table, firewall rule table. Similar to the Neutron FwaaS data
|
||||
model, firewall group tables contains ingress firewall policy
|
||||
Three tables will be added to the Dragonflow Northbound DB, firewall group
|
||||
table, firewall policy table, firewall rule table. Similar to the Neutron FwaaS
|
||||
data model, firewall group tables contains ingress firewall policy
|
||||
and egress firewall policy, as well a list of ports. Each firewall policy
|
||||
tables contains a list of firewall rules in the policy.
|
||||
|
||||
|
|
|
@ -306,18 +306,21 @@ We propose the following new configuration:
|
|||
router ports connected to this subnet are not multicast routers. IGMP
|
||||
packets are treated as regular routed IP packets. MCPs are not routed to
|
||||
sibling networks. IGMP queries are not sent. Default - True
|
||||
*robustness-variable* : Integer - The robustness variable as defined in [1].
|
||||
*robustness-variable* : Integer - The robustness variable as defined in
|
||||
[1].
|
||||
While not used directly, it is used to calculate the *Group membership
|
||||
interval*, default values for *Startup query count*, and *Last member
|
||||
query count*. Default - 2
|
||||
*query-interval* : Integer - the interval between General Queries sent by
|
||||
the MCVR. Default - 125 (Seconds)
|
||||
*query-response-interval* : Integer - used to calculate the maximum amount
|
||||
of time a IGMP group member may respond to a query. Default - 10 (Seconds)
|
||||
of time a IGMP group member may respond to a query. Default - 10
|
||||
(Seconds)
|
||||
*startup-query-interval* : Integer - the interval between General Queries
|
||||
sent by an MCVR on startup. Default - 1/4 of *query-interval*
|
||||
*startup-query-count* : Integer - number of Queries sent out on startup,
|
||||
separated by the *startup-query-interval*. Default - *robustness-variable*
|
||||
separated by the *startup-query-interval*. Default -
|
||||
*robustness-variable*
|
||||
*last-member-query-interval* : Integer - used to calculate the maximum
|
||||
amount of time an IGMP group member may respond to a group-specific query
|
||||
sent in response to a leave message. Default - 1 (Seconds)
|
||||
|
|
|
@ -127,25 +127,25 @@ LoadBalancer
|
|||
|
||||
This is the main object describing the load balancer.
|
||||
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+=====================================+
|
||||
| id | String | LoadBalancer identifier |
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
| topic | String | Project ID |
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
| enabled | Boolean | Is the load balancer enabled? |
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
| listeners | ReferenceList<Listener> | On what protocols/ports to listen? |
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
| network | Reference<LogicalNetwork>| On what network is the VIP? |
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
| subnet | Reference<Subnet> | On what subnet is the VIP? |
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
| port | Reference<LogicalPort> | On what (virtual) port is the VIP? |
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
| ip_address| IPAddress | What's the VIP? |
|
||||
+-----------+--------------------------+-------------------------------------+
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+===================================+
|
||||
| id | String | LoadBalancer identifier |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| topic | String | Project ID |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| enabled | Boolean | Is the load balancer enabled? |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| listeners | ReferenceList<Listener> | On what protocols/ports to listen?|
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| network | Reference<LogicalNetwork>| On what network is the VIP? |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| subnet | Reference<Subnet> | On what subnet is the VIP? |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| port | Reference<LogicalPort> | On what (virtual) port is the VIP?|
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| ip_address| IPAddress | What's the VIP? |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
|
||||
Endpoint
|
||||
~~~~~~~~
|
||||
|
@ -160,103 +160,105 @@ Need to support protocols tcp, udp, icmp, null (raw?), and http (at least)
|
|||
|
||||
TCP or UDP Endpoint:
|
||||
|
||||
+---------------+----------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+======================+=====================================+
|
||||
| protocol | Enum (UDP, TCP) | The protocol for this endpoint |
|
||||
+---------------+----------------------+-------------------------------------+
|
||||
| ports | PortRange | The ports to match on |
|
||||
+---------------+----------------------+-------------------------------------+
|
||||
+---------------+----------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+======================+===================================+
|
||||
| protocol | Enum (UDP, TCP) | The protocol for this endpoint |
|
||||
+---------------+----------------------+-----------------------------------+
|
||||
| ports | PortRange | The ports to match on |
|
||||
+---------------+----------------------+-----------------------------------+
|
||||
|
||||
ICMP Endpoint:
|
||||
|
||||
+---------------+----------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+======================+=====================================+
|
||||
| protocol | Enum (PING) | The protocol for this endpoint |
|
||||
+---------------+----------------------+-------------------------------------+
|
||||
+---------------+----------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+======================+===================================+
|
||||
| protocol | Enum (PING) | The protocol for this endpoint |
|
||||
+---------------+----------------------+-----------------------------------+
|
||||
|
||||
HTTP Endpoint:
|
||||
|
||||
+---------------+---------------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+===========================+=====================================+
|
||||
| protocol | Enum (HTTP) | The protocol for this endpoint |
|
||||
+---------------+---------------------------+-------------------------------------+
|
||||
| policies | ReferenceList<HTTPPolicy> | HTTP match policies |
|
||||
+---------------+---------------------------+-------------------------------------+
|
||||
+--------------+---------------------------+-------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==============+===========================+===============================+
|
||||
| protocol | Enum (HTTP) | The protocol for this endpoint|
|
||||
+--------------+---------------------------+-------------------------------+
|
||||
| policies | ReferenceList<HTTPPolicy> | HTTP match policies |
|
||||
+--------------+---------------------------+-------------------------------+
|
||||
|
||||
Where an HTTP policy object is:
|
||||
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+==============================+=====================================+
|
||||
| action | Embed<Action> | The action of this policy |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| enabled | Boolean | Is the policy enabled? |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| rules | ReferenceList<HTTPRule> | The rules when the policy matches |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
+-----------+---------------------------+----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+===========================+==================================+
|
||||
| action | Embed<Action> | The action of this policy |
|
||||
+-----------+---------------------------+----------------------------------+
|
||||
| enabled | Boolean | Is the policy enabled? |
|
||||
+-----------+---------------------------+----------------------------------+
|
||||
| rules | ReferenceList<HTTPRule> | The rules when the policy matches|
|
||||
+-----------+---------------------------+----------------------------------+
|
||||
|
||||
An action can be one of:
|
||||
|
||||
Reject action:
|
||||
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+==============================+=====================================+
|
||||
| action_type | Enum (Reject) | The action of this policy |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
+---------------+------------------------------+---------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+==============================+===========================+
|
||||
| action_type | Enum (Reject) | The action of this policy |
|
||||
+---------------+------------------------------+---------------------------+
|
||||
|
||||
Redirect to pool action:
|
||||
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+==============================+=====================================+
|
||||
| action_type | Enum (REDIRECT_TO_POOL) | The action of this policy |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| pool | Reference<Pool> | The pool to redirect the session |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
+-------------+--------------------------+---------------------------------+
|
||||
| Name | Type | Description |
|
||||
+=============+==========================+=================================+
|
||||
| action_type | Enum (REDIRECT_TO_POOL) | The action of this policy |
|
||||
+-------------+--------------------------+---------------------------------+
|
||||
| pool | Reference<Pool> | The pool to redirect the session|
|
||||
+-------------+--------------------------+---------------------------------+
|
||||
|
||||
Redirect to URL action:
|
||||
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+==============================+=====================================+
|
||||
| action_type | Enum (REDIRECT_TO_URL) | The action of this policy |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| url | String (Or a URL type) | The URL to redirect the session |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
+---------------+-------------------------+--------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+=========================+================================+
|
||||
| action_type | Enum (REDIRECT_TO_URL) | The action of this policy |
|
||||
+---------------+-------------------------+--------------------------------+
|
||||
| url | String (Or a URL type) | The URL to redirect the session|
|
||||
+---------------+-------------------------+--------------------------------+
|
||||
|
||||
An HTTP Rule object is:
|
||||
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+==============================+=====================================+
|
||||
| operation | Enum (CONTAINS, ...) | The operation this rule tests |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| is_invert | Boolean | Should the operation be inverted? |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| type | Enum(COOKIE, FILE_TYPE, ...) | The type of key in the comparison |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| key | String | The key in the comparison |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
| value | String | The literal to compare against |
|
||||
+---------------+------------------------------+-------------------------------------+
|
||||
+----------+-----------------------------+---------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==========+=============================+=================================+
|
||||
| operation| Enum (CONTAINS, ...) | The operation this rule tests |
|
||||
+----------+-----------------------------+---------------------------------+
|
||||
| is_invert| Boolean | Should the operation be |
|
||||
| | | inverted? |
|
||||
+----------+-----------------------------+---------------------------------+
|
||||
| type | Enum(COOKIE, FILE_TYPE, ...)| The type of key in the |
|
||||
| | | comparison |
|
||||
+----------+-----------------------------+---------------------------------+
|
||||
| key | String | The key in the comparison |
|
||||
+----------+-----------------------------+---------------------------------+
|
||||
| value | String | The literal to compare against |
|
||||
+----------+-----------------------------+---------------------------------+
|
||||
|
||||
A policy matches if any rule matches.
|
||||
|
||||
"Raw" protocol
|
||||
|
||||
+---------------+----------------------+------------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+======================+==========================================+
|
||||
| protocol | Enum (RAW) | The protocol for this endpoint |
|
||||
+---------------+----------------------+------------------------------------------+
|
||||
| location | Integer | The location to start the match |
|
||||
+---------------+----------------------+------------------------------------------+
|
||||
| value | String | The value that should be in the location |
|
||||
+---------------+----------------------+------------------------------------------+
|
||||
+---------------+---------------+------------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+===============+==========================================+
|
||||
| protocol | Enum (RAW) | The protocol for this endpoint |
|
||||
+---------------+---------------+------------------------------------------+
|
||||
| location | Integer | The location to start the match |
|
||||
+---------------+---------------+------------------------------------------+
|
||||
| value | String | The value that should be in the location |
|
||||
+---------------+---------------+------------------------------------------+
|
||||
|
||||
An endpoint for the raw protocol accepts a packet only if the raw data at
|
||||
<location> equals <value>.
|
||||
|
@ -267,128 +269,133 @@ TLS
|
|||
This object contains the information needed for the Listener (or Load Balancer)
|
||||
to terminate TLS connections [2]_.
|
||||
|
||||
+---------------+----------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+======================+=====================================+
|
||||
| tls-container | String | TLS container |
|
||||
+---------------+----------------------+-------------------------------------+
|
||||
| sni-container | String | SNI container |
|
||||
+---------------+----------------------+-------------------------------------+
|
||||
+---------------+--------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===============+====================+=====================================+
|
||||
| tls-container | String | TLS container |
|
||||
+---------------+--------------------+-------------------------------------+
|
||||
| sni-container | String | SNI container |
|
||||
+---------------+--------------------+-------------------------------------+
|
||||
|
||||
Listener
|
||||
~~~~~~~~
|
||||
|
||||
This object represents the listening endpoint of a load balanced service.
|
||||
|
||||
+------------------+-------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==================+===================+=====================================+
|
||||
| id | String | |
|
||||
+------------------+-------------------+-------------------------------------+
|
||||
| topic | String | |
|
||||
+------------------+-------------------+-------------------------------------+
|
||||
| enabled | Boolean | Is the listener enabled? |
|
||||
+------------------+-------------------+-------------------------------------+
|
||||
| conenction_limit | Integer | Max number of connections permitted |
|
||||
+------------------+-------------------+-------------------------------------+
|
||||
| tls | Embed<TLS> | Object needed to terminate HTTPS |
|
||||
+------------------+-------------------+-------------------------------------+
|
||||
| endpoint | Embed<Endpoint> | The protocol (and port) to listen on|
|
||||
+------------------+-------------------+-------------------------------------+
|
||||
| pool | Reference<Pool> | The pool to load-balance |
|
||||
+------------------+-------------------+-------------------------------------+
|
||||
+------------------+-----------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==================+=================+=====================================+
|
||||
| id | String | |
|
||||
+------------------+-----------------+-------------------------------------+
|
||||
| topic | String | |
|
||||
+------------------+-----------------+-------------------------------------+
|
||||
| enabled | Boolean | Is the listener enabled? |
|
||||
+------------------+-----------------+-------------------------------------+
|
||||
| conenction_limit | Integer | Max number of connections permitted |
|
||||
+------------------+-----------------+-------------------------------------+
|
||||
| tls | Embed<TLS> | Object needed to terminate HTTPS |
|
||||
+------------------+-----------------+-------------------------------------+
|
||||
| endpoint | Embed<Endpoint> | The protocol (and port) to listen on|
|
||||
+------------------+-----------------+-------------------------------------+
|
||||
| pool | Reference<Pool> | The pool to load-balance |
|
||||
+------------------+-----------------+-------------------------------------+
|
||||
|
||||
Pool
|
||||
~~~~
|
||||
|
||||
A group of members to which the listener forwards client requests.
|
||||
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+=====================+==============================+=====================================+
|
||||
| id | String | |
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
| topic | String | |
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
| enabled | Boolean | Is the pool enabled? |
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
| health_monitor | Reference<HealthMonitor> | Health monitor object |
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
| algorithm | Enum(ROUND_ROBIN, ...) | supported algorithms |
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
| members | ReferenceList<Member> | List of ppol members |
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
| protocol | Enum(tcp, udp, icmp, ...) | The protocol supported by this pool |
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
| session_persistence | Embed<SessionPersistence> | How to detect session |
|
||||
+---------------------+------------------------------+-------------------------------------+
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
| Name | Type | Description |
|
||||
+=====================+==========================+=========================+
|
||||
| id | String | |
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
| topic | String | |
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
| enabled | Boolean | Is the pool enabled? |
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
| health_monitor | Reference<HealthMonitor> | Health monitor object |
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
| algorithm | Enum(ROUND_ROBIN, ...) | supported algorithms |
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
| members | ReferenceList<Member> | List of ppol members |
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
| protocol | Enum(tcp, udp, icmp, ...)| The protocol supported |
|
||||
| | | by this pool |
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
| session_persistence | Embed<SessionPersistence>| How to detect session |
|
||||
+---------------------+--------------------------+-------------------------+
|
||||
|
||||
There are multiple ways to maintain session persistence. The following is an
|
||||
incomplete list of options.
|
||||
|
||||
No session persistence:
|
||||
|
||||
+-----------+--------------------------+---------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+=======================================+
|
||||
| type | Enum (None) | Must be 'None' |
|
||||
+-----------+--------------------------+---------------------------------------+
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+===================================+
|
||||
| type | Enum (None) | Must be 'None' |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
|
||||
There is no session persistence. Every packet is load-balanced independantly.
|
||||
|
||||
Source IP session persistence:
|
||||
|
||||
+-----------+--------------------------+---------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+=======================================+
|
||||
| type | Enum (SOURCE_IP) | Must be 'SOURCE_IP' |
|
||||
+-----------+--------------------------+---------------------------------------+
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+===================================+
|
||||
| type | Enum (SOURCE_IP) | Must be 'SOURCE_IP' |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
|
||||
Packets from the same source IP will be directed to the same pool member.
|
||||
|
||||
5-tuple session persistence:
|
||||
|
||||
+-----------+--------------------------+---------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+=======================================+
|
||||
| type | Enum (5-TUPLE) | Must be '5-TUPLE' |
|
||||
+-----------+--------------------------+---------------------------------------+
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+===================================+
|
||||
| type | Enum (5-TUPLE) | Must be '5-TUPLE' |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
|
||||
Packets with the same 5-tuple will be directed to the same pool member. In the
|
||||
case of ICMP, or protocls that do not have port numbers, 3-tuples will be used.
|
||||
|
||||
HTTP cookie session persistence:
|
||||
|
||||
+-----------+--------------------------+----------------------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+====================================================+
|
||||
| type | Enum (HTTP_COOKIE) | Must be 'HTTP_COOKIE' |
|
||||
+-----------+--------------------------+----------------------------------------------------+
|
||||
| is_create | Boolean | Should the cookie be created by the load balancer? |
|
||||
+-----------+--------------------------+----------------------------------------------------+
|
||||
| name | String | The name of the cookie to use |
|
||||
+-----------+--------------------------+----------------------------------------------------+
|
||||
+-----------+--------------------+-----------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+====================+=========================================+
|
||||
| type | Enum (HTTP_COOKIE) | Must be 'HTTP_COOKIE' |
|
||||
+-----------+--------------------+-----------------------------------------+
|
||||
| is_create | Boolean | Should the cookie be created by the load|
|
||||
| | | balancer? |
|
||||
+-----------+--------------------+-----------------------------------------+
|
||||
| name | String | The name of the cookie to use |
|
||||
+-----------+--------------------+-----------------------------------------+
|
||||
|
||||
PoolMember
|
||||
~~~~~~~~~~
|
||||
|
||||
This object describes a single pool member.
|
||||
|
||||
+-----------+--------------------------+--------------------------------------------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+====================================================================+
|
||||
| id | String | |
|
||||
+-----------+--------------------------+--------------------------------------------------------------------+
|
||||
| topic | String | |
|
||||
+-----------+--------------------------+--------------------------------------------------------------------+
|
||||
| enabled | Boolean | |
|
||||
+-----------+--------------------------+--------------------------------------------------------------------+
|
||||
| port | Reference<LogicalPort> | The pool members logical port (containing IP, subnet, etc.) |
|
||||
+-----------+--------------------------+--------------------------------------------------------------------+
|
||||
| weight | Integer | The weight of the member, used in the LB algorithms |
|
||||
+-----------+--------------------------+--------------------------------------------------------------------+
|
||||
| endpoint | Embed<Endpoint> | The endpoint the member listens on. Used for translation if needed |
|
||||
+-----------+--------------------------+--------------------------------------------------------------------+
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+===========+==========================+===================================+
|
||||
| id | String | |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| topic | String | |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| enabled | Boolean | |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| port | Reference<LogicalPort> | The pool members logical port |
|
||||
| | | (containing IP, subnet, etc.) |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| weight | Integer | The weight of the member, used in |
|
||||
| | | the LB algorithms |
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
| endpoint | Embed<Endpoint> | The endpoint the member listens |
|
||||
| | | on. Used for translation if needed|
|
||||
+-----------+--------------------------+-----------------------------------+
|
||||
|
||||
Health Monitor
|
||||
~~~~~~~~~~~~~~
|
||||
|
@ -396,23 +403,24 @@ Health Monitor
|
|||
This object represents a health monitor, i.e. a network device that
|
||||
periodically pings the pool members.
|
||||
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==============+================================+===================================+
|
||||
| id | String | |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| topic | String | |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| enabled | Boolean | Is this health monitor enabled? |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| delay | Integer | Interval between probes (seconds) |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| method | Embed<HealthMonitorMethod> | Probe method |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| max_retries | Integer | Number of allowed failed probes |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| timeout | Integer | Probe timeout (seconds) |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
+------------+---------------------------+---------------------------------+
|
||||
| Name | Type | Description |
|
||||
+============+===========================+=================================+
|
||||
| id | String | |
|
||||
+------------+---------------------------+---------------------------------+
|
||||
| topic | String | |
|
||||
+------------+---------------------------+---------------------------------+
|
||||
| enabled | Boolean | Is this health monitor enabled? |
|
||||
+------------+---------------------------+---------------------------------+
|
||||
| delay | Integer | Interval between probes |
|
||||
| | | (seconds) |
|
||||
+------------+---------------------------+---------------------------------+
|
||||
| method | Embed<HealthMonitorMethod>| Probe method |
|
||||
+------------+---------------------------+---------------------------------+
|
||||
| max_retries| Integer | Number of allowed failed probes |
|
||||
+------------+---------------------------+---------------------------------+
|
||||
| timeout | Integer | Probe timeout (seconds) |
|
||||
+------------+---------------------------+---------------------------------+
|
||||
|
||||
Health Monitor Method
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
@ -422,38 +430,39 @@ or an HTTP request.
|
|||
|
||||
Ping method:
|
||||
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==============+================================+===================================+
|
||||
| method | Enum (PING) | Must be PING |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
+--------------+----------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==============+======================+===================================+
|
||||
| method | Enum (PING) | Must be PING |
|
||||
+--------------+----------------------+-----------------------------------+
|
||||
|
||||
This method pings the pool member. It is not available via the Neutron API.
|
||||
|
||||
TCP method:
|
||||
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==============+================================+===================================+
|
||||
| method | Enum (TCP) | Must be TCP |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
+--------------+----------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==============+======================+===================================+
|
||||
| method | Enum (TCP) | Must be TCP |
|
||||
+--------------+----------------------+-----------------------------------+
|
||||
|
||||
This method probes the pool member by trying to connect to it. The port is taken
|
||||
from the member's endpoint field, or the Listener's endpoint field.
|
||||
This method probes the pool member by trying to connect to it. The port is
|
||||
taken from the member's endpoint field, or the Listener's endpoint field.
|
||||
|
||||
HTTP and HTTPS methods:
|
||||
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==============+================================+===================================+
|
||||
| method | Enum (HTTP, HTTPS) | Must be HTTP or HTTPS |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| url | String (or URL type) | The URL to probe |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| http_method | Enum (GET, POST, ...) | The HTTP method to probe with |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| codes | ReferenceList<Integer> | The allowed response codes |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
+------------+-------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+============+=========================+===================================+
|
||||
| method | Enum (HTTP, HTTPS) | Must be HTTP or HTTPS |
|
||||
+------------+-------------------------+-----------------------------------+
|
||||
| url | String (or URL type) | The URL to probe |
|
||||
+------------+-------------------------+-----------------------------------+
|
||||
| http_method| Enum (GET, POST, ...) | The HTTP method to probe with |
|
||||
+------------+-------------------------+-----------------------------------+
|
||||
| codes | ReferenceList<Integer> | The allowed response codes |
|
||||
+------------+-------------------------+-----------------------------------+
|
||||
|
||||
|
||||
Health Monitor Status
|
||||
---------------------
|
||||
|
@ -463,15 +472,15 @@ this table with pool member status, as well as sending updates to Neutron
|
|||
using e.g. Neutron API or the existing status notification mechanism.
|
||||
|
||||
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| Name | Type | Description |
|
||||
+==============+================================+===================================+
|
||||
| member | ID | The monitored pool member's ID |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| chassis | ID | The name of the hosting chassis |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
| status | Enum (ACTIVE, DOWN, ERROR, ...)| The status of the pool member |
|
||||
+--------------+--------------------------------+-----------------------------------+
|
||||
+--------+--------------------------------+--------------------------------+
|
||||
| Name | Type | Description |
|
||||
+========+================================+================================+
|
||||
| member | ID | The monitored pool member's ID |
|
||||
+--------+--------------------------------+--------------------------------+
|
||||
| chassis| ID | The name of the hosting chassis|
|
||||
+--------+--------------------------------+--------------------------------+
|
||||
| status | Enum (ACTIVE, DOWN, ERROR, ...)| The status of the pool member |
|
||||
+--------+--------------------------------+--------------------------------+
|
||||
|
||||
Implementation
|
||||
--------------
|
||||
|
@ -576,8 +585,8 @@ inactivity.
|
|||
Option 1: Controller
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
When an l7 packet is detected, it will be sent to the controller. The controller
|
||||
will verify that this packet matches an endpoint on that IP address.
|
||||
When an l7 packet is detected, it will be sent to the controller. The
|
||||
controller will verify that this packet matches an endpoint on that IP address.
|
||||
|
||||
If the packet does not match any endpoint, it will be returned to be handled
|
||||
by the rest of the pipeline (e.g. L2, L3).
|
||||
|
@ -620,10 +629,10 @@ The HA proxy instance will send probes to peers using their unique_key encoded
|
|||
in the IP destination field. The eth_dst address may also be spoofed to skip
|
||||
the ARP lookup stage.
|
||||
|
||||
The OVS bridge will detect packets coming from the HA proxy. The LBaaS application
|
||||
will install flows which update the layer 2 (eth_dst, eth_src), layer 3 (ip_dst, ip_src),
|
||||
and metadata registers (metadata, reg6, reg7), and send the packet to the
|
||||
destination member.
|
||||
The OVS bridge will detect packets coming from the HA proxy. The LBaaS
|
||||
application will install flows which update the layer 2 (eth_dst, eth_src),
|
||||
layer 3 (ip_dst, ip_src), and metadata registers (metadata, reg6, reg7), and
|
||||
send the packet to the destination member.
|
||||
|
||||
Once a port is detected as down, it will be effectively removed from the pool.
|
||||
It will be marked as down. No new connections will be sent to it.
|
||||
|
|
|
@ -139,8 +139,8 @@ Solution to residual flows
|
|||
--------------------------
|
||||
Residual flows means flows which don't take effect any more but stay in flow
|
||||
table. Backward incompatible upgrade and incorrect implementation may generate
|
||||
this kind of flows. The residual flows may not affect the forwarding but it will
|
||||
occupy flow table space and add difficulty for maintenance.
|
||||
this kind of flows. The residual flows may not affect the forwarding but it
|
||||
will occupy flow table space and add difficulty for maintenance.
|
||||
|
||||
The methods to manage this issue:
|
||||
We could reuse the solution for 'local controller restart', trigger local
|
||||
|
@ -161,11 +161,11 @@ Solution to missing flows
|
|||
-------------------------
|
||||
When there are missing flows, OVS cannot forward the packet by itself, it will
|
||||
forward the packet to local controller. For example, in the context of DVR
|
||||
forwarding, if no corresponding host route flow to destination, OVS will forward
|
||||
the packet to local controller according to the network flow. Upon receive the
|
||||
packet, local controller forward the packet, regenerate host flow and flush it
|
||||
to OVS. We don't plan to discuss it in more detail here and it will be processed
|
||||
by the specific application of Dragonflow.
|
||||
forwarding, if no corresponding host route flow to destination, OVS will
|
||||
forward the packet to local controller according to the network flow. Upon
|
||||
receive the packet, local controller forward the packet, regenerate host flow
|
||||
and flush it to OVS. We don't plan to discuss it in more detail here and it
|
||||
will be processed by the specific application of Dragonflow.
|
||||
|
||||
References
|
||||
==========
|
||||
|
|
|
@ -28,9 +28,11 @@ Proposed Change
|
|||
A new table is added in Dragonflow pipeline for mac spoofing protection.
|
||||
|
||||
This table will have MAC-IP validation rules which blocks any traffic
|
||||
that has different MAC-IP src address than the MAC-IP address configured for the VM.
|
||||
that has different MAC-IP src address than the MAC-IP address configured for
|
||||
the VM.
|
||||
This table can also be used for egress security validations (make sure
|
||||
to dispatch traffic to a certain VM only if it has the correct configured MAC and IP)
|
||||
to dispatch traffic to a certain VM only if it has the correct configured
|
||||
MAC and IP)
|
||||
|
||||
It will also have rules allowing DST broadcast/multicast MAC traffic
|
||||
to pass.
|
||||
|
@ -38,11 +40,12 @@ to pass.
|
|||
Additional drop rules:
|
||||
|
||||
1. Packets with SRC MAC broadcast/multicast bit set.
|
||||
(This option might be needed in some environments, we can leave this as a configurable
|
||||
option in case it is -
|
||||
(This option might be needed in some environments, we can leave this as a
|
||||
configurable option in case it is -
|
||||
https://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/107995-config-catalyst-00.html#mm)
|
||||
|
||||
2. VLAN tagged frames where the TCI "Drop eligible indicator" (TEI) bit is set (congestion)
|
||||
2. VLAN tagged frames where the TCI "Drop eligible indicator" (TEI) bit is set
|
||||
(congestion)
|
||||
|
||||
Following are examples for the flows configured in that table::
|
||||
|
||||
|
@ -101,16 +104,16 @@ We still need to verify that this also block gratitude ARPs.
|
|||
Blocking invalid broadcast/multicast traffic
|
||||
--------------------------------------------
|
||||
As part of the port security feature we should also prevent traffic loops.
|
||||
We drop traffic that has the same src and dst ports classified (the src port register
|
||||
and the dst port register are same).
|
||||
We drop traffic that has the same src and dst ports classified (the src port
|
||||
register and the dst port register are same).
|
||||
This scenario happens when we handle broadcast/multicast traffic and just
|
||||
duplicate packet few times for every port in the broadcast domain or
|
||||
multicast group.
|
||||
|
||||
DHCP protection
|
||||
---------------
|
||||
Protection from DHCP DDoS on the controller (DHCP application) is going to be handled
|
||||
on a different spec that will address controller reliability concerns.
|
||||
Protection from DHCP DDoS on the controller (DHCP application) is going to be
|
||||
handled on a different spec that will address controller reliability concerns.
|
||||
|
||||
References
|
||||
==========
|
||||
|
|
|
@ -119,8 +119,8 @@ For scenario 3, the QoS app will delete the configuration for the port.
|
|||
QoS Object Delete
|
||||
-----------------
|
||||
|
||||
It is not permitted to delete the QoS object attached to some ports. If no ports
|
||||
apply the QoS, it can be deleted from the DragonFlow DB.
|
||||
It is not permitted to delete the QoS object attached to some ports. If no
|
||||
ports apply the QoS, it can be deleted from the DragonFlow DB.
|
||||
|
||||
QoS Object Update
|
||||
-----------------
|
||||
|
|
|
@ -21,8 +21,8 @@ to listen to configuration change events.
|
|||
The flow is as follows:
|
||||
|
||||
* The local controller reads the DB and sync all needed configuration
|
||||
* Then, it waits for change notifications by calling a specific DB driver API with a
|
||||
callback method to report the changes
|
||||
* Then, it waits for change notifications by calling a specific DB driver API
|
||||
with a callback method to report the changes
|
||||
|
||||
It is then the DB driver responsibility to update the controller with
|
||||
any configuration change.
|
||||
|
@ -44,8 +44,8 @@ We require a consistently reliable and scalable publish-subscribe mechanism.
|
|||
In order to be efficient with large scale deployments, we want to sync only
|
||||
relevant configuration and topology with the compute node.
|
||||
|
||||
We therefore want to abstract the publish-subscribe mechanism from the DB driver,
|
||||
so that we can control the published topics in an applicative manner.
|
||||
We therefore want to abstract the publish-subscribe mechanism from the DB
|
||||
driver, so that we can control the published topics in an applicative manner.
|
||||
|
||||
We require that this change will be *optional* thus using the previous
|
||||
behavior as default, i.e. the publish-subscribe is managed by the DB driver
|
||||
|
@ -106,8 +106,8 @@ interoperability with the proposed built-in publish-subscribe capability:
|
|||
+--------------------+ +-----------------------+ changes (callback)
|
||||
|
||||
|
||||
When the controller is brought up it first tries to sync all the relevant policy
|
||||
from the DB using the NB API which calls the specific loaded DB driver.
|
||||
When the controller is brought up it first tries to sync all the relevant
|
||||
policy from the DB using the NB API which calls the specific loaded DB driver.
|
||||
|
||||
When this process ends up, until now the controller had two options, it would
|
||||
ask the DB driver if it supports publish subscribe, if not the controller
|
||||
|
@ -116,23 +116,25 @@ continues with the sync loop and keeps polling for changes.
|
|||
If the DB driver supports publish-subscribe the controller calls a specific
|
||||
API to the driver to register a callback.
|
||||
The DB driver is responsible to update any DB change to the callback,
|
||||
The callback gets table name, action ('create', 'set', 'delete'), the key and value
|
||||
as parameters.
|
||||
The callback gets table name, action ('create', 'set', 'delete'), the key and
|
||||
value as parameters.
|
||||
|
||||
If the user configured pub-sub, the controller instead of calling the DB driver
|
||||
API calls Dragonflow pub-sub module with the callback prior to starting the full sync process.
|
||||
API calls Dragonflow pub-sub module with the callback prior to starting the
|
||||
full sync process.
|
||||
|
||||
The pub-sub module is in-charge of dispatching configuration changes (DB changes)
|
||||
to all the local controllers, it exposes a simple API of "subscriber" or "publisher".
|
||||
The pub-sub module is in-charge of dispatching configuration changes (DB
|
||||
changes) to all the local controllers, it exposes a simple API of "subscriber"
|
||||
or "publisher".
|
||||
|
||||
The pub-sub module will be pluggable and allow different drivers to be developed as
|
||||
the pubsub module driver that will implement the interface of the pubsub api
|
||||
referenced below [2].
|
||||
The pub-sub module will be pluggable and allow different drivers to be
|
||||
developed as the pubsub module driver that will implement the interface of the
|
||||
pubsub api referenced below [2].
|
||||
|
||||
Publisher
|
||||
---------
|
||||
The NB API class is used both by the Dragonflow local controller and the Neutron
|
||||
plugin.
|
||||
The NB API class is used both by the Dragonflow local controller and the
|
||||
Neutron plugin.
|
||||
|
||||
The following diagram shows how DB changes are dispatched to the pub-sub
|
||||
module and published to all local controllers.
|
||||
|
@ -161,11 +163,12 @@ module and published to all local controllers.
|
|||
Neutron Server q-svc process is forked on a multi-core host, in order to work
|
||||
around Python cooperative threading.
|
||||
|
||||
For PubSub solutions that are "bind based" e.g "tcp" (meaning one publisher per host)
|
||||
we will use an IPC mechanism provided by the Publisher driver, in order
|
||||
For PubSub solutions that are "bind based" e.g "tcp" (meaning one publisher per
|
||||
host) we will use an IPC mechanism provided by the Publisher driver, in order
|
||||
to push its events through a shared socket.
|
||||
|
||||
*Publisher Service* diagram below, which binds to a one-per-host publisher socket.
|
||||
*Publisher Service* diagram below, which binds to a one-per-host publisher
|
||||
socket.
|
||||
|
||||
::
|
||||
|
||||
|
@ -261,106 +264,131 @@ an applicative method to track message order and verify delivery.
|
|||
|
||||
Delivery
|
||||
--------
|
||||
Each publisher on startup selects a GUID and publish it to all the subscribers via the
|
||||
hello message descend below.
|
||||
Each publisher on startup selects a GUID and publish it to all the subscribers
|
||||
via the hello message descend below.
|
||||
|
||||
Subscribers will store in memory the publisher UUID on receiving the hello message and its current message ID.
|
||||
Subscribers will store in memory the publisher UUID on receiving the hello
|
||||
message and its current message ID.
|
||||
|
||||
In order to detect message delay/loss, we introduce a *per-pub-per-message* sequence ID.
|
||||
The client verifies the sequence order of messages by tracking *current per-pub-message-id*.
|
||||
In order to detect message delay/loss, we introduce a *per-pub-per-message*
|
||||
sequence ID.
|
||||
The client verifies the sequence order of messages by tracking
|
||||
*current per-pub-message-id*.
|
||||
|
||||
In case the client detects sequence that is >2 IDs from the *current*, it will wait
|
||||
for a period defined by *message delay window* for the missing messages to arrive.
|
||||
In case the client detects sequence that is >2 IDs from the *current*, it will
|
||||
wait for a period defined by *message delay window* for the missing messages to
|
||||
arrive.
|
||||
|
||||
If the time elapsed and some messages did not arrive, the client will perform a full sync against the DB.
|
||||
If the time elapsed and some messages did not arrive, the client will perform a
|
||||
full sync against the DB.
|
||||
|
||||
In case that subscriber receives an hello message from a registered publisher with different
|
||||
sequence number the subscriber will perform a full sync.
|
||||
In case that subscriber receives an hello message from a registered publisher
|
||||
with different sequence number the subscriber will perform a full sync.
|
||||
|
||||
|
||||
Flow 1: Subscriber (re)connects
|
||||
-------------------------------
|
||||
|
||||
When a subscriber connects for the first time, or reconnects after an outage, it will do full-resync.
|
||||
When a subscriber connects for the first time, or reconnects after an outage,
|
||||
it will do full-resync.
|
||||
|
||||
|
||||
Flow 2: Publisher (re)connects
|
||||
------------------------------
|
||||
|
||||
When a publisher connects for the first time, or reconnects after an outage,
|
||||
it will publish its initial sequence number and its UUID, in a special *hello* message.
|
||||
it will publish its initial sequence number and its UUID, in a special *hello*
|
||||
message.
|
||||
|
||||
The subscribers will receive this message and reset their *current per-sub-per-message-id* accordingly.
|
||||
The subscribers will receive this message and reset their
|
||||
*current per-sub-per-message-id* accordingly.
|
||||
|
||||
The publisher UUID and the sequence message id will be sent in an envelope in every published message
|
||||
The publisher UUID and the sequence message id will be sent in an envelope in
|
||||
every published message
|
||||
|
||||
Flow 3: Subscriber missed a message in a mostly-idle system
|
||||
-----------------------------------------------------------
|
||||
|
||||
When the system is mostly idle, a subscriber may miss a message and not detect it for a long time.
|
||||
When the system is mostly idle, a subscriber may miss a message and not detect
|
||||
it for a long time.
|
||||
|
||||
In order to mitigate this, the publisher will emit its *hello* message every configurable *max_idle_time*.
|
||||
In order to mitigate this, the publisher will emit its *hello* message every
|
||||
configurable *max_idle_time*.
|
||||
|
||||
We define Idle Time as a period of time where no messages are published from a specific publisher.
|
||||
We define Idle Time as a period of time where no messages are published from a
|
||||
specific publisher.
|
||||
|
||||
Order
|
||||
-----
|
||||
|
||||
We introduce *versioning* on the object level in the database, in order to track message order.
|
||||
We introduce *versioning* on the object level in the database, in order to
|
||||
track message order.
|
||||
|
||||
We compare this versioning to the local cache, before we update it.
|
||||
|
||||
We only update when local cache version is older, and drop updates that have older version than the local cache.
|
||||
Local cache will be updated with any newer version head even if it is few versions ahead, older
|
||||
version will be dropped.
|
||||
We only update when local cache version is older, and drop updates that have
|
||||
older version than the local cache.
|
||||
Local cache will be updated with any newer version head even if it is few
|
||||
versions ahead, older version will be dropped.
|
||||
|
||||
Neutron Server Publisher discovery
|
||||
==================================
|
||||
|
||||
Each subscriber (i.e. Distributed Dragonflow Controller) uses a local configuration with the
|
||||
addresses of the publishers.
|
||||
Each subscriber (i.e. Distributed Dragonflow Controller) uses a local
|
||||
configuration with the addresses of the publishers.
|
||||
|
||||
We will optimize this by adding a Service Directory in the Dragonflow database.
|
||||
Each publisher on startup will register itself into this discovery table with a timestamp
|
||||
and will renew its lease every x minutes
|
||||
Each publisher on startup will register itself into this discovery table with a
|
||||
timestamp and will renew its lease every x minutes
|
||||
|
||||
A Discovery table garbage collector will remove publisher with out a valid lease
|
||||
A Discovery table garbage collector will remove publisher with out a valid
|
||||
lease
|
||||
|
||||
Controller-to-Controller Publisher Proposed Solution
|
||||
====================================================
|
||||
|
||||
Local Controllers publish messages through the Neutron Server, by writing to the Dragonflow database.
|
||||
Local Controllers publish messages through the Neutron Server, by writing to
|
||||
the Dragonflow database.
|
||||
|
||||
A polling mechanism in the Dragonflow publisher service detects such updates and publish them to everyone.
|
||||
A polling mechanism in the Dragonflow publisher service detects such updates
|
||||
and publish them to everyone.
|
||||
|
||||
This mechanism is enough for handling rarely-occurring events, such as chassis registration
|
||||
(i.e. adding new compute nodes).
|
||||
This mechanism is enough for handling rarely-occurring events, such as chassis
|
||||
registration (i.e. adding new compute nodes).
|
||||
|
||||
TODO: if we will see significant increase in Controller-to-Controller publishing traffic, we will
|
||||
implement an enhancement to
|
||||
enable multi-publisher-multi-subscriber mechanism, using something like ZMQ EPGM.
|
||||
TODO: if we will see significant increase in Controller-to-Controller
|
||||
publishing traffic, we will implement an enhancement to
|
||||
enable multi-publisher-multi-subscriber mechanism, using something like ZMQ
|
||||
EPGM.
|
||||
|
||||
Configuration Options
|
||||
=====================
|
||||
|
||||
'enable_df_pub_sub', default=False, help=_("Enable use of Dragonflow built-in pub/sub")),
|
||||
'enable_df_pub_sub', default=False, help=_("Enable use of Dragonflow built-in
|
||||
pub/sub")),
|
||||
|
||||
'pub_sub_driver', default='zmq_pubsub_driver', help=_('Drivers to use for the Dragonflow pub/sub')),
|
||||
'pub_sub_driver', default='zmq_pubsub_driver', help=_('Drivers to use for the
|
||||
Dragonflow pub/sub')),
|
||||
|
||||
'publishers_ips', default=['$local_ip'], help=_('List of the Neutron Server Publisher IPs.')),
|
||||
'publishers_ips', default=['$local_ip'], help=_('List of the Neutron Server
|
||||
Publisher IPs.')),
|
||||
|
||||
'publisher_port', default=8866, help=_('Neutron Server Publishers Port'))
|
||||
|
||||
'pub_sub_use_multiproc', default=True, help=_('Use inter-process publish/subscribe. '
|
||||
'Publishers send events via the publisher service.')
|
||||
'pub_sub_use_multiproc', default=True, help=_('Use inter-process
|
||||
publish/subscribe.' 'Publishers send events via the publisher service.')
|
||||
|
||||
'publisher_transport', default='tcp', help=_('Neutron Server Publishers transport protocol')),
|
||||
'publisher_transport', default='tcp', help=_('Neutron Server Publishers
|
||||
transport protocol')),
|
||||
|
||||
'publisher_bind_address', default='*', help=_('Neutron Server Publishers bind address')),
|
||||
'publisher_bind_address', default='*', help=_('Neutron Server Publishers bind
|
||||
address')),
|
||||
|
||||
'pub_sub_multiproc_driver', default='zmq_pubsub_multiproc_driver', help=_('Drivers to use for the Dragonflow pub/sub')),
|
||||
'pub_sub_multiproc_driver', default='zmq_pubsub_multiproc_driver',
|
||||
help=_('Drivers to use for the Dragonflow pub/sub')),
|
||||
|
||||
'publisher_multiproc_socket', default='/var/run/zmq_pubsub/zmq-publisher-socket',
|
||||
'publisher_multiproc_socket',
|
||||
default='/var/run/zmq_pubsub/zmq-publisher-socket',
|
||||
help=_('Neutron Server Publisher inter-process socket address')),
|
||||
|
||||
References
|
||||
|
|
|
@ -30,7 +30,8 @@ Moreover this superficial problem will lead to the rest of the mess.
|
|||
to be the generic one. The gap is that it requires df-local-controller run
|
||||
publisher and neutron-server run subscriber.
|
||||
|
||||
* The active-port-detection app also requires df-local-controller run publisher.
|
||||
* The active-port-detection app also requires df-local-controller run
|
||||
publisher.
|
||||
|
||||
* We set up publisher port in the common [df] section in the configuration.
|
||||
When we run both df-local-controller and df-publisher-service, they read
|
||||
|
@ -122,8 +123,9 @@ Publisher Subscriber Impact
|
|||
Other Impact
|
||||
------------
|
||||
|
||||
* Make api_nb module singleton in df-local-controller. We don't need to initialize
|
||||
it each time, for instance, in port-status-driver, just passing its reference.
|
||||
* Make api_nb module singleton in df-local-controller. We don't need to
|
||||
initialize it each time, for instance, in port-status-driver, just passing
|
||||
its reference.
|
||||
|
||||
* Make redis-port-status driver work for all the db drivers.
|
||||
|
||||
|
|
|
@ -54,7 +54,8 @@ to all Dragonflow controllers to check if the DB cluster nodes changed.
|
|||
And controllers should subscribe a "HA" topic to receive messages from
|
||||
plugin.
|
||||
In Dragonflow controller, it never read nodes information from Redis cluster
|
||||
after initialization but only listen the messages from detecting task from plugin.
|
||||
after initialization but only listen the messages from detecting task from
|
||||
plugin.
|
||||
|
||||
There are 2 types of connections between Redis client and cluster:
|
||||
1. read/write connection, client connects to every Redis master nodes.
|
||||
|
|
|
@ -14,11 +14,14 @@ Add a DataBase support for DragonFlow
|
|||
|
||||
Problem Description
|
||||
===================
|
||||
Dragonflow will use Publish/Subscribe as a method to realize communication between components.
|
||||
Dragonflow will use Publish/Subscribe as a method to realize communication
|
||||
between components.
|
||||
Redis DB has a good performance of Publish/Subscribe.
|
||||
So,We need a driver for Redis DB,meanwhile,
|
||||
if other DB which support Publish/Subscribe only needs overwrite new APIS added here.
|
||||
dragonflow will have a efficient way to realize communication between different component.
|
||||
if other DB which support Publish/Subscribe only needs overwrite new APIS added
|
||||
here.
|
||||
dragonflow will have a efficient way to realize communication between different
|
||||
component.
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
@ -31,8 +34,8 @@ Populating a Database API
|
|||
-------------------------
|
||||
Basic operation for Redis DB,including add/delete/get/modify and so on.
|
||||
Realization is based on Grokzen lib.
|
||||
The following diagram shows which components will populate Redis DB Cluster with
|
||||
driver[4].::
|
||||
The following diagram shows which components will populate Redis DB Cluster
|
||||
with driver[4].::
|
||||
|
||||
+------------------+ +----------------+
|
||||
|Neutron server | | Redis Driver |
|
||||
|
@ -56,7 +59,8 @@ driver[4].::
|
|||
Publish API
|
||||
-----------
|
||||
The new API realizes publish function with channel, based on andymccurdy lib
|
||||
The following diagram shows how Neutron config changes are published to all local controllers.
|
||||
The following diagram shows how Neutron config changes are published to all
|
||||
local controllers.
|
||||
It is only a example.::
|
||||
|
||||
+---------------+
|
||||
|
@ -86,14 +90,15 @@ Main process of realization:
|
|||
|
||||
Special Notice:
|
||||
'Some data' will be coded into json pattern.
|
||||
Above example ,subscriber which subscribes the channel'my-first-channel'will receive what
|
||||
is published.
|
||||
Above example ,subscriber which subscribes the channel 'my-first-channel' will
|
||||
receive what is published.
|
||||
More details please refer Publish / Subscribe section of [1]
|
||||
|
||||
Subscribe API
|
||||
-------------
|
||||
If you want to receive message that you publish,you first should do a subscription,if you
|
||||
do not want to receive message,you should withdraw subscription.
|
||||
If you want to receive message that you publish,you first should do a
|
||||
subscription, if you do not want to receive message,you should withdraw
|
||||
subscription.
|
||||
Realization is based on andymccurdy lib.
|
||||
|
||||
Here is a example of subscription process:
|
||||
|
@ -112,52 +117,56 @@ Here is an example of message driver may received:
|
|||
|
||||
{'channel': 'my-first-channel', 'data': 'some data', 'pattern': None, 'type': 'message'}
|
||||
|
||||
type: One of the following: 'subscribe', 'unsubscribe', 'psubscribe', 'punsubscribe',
|
||||
'message', 'pmessage'.
|
||||
type: One of the following: 'subscribe', 'unsubscribe', 'psubscribe',
|
||||
'punsubscribe', 'message', 'pmessage'.
|
||||
|
||||
channel: The channel [un]subscribed to or the channel a message was published to.
|
||||
channel: The channel [un]subscribed to or the channel a message was published
|
||||
to.
|
||||
|
||||
pattern: The pattern that matched a published message's channel.
|
||||
Will be None in all cases except for 'pmessage' types.
|
||||
data:
|
||||
The message data. With [un]subscribe messages,
|
||||
this value will be the number of channels and patterns the connection is currently subscribed to.
|
||||
this value will be the number of channels and patterns the connection is
|
||||
currently subscribed to.
|
||||
With [p]message messages, this value will be the actual published message.
|
||||
|
||||
Special Notice:
|
||||
This message is only processed by driver.
|
||||
Message data will be decoded by driver and send into queue.
|
||||
Psubscribe,Punsubscribe and Pmessage are not used here,they are used for pattern match.[1]
|
||||
Psubscribe,Punsubscribe and Pmessage are not used here,they are used for
|
||||
pattern match.[1]
|
||||
|
||||
Subscribe Thread For Reading Messages
|
||||
-------------------------------------
|
||||
The subscribe thread is in charge of receiving the notifications and sending
|
||||
them back to the controller. Realization is based on andymccurdy lib.
|
||||
|
||||
The subscribe thread loop is depicted in the following diagram::
|
||||
The subscribe thread loop is depicted in the following diagram:
|
||||
|
||||
::
|
||||
|
||||
+-----------------+ +---------------+
|
||||
| | | |
|
||||
| | | Process |
|
||||
| | +-----------------+fun call | Function1 |
|
||||
| | | +-------->| |
|
||||
|Subscribe Thread | | Message Dispatch| +---------------+
|
||||
| | | |
|
||||
| Wait For Message| | |
|
||||
| | | Read Message | +---------------+
|
||||
| | Send into Queue | From Queue |fun call | Process |
|
||||
| New Message +-----------------------> +-------->| Function2 |
|
||||
| | | Dispatch Message| | |
|
||||
| | | | +---------------+
|
||||
| | | |
|
||||
| | | |
|
||||
| | | | +---------------+
|
||||
| | | |fun call | Process |
|
||||
| | | +---------> Function3 |
|
||||
| | | | | |
|
||||
+-----------------+ +-----------------+ | |
|
||||
+---------------+
|
||||
+-----------------+ +---------------+
|
||||
| | | |
|
||||
| | | Process |
|
||||
| | +-----------------+fun call | Function1 |
|
||||
| | | +-------->| |
|
||||
|Subscribe Thread | | Message Dispatch| +---------------+
|
||||
| | | |
|
||||
| Wait For Message| | |
|
||||
| | | Read Message | +---------------+
|
||||
| | Send into Queue | From Queue |fun call | Process |
|
||||
| New Message +-----------------------> +-------->| Function2 |
|
||||
| | | Dispatch Message| | |
|
||||
| | | | +---------------+
|
||||
| | | |
|
||||
| | | |
|
||||
| | | | +---------------+
|
||||
| | | |fun call | Process |
|
||||
| | | +---------> Function3 |
|
||||
| | | | | |
|
||||
+-----------------+ +-----------------+ | |
|
||||
+---------------+
|
||||
|
||||
Realization Example:
|
||||
|
||||
|
@ -169,54 +178,60 @@ Realization Example:
|
|||
|
||||
Special Notice:
|
||||
Not only three Process Functions.
|
||||
Driver Subscriber thread is only one thread to do message dispatch according to channel.
|
||||
Driver Subscriber thread is only one thread to do message dispatch according to
|
||||
channel.
|
||||
listen() is a generator that blocks until a message is available.
|
||||
|
||||
|
||||
Subscriber management
|
||||
---------------------
|
||||
This resubscription should be done only when connection to DB server is recovered.
|
||||
This resubscription should be done only when connection to DB server is
|
||||
recovered.
|
||||
|
||||
driver only does connection fix,throw exception when connection is recovered,
|
||||
driver will clear all subscription and user of Subscription do resubscribe.
|
||||
|
||||
Connection Setup
|
||||
----------------
|
||||
When driver is initialized,it will connect to all db nodes for read/write/get/modify operation.
|
||||
When driver is initialized,it will connect to all db nodes for
|
||||
read/write/get/modify operation.
|
||||
But for pub/sub, driver will connect to one db node for one pub or one sub.
|
||||
Driver guarantee connections for pub/sub will be scattered among db nodes.
|
||||
|
||||
|
||||
Exception
|
||||
---------
|
||||
First Notice:exception of cluster client and single client are different, need processed separately.
|
||||
First Notice:exception of cluster client and single client are different, need
|
||||
processed separately.
|
||||
case1:populate db failed
|
||||
If add operation is failed, driver will delete what you add,
|
||||
driver will check connection and reconnect if reason is connection lost,
|
||||
driver will try several times( for example 3), if all trials failed,
|
||||
driver will return failed, if reason is not connection
|
||||
problem, driver will also return failed directly. You should return failed to up level,
|
||||
do not publish, if driver returned failed.
|
||||
problem, driver will also return failed directly. You should return failed to
|
||||
up level, do not publish, if driver returned failed.
|
||||
|
||||
If delete operation is failed, the process is same as above,
|
||||
except for driver will not rollback delete operation.
|
||||
|
||||
case2:publish failed
|
||||
If this happened,
|
||||
driver will return failed and check connection also reconnect if reason is connection lost.
|
||||
If driver return failed, user of API should undo what you done before publish and return failed
|
||||
to up level
|
||||
driver will return failed and check connection also reconnect if reason is
|
||||
connection lost.
|
||||
If driver return failed, user of API should undo what you done before publish
|
||||
and return failed to up level
|
||||
|
||||
case3:subscribe failed
|
||||
If this happened,
|
||||
driver will return failed and check connection also reconnect if reason is connection lost.
|
||||
driver will return failed and check connection also reconnect if reason is
|
||||
connection lost.
|
||||
If driver return failed, user of api return failed to up level.
|
||||
|
||||
case4:subscribe listen exception
|
||||
If this happened,
|
||||
Driver will clear all subscription and then try reconnect,
|
||||
after fix connection then send a message to subscriber, tell that you subscribed is recovered,
|
||||
subscriber should subscribe again.
|
||||
after fix connection then send a message to subscriber, tell that you
|
||||
subscribed is recovered, subscriber should subscribe again.
|
||||
|
||||
References
|
||||
==========
|
||||
|
|
|
@ -31,7 +31,7 @@ Dragonflow.
|
|||
|
||||
Data path performance
|
||||
---------------------
|
||||
Its important to note that Security groups are stateful — responses to allowed
|
||||
Its important to note that Security groups are stateful - responses to allowed
|
||||
ingress traffic are allowed to flow out regardless of egress rules, and vice
|
||||
versa. Current Neutron implementation adds a linux bridge in the path between
|
||||
each port (VM) and OVS bridge.
|
||||
|
|
|
@ -17,18 +17,20 @@ topology only to Dragonflow local controllers that need it.
|
|||
Problem Description
|
||||
===================
|
||||
Currently, Dragonflow local controllers cache all the topology, such as all the
|
||||
networks, ports and routers etc. In fact, one compute node only has dozens of VMs.
|
||||
networks, ports and routers etc. In fact, one compute node only has dozens of
|
||||
VMs.
|
||||
Topology used by these VMs is merely a tiny proportion of the whole data center
|
||||
networking topology. Most of the topology cached by Dragonflow local controllers
|
||||
will never be used.
|
||||
networking topology. Most of the topology cached by Dragonflow local
|
||||
controllers will never be used.
|
||||
|
||||
Moreover, in order to keep all the cached topology up to date, local controllers
|
||||
have to repeatedly communicate with the Dragonflow database to refresh the data.
|
||||
With the increase of compute nodes, communication of this type will also increase
|
||||
correspondingly. For Dragonflow local controllers, this method will cause high
|
||||
CPU and memory occupation rate. For the Dragonflow database, it's more intolerable,
|
||||
for there will be too many update requests from tens of thousands compute nodes
|
||||
to process.
|
||||
Moreover, in order to keep all the cached topology up to date, local
|
||||
controllers have to repeatedly communicate with the Dragonflow database to
|
||||
refresh the data.
|
||||
With the increase of compute nodes, communication of this type will also
|
||||
increase correspondingly. For Dragonflow local controllers, this method will
|
||||
cause high CPU and memory occupation rate. For the Dragonflow database, it's
|
||||
more intolerable, for there will be too many update requests from tens of
|
||||
thousands compute nodes to process.
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
@ -38,37 +40,39 @@ basic idea
|
|||
|
||||
The idea is quite simple:
|
||||
|
||||
* Each Dragonflow local controller only subscribes topology it's interested in
|
||||
from the sub-pub server.
|
||||
* Each Dragonflow local controller only subscribes topology it's interested
|
||||
in from the pub-sub server.
|
||||
|
||||
* When northbound topology changed, in addition to save it to the Dragonflow
|
||||
database, Neutron plugin also publish the change to the sub-pub server.
|
||||
database, Neutron plugin also publish the change to the pub-sub server.
|
||||
|
||||
* When southbound topology changed, in addition to save it to the Dragonflow
|
||||
database, Dragonflow local controllers also publish the change to the sub-pub
|
||||
server.
|
||||
database, Dragonflow local controllers also publish the change to the
|
||||
pub-sub server.
|
||||
|
||||
* On receiving a publish request from Neutron plugin or Dragonflow local controller,
|
||||
the sub-pub server publishes the change to whom subscribe the change.
|
||||
* On receiving a publish request from Neutron plugin or Dragonflow local
|
||||
controller, the pub-sub server publishes the change to whom subscribe the
|
||||
change.
|
||||
|
||||
* When receives a published event, Dragonflow local controller updates its local
|
||||
cache and flow entries if needed.
|
||||
* When receives a published event, Dragonflow local controller updates its
|
||||
local cache and flow entries if needed.
|
||||
|
||||
* When receives a published event, Neutron plugin updates it's status if needed.
|
||||
* When receives a published event, Neutron plugin updates it's status if
|
||||
needed.
|
||||
|
||||
Publisher subscriber pattern
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Selective topology distribution depends on the sub-pub function which is shown in
|
||||
the following diagram. When there is a topology change, for example, a new port
|
||||
is created by a tenant, a publisher (in this case, the Dragonflow Neutron plugin) will
|
||||
publish this event together with detailed info of the new port to a topic. Every
|
||||
topic maintains a list of subscribers. On receiving an event from publisher, the topic
|
||||
then sends the event to every subscriber in the list. For the port creating example,
|
||||
there may be many Dragonflow local controllers which have ports connecting to the
|
||||
same network with the new port. These local controllers care about changes of ports
|
||||
in the network and will subscribe change of this network by registering to the
|
||||
corresponding topic. These controllers will get notified by the topic when this
|
||||
new port is created.
|
||||
Selective topology distribution depends on the pub-sub function which is shown
|
||||
in the following diagram. When there is a topology change, for example, a new
|
||||
port is created by a tenant, a publisher (in this case, the Dragonflow Neutron
|
||||
plugin) will publish this event together with detailed info of the new port to
|
||||
a topic. Every topic maintains a list of subscribers. On receiving an event
|
||||
from publisher, the topic then sends the event to every subscriber in the list.
|
||||
For the port creating example, there may be many Dragonflow local controllers
|
||||
which have ports connecting to the same network with the new port. These local
|
||||
controllers care about changes of ports in the network and will subscribe
|
||||
change of this network by registering to the corresponding topic. These
|
||||
controllers will get notified by the topic when this new port is created.
|
||||
|
||||
::
|
||||
|
||||
|
@ -92,9 +96,9 @@ Two ways to distribute topology selectively
|
|||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
There are many types of topology change. In large scale data center, number of
|
||||
tenants is also considerable. There will be tons of topics if publish different
|
||||
events to different topics. To simplify the implementation, some degree of convergence
|
||||
has to be taken into account. There are two ways to converge the types of topology
|
||||
change.
|
||||
events to different topics. To simplify the implementation, some degree of
|
||||
convergence has to be taken into account. There are two ways to converge the
|
||||
types of topology change.
|
||||
|
||||
One topic per tenant
|
||||
""""""""""""""""""""
|
||||
|
@ -109,24 +113,25 @@ Easier for Subscribers to decide which topic to subscribe.
|
|||
|
||||
**Cons**
|
||||
|
||||
Subscriber may receive redundant event and store additional topology it will never
|
||||
use.
|
||||
Subscriber may receive redundant event and store additional topology it will
|
||||
never use.
|
||||
|
||||
One topic per vpc or router
|
||||
"""""""""""""""""""""""""""
|
||||
Every vpc or router has its own topic. For isolated networks which don't connect
|
||||
to any routers, they also have their own topics. Changes in topology are published
|
||||
to the corresponding topics.
|
||||
Every vpc or router has its own topic. For isolated networks which don't
|
||||
connect to any routers, they also have their own topics. Changes in topology
|
||||
are published to the corresponding topics.
|
||||
|
||||
**Pros**
|
||||
|
||||
Finer grained. When a tenant has many vpc or routers or isolated networks, topology
|
||||
changes of different vpc, routers or networks will not affect each other.
|
||||
Finer grained. When a tenant has many vpc or routers or isolated networks,
|
||||
topology changes of different vpc, routers or networks will not affect each
|
||||
other.
|
||||
|
||||
**Cons**
|
||||
|
||||
Harder for subscribers and publishers to decide which topic they should subscribe
|
||||
or publish.
|
||||
Harder for subscribers and publishers to decide which topic they should
|
||||
subscribe or publish.
|
||||
|
||||
Here, I will only discuss the first way for simplicity.
|
||||
|
||||
|
@ -139,13 +144,14 @@ Northbound Topology Change
|
|||
When a tenant named tenant1 create a port through Neutron's northbound api.
|
||||
|
||||
* Dragonflow Neutron plugin will publish a event to the tenant's topic in the
|
||||
sub-pub server.
|
||||
pub-sub server.
|
||||
|
||||
* The sub-pub server will then check who have subscribed the topic and publish
|
||||
* The pub-sub server will then check who have subscribed the topic and publish
|
||||
the event to them.
|
||||
|
||||
* On receiving the event, local controllers will save the new port's information
|
||||
and install some flow entries on OVS which is not covered in this spec.
|
||||
* On receiving the event, local controllers will save the new port's
|
||||
information and install some flow entries on OVS which is not covered in this
|
||||
spec.
|
||||
|
||||
::
|
||||
|
||||
|
@ -163,7 +169,8 @@ When a tenant named tenant1 create a port through Neutron's northbound api.
|
|||
| | | |
|
||||
+ + + +
|
||||
|
||||
In the above diagram, Dragonflow local controller 2 has no VMs belong to tenant1.
|
||||
In the above diagram, Dragonflow local controller 2 has no VMs belong to
|
||||
tenant1.
|
||||
It will not subscribe tenant1's topic and therefore will not get notified.
|
||||
|
||||
Processing of other northbound topology changes, such as creating, deleting or
|
||||
|
@ -172,24 +179,25 @@ modifying router, network and port is same as the above example.
|
|||
Southbound Topology Change
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
When nova starts a VM in a compute node, it will insert a port on the corresponding
|
||||
OVS bridge.
|
||||
When nova starts a VM in a compute node, it will insert a port on the
|
||||
corresponding OVS bridge.
|
||||
|
||||
* By monitoring OVSDB, Dragonflow local controller get notified when the new port
|
||||
is added on OVS bridge.
|
||||
* By monitoring OVSDB, Dragonflow local controller get notified when the new
|
||||
port is added on OVS bridge.
|
||||
|
||||
* Dragonflow local controller queries the port's topology from Dragonflow database
|
||||
and knows which tenant the port belongs to.
|
||||
* Dragonflow local controller queries the port's topology from Dragonflow
|
||||
database and knows which tenant the port belongs to.
|
||||
|
||||
* Dragonflow local controller queries local cache to see if it has subscribed the
|
||||
tenant topic.
|
||||
* Dragonflow local controller queries local cache to see if it has subscribed
|
||||
the tenant topic.
|
||||
|
||||
+ If local controller has already subscribed the tenant's topic. This means there
|
||||
already are local ports of the same tenant. It will not subscribe the topic again.
|
||||
+ If local controller has already subscribed the tenant's topic. This means
|
||||
there already are local ports of the same tenant. It will not subscribe the
|
||||
topic again.
|
||||
|
||||
+ If local controller hasn't subscribed the tenant's topic. This means the new
|
||||
port is the only local port in the compute node belongs to the tenant. Local
|
||||
controller will subscribe the tenant's topic.
|
||||
+ If local controller hasn't subscribed the tenant's topic. This means the
|
||||
new port is the only local port in the compute node belongs to the tenant.
|
||||
Local controller will subscribe the tenant's topic.
|
||||
|
||||
::
|
||||
|
||||
|
@ -214,19 +222,21 @@ OVS bridge.
|
|||
|
||||
|
||||
If nova remove a port from OVS bridge, local controller will check if it's the
|
||||
tenant's last port on the compute node. If it is, local controller will unsubscribe
|
||||
the tenant's topic and will not receive any further event of the tenant's topology
|
||||
changes.
|
||||
tenant's last port on the compute node. If it is, local controller will
|
||||
unsubscribe he tenant's topic and will not receive any further event of the
|
||||
tenant's topology changes.
|
||||
|
||||
Dragonflow Local Controller Startup
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
On startup, local controller will get all ports being attached to OVS bridge by
|
||||
querying OVSDB. Once getting all these local ports, local controller will query
|
||||
ports' topology from Dragonflow database and subscribe the corresponding topics of
|
||||
the ports. This is done for every local port, as described in the previous section.
|
||||
ports' topology from Dragonflow database and subscribe the corresponding topics
|
||||
of the ports. This is done for every local port, as described in the previous
|
||||
section.
|
||||
|
||||
Dragonflow Local Controller Offline
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
If one local controller exit, for example, killed by administrator for maintenance,
|
||||
connection to the sub-pub server will lose. It's the sub-pub server's responsibility
|
||||
to remove the local controller from all topics it has subscribed.
|
||||
If one local controller exit, for example, killed by administrator for
|
||||
maintenance, connection to the pub-sub server will lose. It's the pub-sub
|
||||
server's responsibility to remove the local controller from all topics it has
|
||||
subscribed.
|
||||
|
|
|
@ -401,8 +401,8 @@ Tests
|
|||
#. Traversing the SFC - given an SFC and SF layout, we can check that our
|
||||
packet takes a logical route and visits all SFs in a logical order.
|
||||
|
||||
#. Graphs - re-classification occurs only between SFCs that are part of the same
|
||||
graph
|
||||
#. Graphs - re-classification occurs only between SFCs that are part of the
|
||||
same graph
|
||||
|
||||
|
||||
Work Items
|
||||
|
|
|
@ -74,8 +74,8 @@ Database.
|
|||
NB Data Model Impact
|
||||
--------------------
|
||||
|
||||
A service table will be added in Dragonflow NB database, which contains following
|
||||
information regarding each service.
|
||||
A service table will be added in Dragonflow NB database, which contains
|
||||
following information regarding each service.
|
||||
|
||||
::
|
||||
|
||||
|
@ -155,15 +155,16 @@ Plan is to store information on following bassis, if possible.
|
|||
-
|
||||
}
|
||||
|
||||
The assumption for the above data management is, there can be only one instance of
|
||||
a service on a node that has to be registered.
|
||||
The assumption for the above data management is, there can be only one instance
|
||||
of a service on a node that has to be registered.
|
||||
|
||||
It does not add any overhead during status reporting, services has to report their
|
||||
binary and host. And updation of service status can be done easily in constant time.
|
||||
It does not add any overhead during status reporting, services has to report
|
||||
their binary and host. And updation of service status can be done easily in
|
||||
constant time.
|
||||
|
||||
It provides benefit for queries asking for example "list all the host running
|
||||
publishers." or "list all the publishers in the cluster". These queries seems to be
|
||||
more frequent as load has to be balanced between services etc.
|
||||
publishers." or "list all the publishers in the cluster". These queries seems
|
||||
to be more frequent as load has to be balanced between services etc.
|
||||
|
||||
Publisher Subscriber Impact
|
||||
---------------------------
|
||||
|
|
|
@ -159,15 +159,16 @@ to external CN:
|
|||
|
||||
filter: tun_id=DEST_TUN_ID action:output:OVERLAY_NET_PORT
|
||||
|
||||
+----------------------+---------------------------------------------------------+
|
||||
| Field Name | Description |
|
||||
+======================+=========================================================+
|
||||
| ``DEST_TUN_ID`` | a tunnel number will specify a destination VM |
|
||||
+----------------------+---------------------------------------------------------+
|
||||
| ``DEST_LOCAL_PORT`` | destination OVS port number (in case it is on same CN) |
|
||||
+----------------------+---------------------------------------------------------+
|
||||
| ``OVERLAY_NET_PORT`` | packet will be forwarded to other CN |
|
||||
+----------------------+---------------------------------------------------------+
|
||||
+----------------------+----------------------------------------------------+
|
||||
| Field Name | Description |
|
||||
+======================+====================================================+
|
||||
| ``DEST_TUN_ID`` | a tunnel number will specify a destination VM |
|
||||
+----------------------+----------------------------------------------------+
|
||||
| ``DEST_LOCAL_PORT`` | destination OVS port number (in case it is on same|
|
||||
| | CN) |
|
||||
+----------------------+----------------------------------------------------+
|
||||
| ``OVERLAY_NET_PORT`` | packet will be forwarded to other CN |
|
||||
+----------------------+----------------------------------------------------+
|
||||
|
||||
Assigning tunnel id for each TapService
|
||||
---------------------------------------
|
||||
|
@ -175,14 +176,14 @@ Assigning tunnel id for each TapService
|
|||
Each TapService will have a unique id that corresponds to the overlay network
|
||||
tunnel id.
|
||||
|
||||
By default, each network has it's own id called segment id allocated from neutron
|
||||
segment pool. A naive approach will be to assign a unique id to be used for
|
||||
TapService from this pool but we concern that admins setup network vnis and they
|
||||
expects number of network to be supported.
|
||||
By default, each network has it's own id called segment id allocated from
|
||||
neutron segment pool. A naive approach will be to assign a unique id to be used
|
||||
for TapService from this pool but we concern that admins setup network vnis and
|
||||
they expects number of network to be supported.
|
||||
|
||||
More advance solution will be to create a new segment pool to be used exclusively
|
||||
for TapServices. This new network pool should not coincide with the one used in
|
||||
neutron.
|
||||
More advance solution will be to create a new segment pool to be used
|
||||
exclusively for TapServices. This new network pool should not coincide with the
|
||||
one used in neutron.
|
||||
|
||||
In case we run out of free ids in the new segment pool, as a fallback solution,
|
||||
we will assign segment id from the neutron segment pool.
|
||||
|
@ -220,8 +221,8 @@ Tap position is ``BEFORESG``
|
|||
This configuration mode actually implies that packets will be mirrored without
|
||||
filtering by security group.
|
||||
|
||||
Change ``table=1`` (``EGRESS_PORT_SECURITY_TABLE``) to be ``table=2`` and install our
|
||||
tap rules in ``table=1``.
|
||||
Change ``table=1`` (``EGRESS_PORT_SECURITY_TABLE``) to be ``table=2`` and
|
||||
install our tap rules in ``table=1``.
|
||||
|
||||
In new ``table=1`` we will add the following rules:
|
||||
|
||||
|
@ -270,7 +271,8 @@ Tap on the Input
|
|||
Tap position is ``AFTERSG``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
After passing the firewall, packets are forwarded to ``table=78`` (``INGRESS_DISPATCH_TABLE``).
|
||||
After passing the firewall, packets are forwarded to ``table=78``
|
||||
(``INGRESS_DISPATCH_TABLE``).
|
||||
|
||||
We should move all rules from ``table=78`` to a new table (e.g. ``table=79``).
|
||||
|
||||
|
|
|
@ -48,10 +48,10 @@ design at the same time as use cases, if desired. Note that by high-level,
|
|||
we mean the "view from orbit" rough cut at how things will happen.
|
||||
|
||||
This section should 'scope' the effort from a feature standpoint: how is the
|
||||
'dragonflow end-to-end system' going to look like after this change? What Dragonflow
|
||||
areas do you intend to touch and how do you intend to work on them? The list
|
||||
below is not meant to be a template to fill in, but rather a jumpstart on the
|
||||
sorts of areas to consider in your proposed change description.
|
||||
'dragonflow end-to-end system' going to look like after this change? What
|
||||
Dragonflow areas do you intend to touch and how do you intend to work on them?
|
||||
The list below is not meant to be a template to fill in, but rather a jumpstart
|
||||
on the sorts of areas to consider in your proposed change description.
|
||||
|
||||
You do not need to detail API or data model changes.
|
||||
|
||||
|
|
|
@ -19,7 +19,8 @@ Problem Description
|
|||
===================
|
||||
Tunneling is currently handled both by the df_local_controller and by the
|
||||
L2 application for both ingress and egress packets propagation.
|
||||
The tunneling related flows are set when either local or remote port is updated.
|
||||
The tunneling related flows are set when either local or remote port is
|
||||
updated.
|
||||
|
||||
Upon VM port creation, its properties such as its overlay network membership,
|
||||
ports database id is used to match flows to/from it. A tunnel port is created
|
||||
|
@ -54,8 +55,8 @@ Ingress Processing
|
|||
|
||||
The classification flow should match against the tunnel in_port, a port of a
|
||||
specific segmentation id and tunnel's virtual network key, according
|
||||
to which metadata will be set as network_id to identify incoming traffic being part
|
||||
of the network identified by the segmentation id.
|
||||
to which metadata will be set as network_id to identify incoming traffic being
|
||||
part of the network identified by the segmentation id.
|
||||
|
||||
An example classification flow, for two VMS, one is part of "admin-private"
|
||||
network, with segmentation id 0x13 and the second which is part of "private"
|
||||
|
@ -69,10 +70,10 @@ are:
|
|||
table=0, priority=100,in_port=4,tun_id=0x13 actions=load:0x3->OXM_OF_METADATA[],resubmit(,100)
|
||||
table=0, priority=100,in_port=4,tun_id=0x49 actions=load:0x1->OXM_OF_METADATA[],resubmit(,100)
|
||||
|
||||
Currently in the ingress port lookup table (100), a match is done according to unique
|
||||
network id and vm's port mac, where vm's port key is set into reg7, and sent to
|
||||
table (105), the ingress conn-track table which passes the packet to ingress
|
||||
dispatch table
|
||||
Currently in the ingress port lookup table (100), a match is done according to
|
||||
unique network id and vm's port mac, where vm's port key is set into reg7, and
|
||||
sent to table (105), the ingress conn-track table which passes the packet to
|
||||
ingress dispatch table
|
||||
|
||||
Set fields are:
|
||||
* reg7 <- Unique port key
|
||||
|
@ -85,7 +86,8 @@ Set fields are:
|
|||
table=100, priority=200,metadata=0x1,dl_dst=fa:16:3e:bc:5b:08 actions=load:0x6->NXM_NX_REG7[],resubmit(,105)
|
||||
|
||||
Eventually the processing reaches table (115), the ingress dispatch table where
|
||||
the packet is matched against vm's port's key, and forwarded to the local vm port.
|
||||
the packet is matched against vm's port's key, and forwarded to the local vm
|
||||
port.
|
||||
|
||||
::
|
||||
|
||||
|
@ -102,15 +104,15 @@ and metadata ovs registers, respectively.
|
|||
table=0, priority=100,in_port=8 actions=load:0xa->NXM_NX_REG6[],load:0x1->OXM_OF_METADATA[],resubmit(,5)
|
||||
|
||||
In the security table (5), the flow makes sure that the packet has originated
|
||||
from VM's assigned address and prevent network address spoofing, making the packet goto
|
||||
the 'connection track' table (10).
|
||||
from VM's assigned address and prevent network address spoofing, making the
|
||||
packet goto the 'connection track' table (10).
|
||||
|
||||
::
|
||||
|
||||
table=5, priority=200,in_port=8,dl_src=fa:16:3e:95:bf:e9,nw_src=10.0.0.5 actions=resubmit(,10)
|
||||
|
||||
The 'connection track' table is used to create a connection track entry in Linux
|
||||
Kernel, and pass the packet to the service classification table.
|
||||
The 'connection track' table is used to create a connection track entry in
|
||||
Linux Kernel, and pass the packet to the service classification table.
|
||||
The service classification table filters out service oriented packets and pass
|
||||
the packet to the L2 lookup table, same as any other network type.
|
||||
|
||||
|
@ -139,8 +141,8 @@ following flow is set
|
|||
|
||||
Add remote port
|
||||
---------------
|
||||
A flow in the L2 lookup (55) is planted to translate VM mac and network membership
|
||||
to it's port key in Reg7 and pass to egress table 75
|
||||
A flow in the L2 lookup (55) is planted to translate VM mac and network
|
||||
membership to it's port key in Reg7 and pass to egress table 75
|
||||
|
||||
::
|
||||
|
||||
|
@ -189,11 +191,13 @@ egress bum traffic flows.
|
|||
|
||||
Impact on other DF applications
|
||||
-------------------------------
|
||||
The changes in the L2 application will affect the Provider Networks App. DNAT App, SNAT App et al.
|
||||
The changes in the L2 application will affect the Provider Networks App. DNAT
|
||||
App, SNAT App et al.
|
||||
|
||||
According to the propsed design, L2 application deals with local chassis flows, while
|
||||
the 'tunneling app', 'provider networks app', 'DNAT app' and 'SNAT app', should deal
|
||||
with setting the flows for incoming/outgoing packets from/to external nodes.
|
||||
According to the propsed design, L2 application deals with local chassis flows,
|
||||
while the 'tunneling app', 'provider networks app', 'DNAT app' and 'SNAT app',
|
||||
should deal with setting the flows for incoming/outgoing packets from/to
|
||||
external nodes.
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
|
|
@ -100,12 +100,12 @@ remote port will create a tunnel port for each remote chassis, and maintain a
|
|||
relationship of remote port and remote chassis in the cache of Dragonflow
|
||||
controller. If there is no remote port in the remote chassis, the tunnel port
|
||||
for remote chassis will be deleted. By using the virtual tunnel port, there is
|
||||
no need to maintain tunnel port for remote chassis. When a remote port is added,
|
||||
the IP address and tunnel type of the remote chassis will be added in the
|
||||
binding_profile. The tunnel type value will be the type of the network of the
|
||||
remote port. If a network packet needs to go to the remote port, it will go to
|
||||
the virtual tunnel port, instead of the specific tunnel port created in the old
|
||||
implementation. The OpenFlow will designate the destination IP address,
|
||||
no need to maintain tunnel port for remote chassis. When a remote port is
|
||||
added, the IP address and tunnel type of the remote chassis will be added in
|
||||
the binding_profile. The tunnel type value will be the type of the network of
|
||||
the remote port. If a network packet needs to go to the remote port, it will go
|
||||
to the virtual tunnel port, instead of the specific tunnel port created in the
|
||||
old implementation. The OpenFlow will designate the destination IP address,
|
||||
according to the information of remote port's binding_profile. When the remote
|
||||
port is deleted, only the related OpenFlows needs to be deleted.
|
||||
|
||||
|
|
|
@ -126,13 +126,15 @@ if this port is the first port of the network on the host,
|
|||
controller will install broadcast flows on OVS like this:
|
||||
1.Table=L2_Lookup,
|
||||
Match: metadata=network_id, dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,
|
||||
Actions: resubmit(,EGRESSTABLE), load_reg7=port_unique_key,resubmit(,EGRESSTABLE)
|
||||
Actions: resubmit(,EGRESSTABLE), load_reg7=port_unique_key,
|
||||
resubmit(,EGRESSTABLE)
|
||||
|
||||
2.Table=Egress_Table,
|
||||
Match: metadata=network_id,
|
||||
Actions:mod_vlan=vlan_id,output:path_br_1
|
||||
|
||||
If this port is not the first one, controller only updates the first flow above.
|
||||
If this port is not the first one, controller only updates the first flow
|
||||
above.
|
||||
|
||||
Remote Port
|
||||
~~~~~~~~~~~
|
||||
|
@ -141,9 +143,11 @@ broadcast flows. Because with broadcast, OVS just needs to forward it to br-1.
|
|||
This has been done when local port updated.like this.
|
||||
1.Table=L2_Lookup,
|
||||
Match: metadata=network_id, dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,
|
||||
Actions: resubmit(,EGRESSTABLE), load_reg7=port_unique_key,resubmit(,EGRESSTABLE)
|
||||
Actions: resubmit(,EGRESSTABLE), load_reg7=port_unique_key,
|
||||
resubmit(,EGRESSTABLE)
|
||||
|
||||
The first action 'resubmit(,EGRESSTABLE)' has included remote broadcast scenario.
|
||||
The first action 'resubmit(,EGRESSTABLE)' has included remote broadcast
|
||||
scenario.
|
||||
|
||||
|
||||
Outbound-Unicast
|
||||
|
@ -178,7 +182,8 @@ With inbound, a flow item will be installed to table 0, which will strip VLAN
|
|||
tag and set metadata for next table. Flow item like this:
|
||||
Table=0,
|
||||
Match:dl_vlan=network_vlan_id,
|
||||
Actions:metadata=network_id, strip_vlan, goto "Destination Port Classification".
|
||||
Actions:metadata=network_id, strip_vlan, goto "Destination Port
|
||||
Classification".
|
||||
|
||||
For simplicity, I will omit some flow tables that are not so directly related
|
||||
with VLAN networking.
|
||||
|
|
Loading…
Reference in New Issue