New releases of oslo.config support a 'mutable' parameter to Opts.
This is only respected when the new method mutate_config_files is
called instead of reload_config_files. Neutron delegates making this
call to oslo.service. This was provided in patchset
Icec3e664f3fe72614e373b2938e8dee53cf8bc5e
Further patches will be needed to make select config options be
marked as mutable. This change enables support for oslo provided
config options to be updated via SIGHUP such as log level.
Task: 6389
Story: 2001545
Change-Id: I9442965607f3248706464643c6d87a04edcae24e
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
NeutronLibImpact
Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
Inter Tenant Traffic between two different networks that belong
to two different Tenants is not possible when connected through
a shared network that are internally connected through DVR
routers.
This issue can be seen in multinode environment where there
is network isolation.
The issue is, we have two different IP for the ports that are
connecting the two routers and DVR does not expose the router
interfaces outside a compute and is blocked by ovs tunnel bridge
rules.
This patch fixes the issue by not applying the DVR specific
rules in the tunnel-bridge to the shared network ports that
are connecting the routers.
Closes-Bug: #1751396
Change-Id: I0717f29209f1354605d2f4128949ddbaefd99629
The oslo.messaging bug that triggered adding the hack is fixed in
Queens, so we don't need it anymore.
Change-Id: I775b466dce19c4168985b19e1ddb938118f48dcb
Related-Bug: #1705351
Adding ability to set DSCP field in OVS tunnels outer header, or
inherit it from the inner header's DSCP value for OVS and linuxbridge.
Change-Id: Ia59753ded73cd23019605668e60cfbc8841e803d
Closes-Bug: #1692951
For OVS based DPDK ports ingress bandwidth limit is now implemented
using egress-policer qos type.
Additionally limit values are set in other_config of QoS because there
is no queue used in this case.
This patch moves also helper methods used to conversion between
bytes and bits and between bits and kilobits to neutron.common.utils
to be able to use it also in ovs_lib module.
Change-Id: I94d1e8dfb82df5c602476db8aaa884ae91fecd7f
Closes-Bug: #1724729
When Openvswitch agent will get "port_update" event
(e.g. to set port as unbound) and port is already removed
from br-int when agent tries to get vif port in
treat_devices_added_updated() method (port is removed
because e.g. nova-compute removes it) then resources set
for port by L2 agent extension drivers (like qos) are not
cleaned properly.
In such case port is added to skipped_ports and is set
as DOWN in neutron-db but ext_manager is not called then
for such port so it will not clear stuff like bandwidth
limit's QoS and queue records and also DSCP marking
open flow rules for this port.
This patch fixes this issue by adding call of
ext_manager.delete_port() method for all skipped ports.
Change-Id: I3cf5c57c7f232deaa190ab6b0129e398fdabe592
Closes-Bug: #1737892
If a user clears the gateway_ip of a subnet and the OVS
agent is re-started, it will throw an exception trying
to install the DVR IPv4 flow. Do not install the flow
in this case since it is not required.
Change-Id: I79aba63498aa9af1156e37530627fcaec853a740
Closes-bug: #1728665
This patch switches callbacks over to the payload object style events
[1] for BEFORE_READ based notifications. To do so an EventPayload object
is used with the publish() method to pass along the API related data.
In addition a few UTs are updated to work with the changes.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: Iff3e96c56867b4bf4272fed676f39cd6796d756c
The patch creates tables where other services using openflow can
implement rules for further packet processing. 3 new tables were created
for packets accepted by egress, ingress pipeline and packets dropped by
firewall.
Partially-implements: blueprint security-group-logging
Related-Bug: #1468366
Change-Id: I7900126de235ee9df902bef9556879f586d33ae8
When we delete vm port with attached QoS policy,
it is just doing nothing if vif_port does not exist.
This is fine for egress bandwidth limit as it is configured
directly on vif_port in OVS.
For ingress bw limit however it uses additional records in
Openvswitch database: qos and queue. Those records are not
cleaned up in such case.
This patch also records port in self.ports in the case of
bandwidth limit rules, just as in the case of dscp rules.
Never execute port clear if vif_port not exists. Finally, ovs
driver can clean such qos and queue records
Change-Id: Iddeb49e1e6538a178ca468df0fdf9e0617ca4f1c
Closes-Bug: #1726732
neutron-lib contains the QoS constants. This patch removes them from
neutron and replaces all such uses to lib's QoS constants.
NeutronLibImpact
Change-Id: I6ed379b178a2b79fd14385a1d1e87d87eb04bfb9
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.
NeutronLibImpact
Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
The neutron-lib commit I360545b6ee4291547e0c5c8e668ad03d3efa4725 moved
the externally consumed globals from neutron.common.constants into lib.
With the exception of PROVISIONAL_IPV6_PD_PREFIX all other constants
in neutron.common.constants should only be used in neutron, and will
hopefully remain that way. External consumers needing access to other
common constants should move them into lib first.
NeutronLibImpact
Change-Id: Ie4bcffccf626a6e1de84af01f3487feb825f8b65
When the OVS agent skips processing a port because it was
not found on the integration bridge, it doesn't send back
any status to the server to notify it. This can cause the
port to get stuck in the BUILD state indefinitely, since
that is the default state it gets before the server tells
the agent to update it.
The OVS agent will now notify the server that any skipped
device should be considered DOWN if it did not exist.
Change-Id: I15dc55951cdb75c6d87d7c645f8e2cbf82b2f3e4
Closes-bug: #1719011
As ingres traffic to instance ports when using DVR uses same matching
openflow rule as openvswitch firewall driver, it happens that setting
admin_state_up of router deletes firewall rules.
This patch makes the deletion more strict because DVR and ovs-firewall
flows differ in priority. Thus using priority when removing DVR flows
won't affect ovs-firewall flows.
Closes-bug: #1721084
Change-Id: I4eb61b2824579a4f8ba219cd1b1dcf57d38ebc89
Previously, DP ID was converted to integer and then back to string. As a
consequence of the conversion, DP IDs like 000123 were converted to 123
losing leading zeros. In case self._get_dp_by_dpid() method raises a
RuntimeError exception current DP ID of the bridge was compared to
cached DP ID and if IDs were different, original exception coming from
ryu library was swallowed. As conversion for cached DP ID removes
leading zeros, original exception was always swallowed if bridge's DP ID
started with zero.
This patch uses the integer for comparison between current and cached
bridge DP ID hence any exception coming from ryu is not swallowed.
Closes-bug: #1718235
Change-Id: I445aa61acc758b56c51a9403df4d92d9c1d40ace
Otherwise we don't see some of them for the agent, for example,
AGENT.root_helper is missing.
To make sure the logging is as early as possible, and to make sure that
options that may be registered by extensions are also logged, some
refactoring was applied to the code to move the extension manager
loading as early as possible, even before agent's __init__ is called.
Related-Bug: #1718767
Change-Id: I823150cf6406f709d1e4ffa74897d598e80f5329
In I77650be5f04775a72e2bdf694f93988825a84b72 we added
vnic_type direct to the ovs mechanism drivers supported
vnic_types. This cause problems when working with ovs and sriovnicswitch
mechanism drivers in that order. In this case the ovs will bind
the direct port instead of the sriovnicswitch.
This change make ovs mech driver to bind the direct port only
if user requested --binding-profile '{"capabilities": ["switchdev"]}'
in the direct port if a user don't request this capability the SR-IOV
legacy NIC mode is used.
When enable-sriov-nic-features will be implemented in nova and
libvirt will expose the switchdev capability then nova will be
able to select a host which supports SR-IOV nic with switchdev
mode.
[1] - https://review.openstack.org/#/c/435954/11/specs/pike/approved/enable-sriov-nic-features.rst
[2] - https://www.redhat.com/archives/libvir-list/2017-August/msg00583.html
Closes-Bug: #1713590
Change-Id: I0b5f062bcbf02381bdf4f694fc039f9bb17a2db5
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
This prevents oslo messaging dispatch from trying
to treat the receiver function as a target. When it
thought it was a target it was referencing attributes
it didn't have.
Closes-Bug: #1710064
Change-Id: I1832a05f23cd76c1732d3b98539988bc9519d79e
This patch introduces a workaround on the Neutron side
for bug 1705351 that results in the Neutron server blocking
trying to send to topics the agents no longer subscribe to.
The workaround is to just subscribe to those topics and do
nothing with the messages that come in until oslo.messaging
can properly recover from loss of listeners.
Change-Id: I946a33dfd0c0da26bb47b524f75f53bf59d3fbd5
Closes-Bug: #1705351
The push notification resource cache will already give us
the port update and delete notifications so there is no need
to log here now.
Change-Id: I6000691dfd61facf47d70360d4050cfdefc3e7e2
Partial-Bug: #1707307
Calculate all security group info on the agent from
the push notification cache.
Partially-Implements: blueprint push-notifications
Change-Id: I5c74ba17223a431dad924d31bbe08ad958de3877
Added datapath_type to vif_details returned by OVS
mech driver.
Depends-On: Ie523c821995c046c7f77783a34e75053fc0abb3d
Partial-Bug: #1632372
Change-Id: Ief83150caf1a32a2c043b0245b36e5ebc3a16379
This was deprecated over a year ago in [1] so let's
get rid of it to clean up some code.
1. Ib63ba8ae7050465a0786ea3d50c65f413f4ebe38
Change-Id: I6039fb7e743c5d9a1a313e3c174ada36c9874c70
In Kernel 4.8 we introduced Traffic Control (TC see [1]) hardware offloads
framework for SR-IOV VFs which allows us to configure the NIC [2].
Subsequent OVS patches [3] allow us to use the TC framework
to offload OVS datapath rules.
This patch allow OVS mech driver to bind direct (SR-IOV) port.
This will allow to offload the OVS flows using tc to the SR-IOV NIC
and gain accelerate OVS.
[1] https://linux.die.net/man/8/tc
[2] http://netdevconf.org/1.2/papers/efraim-gerlitz-sriov-ovs-final.pdf
[3] https://mail.openvswitch.org/pipermail/ovs-dev/2017-April/330606.html
DocImpact: Add SR-IOV offload support for OVS mech driver
Partial-Bug: #1627987
Depends-On: I6bc2539a1ddbf7990164abeb8bb951ddcb45c993
Change-Id: I77650be5f04775a72e2bdf694f93988825a84b72
This will help us debug why flows are unexpectedly being
cleaned up if the related bug ever resurfaces.
Related-Bug: #1697243
Change-Id: I517b16c550037f41a5f4915b98963c2232daa78c
Other than the of_interface methods, today we are forced to use ovs_lib
methods to program flows, which call ovs-ofctl in turn. Let's make
install_instructions accept string actions regardless of of_interface
driver.
Related-bug: #1672340
Change-Id: Ie2fdeab1f4ac567ef336abae03768e43f426311a
DVR flows are not compatible with OVS firewall flows as firewall flows
have higher priority. As a consequence, rules for DVR were never match
as firewall uses output directly.
This patch replaces flows using normal or output actions and resends
packets to TRANSIENT table instead. This transient table then uses
either those normal or output action rules. With this split, we will be
able to match egress/ingress flows in TRANSIENT table instead of
LOCAL_SWITCHING putting DVR pipeline in front of OVS firewall pipeline.
Change-Id: I9f738047f131b42d11a90f539435006d16ea7883
Closes-bug: #1696983
Now with the merge of push notifications, processing a port update
no longer automatically implies a transition from ACTIVE to BUILD
to ACTIVE again.
This resulted in a bug where Nova would unplug and replug an interface
quickly during rebuild and it would never get a vif-plugged event.
Nothing in the data model was actually being updated that resulted in
the status being set to DOWN or BUILD and the port would return before
the agent would process it as a removed port to mark it as DOWN.
This fixes the bug by making the agent force the port to DOWN whenever
it loses its VLAN. Watching for the VLAN loss was already introduced
to detect these fast unplug/plug events before so this just adds the
status update.
Closes-Bug: #1694371
Change-Id: Ice24eea2534fd6f3b103ec014218a65a45492b1f
Add support for QoS ingress bandwidth limiting in
openvswitch agent.
It uses default ovs QoS policies on bandwidth limiting
mechanism.
DocImpact: Ingress bandwidth limit in QoS supported by
Openvswitch agent
Change-Id: I9d94e27db5d574b61061689dc99f12f095625ca0
Partial-Bug: #1560961
When native ofctl interface is used (the default), the agent main() is
running in a separate gevent thread. Unless we explicitly request from
ryu to raise errors that may have happened in the agent app, it will
ignore them (only logging a warning message). This may interfere with
service management software like systemd that may use the return code to
decide whether to restart the dead service.
This patch makes ryu raise any uncaught errors happening inside the
agent. It also makes the agent 'wrapper' helper function not to swallow
raised exceptions on logging the error. Those two changes combined make
the agent exit with rc=1 if an exception happens inside the main()
function when in native mode.
This patch doesn't include any unit tests because those would be very
silly (like checking that we indeed pass the needed arguments to ryu).
Change-Id: Ic86b5eeae25a916c3c51f21e6820f5b0212dd5f8
Closes-Bug: #1694505
Replace the calls to the OVSPluginAPI info retrieval functions
with reads directly from the push notification cache.
Since we now depend on the cache for the source of truth, the
'port_update'/'port_delete'/'network_update' handlers are configured
to be called whenever the cache receives a corresponding resource update.
The OVS agent will no longer subscribe to topic notifications for ports
or networks from the legacy notification API.
Partially-Implements: blueprint push-notifications
Change-Id: Ib2234ec1f5d328649c6bb1c3fe07799d3e351f48
This moves the get_vif_type hook point from OVS into the mech agent
base so other mechanism drivers can also return different VIF types
depending on the context of the binding.
This will be used by linux bridge in change
I23c5faaeab69aede1fd038a36f4a0b8f928498ce.
Change-Id: I3b695909c954158df90f436a7ed259890977d25a
In some cases we would want to refrain from cleaning up specific
openvswitch ports.
In Octavia, the health manager service is using a predefined[1]
openvswitch port which will gets nuked by the ovs_cleanup script in the
boot process.
That port is created by the operating system NIC configuration file
(by using OVS_EXTRA[2]), but due to the order of actions in the boot
process, the ovs_cleanup script gets invoked by systemd only at a later
stage. As a result the port will be deleted each time and the Octavia
health manager service will fail to bind.
This patch takes advantage of the 'external_ids' column that already
exists for ovs ports, in order to filter out ports we would like to
skip. We filter those ports by adding 'skip_cleanup' to the
'external_ids' column.
It is important to note that this will work if we append the following
to the port: -- set Interface o-hm0 external_ids:skip_cleanup=true"
Related-Bug: #1685223
[1] http://git.openstack.org/cgit/openstack/octavia/tree/devstack/plugin.sh?h=stable/ocata#n190
[2] https://github.com/osrg/openvswitch/blob/master/rhel/README.RHEL#L102
Change-Id: If483d0ee027596999370ab0d21b1743d4ef16acb
Eventlet now deprecate TimeoutError exception [1].
Updating eventlet to latest 0.21.0 can cause UT fail because
of misunderstood between deprecated exception and the new exception class
Timeout. [2]
This patch remove deprecated exception usage, fix the UT failure with eventlet
0.21.0 in order to upgrade eventlet upper-constraint to latest version.
[1]. https://github.com/eventlet/eventlet/blob/master/eventlet/__init__.py#L55
[2]. https://review.openstack.org/#/c/460424/
Change-Id: I48179953ef19ec6b2f3f6706337a1dae1b3830bd
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
according to https://wiki.openstack.org/wiki/Python3, now we should avoid
using six.iteritems and replace it with dict.items.
Change-Id: I58a399baa2275f280acc0e6d649f81838648ce5c
Closes-Bug: #1680761
This is the minimum change to pass tempest's
test_update_instance_port_admin_state test. Alternatively, the delete_flows
could be changed to just deal with drop_port flows, which can affect
3rd party codes.
Change-Id: Id15eed5f21bc6842daceb28ee9bc660f20e9b04a
Closes-Bug: #1559920
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.
NeutronLibImpact
- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.
[1] 87e42f993c07ae320159d5123662ee9f3bd4d903
Change-Id: I669af9b4c712877772d91a03857ab108714001d4
We have made os-xenapi repository to deal with XenServer Dom0
specific functions, this patch is to change neutron to use
os-xenapi when XenServer is hypervisor and move the building
RPM scripts into os-xenapi repo
Depends-On: I8a31c81d9475387fe4ed7030b70b26098e588771
Change-Id: Ia958c366189386b1b5abbadbb4d74950aaa23bb2
Follow up to a comment by kevinbenton on
Idd0531cedda87224531cb8fb6a912ccd0f1554d5 .
The rationale is that one should never call delete a flow
with cookie=ANY_COOKIE and give a cookie_mask at the same time.
Change-Id: I20a03cb04f1bd3eb7a851e5458e2c662cfa3b132
Instead of having OVSCookieBridge as a passthrough class that does not
provide the intended behavior (see bug 1557620), this change implements
a cookie bridge as a patched copy of the underlying bridge:
- the underlying bridge is copied
- the copy is given an extension-specific cookie
The 'extension bridge should only touch its flows' effect is obtained
by a separate change (Idd0531cedda87224531cb8fb6a912ccd0f1554d5).
The two problems in the bug are addressed:
- the extension-specific cookie is now applied even for calls to
methods other than add/delete/mod_flows
- the extension-specific cookie is now applied in the case of the
native/ryu implementation
This commit also re-enable the use of uninstall_flows in the QoS OVS driver,
which had to be disabled in Idd0531cedda87224531cb8fb6a912ccd0f1554d5, but
can now be re-enabled with this bug addressed.
This change complements the unit tests to confirm that the bug is
fixed.
Change-Id: I55835a34d8fba7a139dce93f99cbff54584d695c
Closes-Bug: #1557620
Needed-By: I8570441a0b8d5ee3ad7f88e07affac2f1b782021
With this change delete_flows will only remove flows matching the default
cookie of the bridge.
The uninstall_flows implementation in the native bridge is also modified
to touch only the flows with the bridge cookie.
To still allow deletion of all cookies, cookie=COOKIE_ANY is introduced
as a special value, and used in the agent code in the places where the
intent is indeed to clean all flows whatever their cookie is.
Partial-Bug: #1557620
Change-Id: Idd0531cedda87224531cb8fb6a912ccd0f1554d5
Sławek Kapłoński
Swaminathan Vasudevan
Boden R
Ihar Hrachyshka
Ali Sanhaji
Nguyen Phuong An
OmarM
Brian Haley
Jakub Libosvar
Zachary
chenghuiyu
Moshe Levi
Inessa Vasilevskaya
Kevin Benton
Kevin Benton
Rodolfo Alonso Hernandez
IWAMOTO Toshihiro
Nir Magnezi
Hieu LE
fpxie
Huan Xie
Thomas Morin