Commit Graph

42 Commits (master)

Author SHA1 Message Date
Kailun Qin a01b7125cd Use network segment ranges for segment allocation
This patch makes necessary changes to ML2 type drivers and plugin
manager for network segment range extension support when it is loaded.

When the network segment range extension is not loaded, no impact to the
current flow.

When the extension is loaded,
- populating a range that is managed from the configuration file [1]_,
  such as "VLAN IDs", "VXLAN VNI IDs", "GRE tunnel IDs",
  "Geneve VNI IDs" to the network segment range DB table as a "default"
  and "shared" entry to maintain backward compatibility;
- reloading the "default" segment ranges when Neutron server
- creating a set of "default" network segment ranges out of the
  ML2-config-file-defined ranges [1]_ and the segment allocation
  operations are always retrieving the information from the DB to have
  the network segment ranges fully administered via API;
- when a tenant allocates a segment, it will first allocate from an
  available segment range assigned to the tenant, and then a shared
  range if no tenant specific allocation is possible.

[1] /etc/neutron/plugins/ml2/ml2_conf.ini

Co-authored-by: Allain Legacy <>

Partially-implements: blueprint network-segment-range-management
Change-Id: I522940fc4d054f5eec1110eb2c424e32e8ae6bad
4 years ago
Boden R 95f1e03446 use plugin constants from neutron-lib
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.


Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
6 years ago
Inessa Vasilevskaya 7322bd6efb Make code follow log translation guideline
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.

Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
6 years ago
Anindita Das 995e646b11 Integration of Allocation/Endpoints OVO
This patch integrates the Oslo-Versioned Objects created for
VlanAllocation, VxlanAllocation, VxlanEndpoints, GreAllocation,
GreEndpoints, GeneveAllocation and GeneveEndpoints into the
code base.

Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I0d592bae452876b24c28ca4cc4bf6392b5ab6493
Co-Authored-By: Victor Morales <>
6 years ago
Ihar Hrachyshka 88e99f6f28 Clean up deprecated sqla model import paths
Those were moved under neutron.db.models start of Ocata. We should be
able to clean them up now.


Change-Id: Iadbf44d52ee8e30712807384152a29ce1a8b8f72
6 years ago
Aradhana Singh 62fd09afc7 Refactoring config options for ml2 plugin drivers
Refactoring neutron ml2 drivers configuration options to be in
neutron/conf/plugins/ml2/drivers. This would allow centralization of
all configuration options and provides an easy way to import.

Change-Id: I825bfed769bff92de010733b16f4a4b5171a052c
Partial-Bug: #1563069
7 years ago
Henry Gessau 99cfd671be debtcollector for globals
The deprecation shim created by Doug turns out to be rather useful.
It emits a warning when an global (attribute of a module) is
referenced but that global has been moved to another module.

This update makes the following changes to the shim:

 - Rename it to _MovedGlobals to better describe what it is a
   debtcollector for.

 - Use inspect to get the original reference and to check that
   _MovedGlobals is called from the last line of a module.

 - Save the old reference automatically in the instance to prevent it
   from getting garbage collected.

 - Beef up the _moved_global() method for moving/renaming individual
   globals, allowing it to move and rename or rename in place.

Change-Id: I868aa4a3129dd05467a103364088efbb86bc5d0f
7 years ago
Mohit Malik d7e6ce8a6e Relocate GRE Db models
Change-Id: I00afedc08c9ed7ab03ead9ef4bee94a80f50bb75
Partial-Bug: #1597913
7 years ago
Henry Gessau ae5bad49cc Use exceptions from neutron-lib
Related-Blueprint: neutron-lib

Change-Id: Ia014468bd621c4ee6aea95bf19328c61070174c4
7 years ago
Akihiro Motoki 2d8632e412 Use _ from neutron._i18n
Partial-Bug: #1520094
Change-Id: I874a4aa1d71d1f7034a1ff0b7450b419ef5c6864
8 years ago
Doug Wiegley dd726ed494 Move i18n to _i18n, as per oslo_i18n guidelines
- This does NOT break other projects that rely on neutron.i18n,
  as this change includes a debtcollector shim to maintain those
  older entry points, until they can migrate.
- Also updates to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:

Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
8 years ago
Cedric Brandily 7c273fe218 Abstract sync_allocations
Currently gre/vxlan type drivers have specific sync_allocations
implementations to perform the same feature. This change abstracts
(vxlan) implementation and moves it to TunnelTypeDriver[1] in order to
share implementation between gre/vxlan; this change also adds some

Current gre/vxlan and new implementations of sync_allocations are not
Galera-compliant with multi-writers, a follow-up will update it to
fully support Galera.

[1] neutron.plugins.ml2.drivers.type_tunnel

Change-Id: I188a7cf718d811084475f6783d844588de5d60ea
8 years ago
Cedric Brandily a5bf502fab Remove duplicate tunnel id check in sync_allocations
Currently, gre/vxlan sync_allocations and _parse_tunnel_ranges both
check tunnel id values. This change removes the check in gre/vxlan
sync_allocations as they duplicate _parse_tunnel_ranges check and is
less fine.

Change-Id: I5827468aeaec5d6c79d469132b129aeb7da171e2
8 years ago
Cedric Brandily 46223363bd Refactor type_tunnel/gre/vxlan to reduce duplicate code
gre and vxlan type drivers have similar implementations for multiple
* get_endpoint_by_host
* get_endpoint_by_ip
* delete_endpoint
* get_endpoints
* add_endpoint

This change abstracts these methods and moves the abstractions to the
new class EndpointTunnelTypeDriver.

Change-Id: Iab97f8283b6bf5586334958de950664f6e74202a
8 years ago
Adrien Vergé 5836bbca83 Python 3: Use six.moves.range
The function `xrange` was renamed to `range` in Python 3.

* Remove `xrange` occurences so that Python 3 tests can pass. Use
  `six.moves.range` instead to get the right function in both cases.
* Generalize the use of the efficient `range` (ex-`xrange`) in
  critical sections (when iterating over large lists).
* Simplify code.
* Add a hacking check to prevent future usage of `xrange`.

Change-Id: I080acaaa1d4753619fbbb76dddba6d946d84e73f
Partially implements: blueprint neutron-python3
8 years ago
Robert Pothier f540a48ae7 Add MTU selection to ML2
ML2 will check the config parameters for the MTU settings.
It will check the segment, path and physnet mtu settings.

Change-Id: I58b57e01ec9bcafd7cdcfbf03149e98c3a1291ed
Partially-Implements: blueprint mtu-selection-and-advertisement
8 years ago
Ihar Hrachyshka 22328baf1f Migrate to oslo.log
It's mostly a matter of changing imports to a new location.

Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
  from its thread local store and not from incubator
* don't store context at now that there is no code that
  would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
  not exist
* WritableLogger is now located in oslo_log.loggers

Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).

Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.

Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.

Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
8 years ago
Jakub Libosvar 50b469fae2 Don't crash when adding duplicate gre allocation
This patch catches DBDuplicateError when initializing ML2 GRE type
driver and allocation already exists in DB. Because current allocations
are queried and then only those that doesn't exist in database are
added, DBDuplicateError should never occur.

But the race can happen when running multiple neutron-servers and one
of servers adds allocations between allocations are queried and added.

Change-Id: I427b7020d61b0d2c06292ff2804ba1f4483696c6
Closes-bug: 1417560
8 years ago
Ihar Hrachyshka 7a2a85623d oslo: migrate to namespace-less import paths
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.

This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils

Added hacking check to enforce new import paths for all oslo libraries.

Updated setup.cfg entry points.

We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next



Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
8 years ago
Jenkins 61cf287407 Merge "Overload correctly BASEV2.__table_args__" 8 years ago
Cedric Brandily 0c5d35f37c Overload correctly BASEV2.__table_args__
neutron.db.model_base.BASEV2 defines __table_args__ attribute. Its
subclasses overloading __table_args__ should inherit __table_args__ from
parent class.

A migration script is not needed as __table_args__ are not overloaded in
neutron.db.migration.models.frozen module.

Closes-Bug: #1413985
Change-Id: Ied625252714e2e952f9831d0d03c4573306fe29d
8 years ago
Elena Ezhova f72b07e5d1 Add index on db "allocated" columns
ml2_vxlan_allocations, ml2_gre_allocations, ml2_vlan_allocations tables
have the 'allocated' field.

There are a lot of similar queries to these tables which look
like the following:

SELECT ml2_vxlan_allocations.vxlan_vni
AS ml2_vxlan_allocations_vxlan_vni,
AS ml2_vxlan_allocations_allocated
FROM ml2_vxlan_allocations
WHERE ml2_vxlan_allocations.allocated = 0 LIMIT 1;

Performing such selects can take quite a lot of time and if a transaction
which performs allocation is executed in parallel, it can lead to
allocation failure and retry.

Adding an index on "allocated" column significantly improves
the performance. For ml2_vlan_allocations table created
an index on (physical_network, allocation) together.

Example for MySQL for execution of query
select * from ml2_vxlan_allocations where allocated = 0;
when on the table with ~3 mln entries, ~500K of which
have allocated = 0:
|No index on "allocated"| Index on "allocated"|
|      2.02 sec         |       0.43 sec      |

Closes-Bug: #1412348
Change-Id: Ie90ba611dcae6bd0cb7686a0c7b29b9484eae693
8 years ago
Aman Kumar d580f78a0e Code improvement in and files
This patch set basically addressing the review comment
given in the following patch-set:

Co-Authored-By: Romil Gupta <>

Change-Id: I12cbb4737e4938b8da808cb19f5e9d1e290f2559
9 years ago
Romil Gupta 3db0e619c8 Stale VXLAN & GRE tunnel endpoint deletion from DB
Stale GRE and VXLAN tunnel endpoints persists in neutron db this should be
deleted from the database. Also, if local_ip of L2 agent changes the
stale tunnel ports and flows persists on br-tun on other Compute Nodes and
Network Nodes for that remote ip this should also be removed.


Plugin changes:
Added host column in 'ml2_vxlan_endpoints' and 'ml2_gre_endpoints' table.
Added delete_endpoint method for deleting the stale endpoints from db.
Modified tunnel_sync() method to accommodate these changes.
Modified testcases in
Modified testcases in

Agent changes:
Added tunnel_delete rpc for removing stale ports and flows from br-tun.
tunnel_sync rpc signature upgrade to obtain 'host'.
Added testcases for TunnelRpcCallbackMixin().

This patch-set only deals with plugin side changes.

Partial-Bug: #1179223

Change-Id: I75c6581fcc9f47a68bde29cbefcaa1a2a082344e
9 years ago
Ihar Hrachyshka 3f44c9e278 Migrate to oslo.i18n
Mostly trivial import changes.

- oslo.i18n no longer provide install() method to inject _() into
  globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.

Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
9 years ago
Romil Gupta 6d67c14e1a Update i18n translation for ML2 plugin log msg's
All the existing, LOG.warning, LOG.error and LOG.critical
messages should have _LI, _LW, _LE and _LC respectively.
 Also, debug level log shouldn't be translated.

This patch set will cover the ml2 directory under neutron/plugins.

Partial-Bug: #1320867
Change-Id: I9d78d23bbc14e7c536c6ddf2dc4f52c67faeb667
9 years ago
yangxurong 22bec67395 Tunnel ID range validation for VXLAN/GRE networks
Currently, there is no check which validates the values of
tunnel range for VXLAN/GRE networks. The VXLAN VNI is 24 bit
which have range between 1 to 2^24 - 1. Similarly, GRE key field
is 32 bit which have range between 1 to 2^32 - 1.

Closes-Bug: #1306488

Co-Authored-By: romilg

Change-Id: Idb3d0f41166df589a1e90394d9319901b5f9b439
9 years ago
Cedric Brandily 1100367d4f Refactor type_tunnel/gre/vxlan to reduce duplicate code
gre and vxlan type drivers have similar implementation for multiple
methods. This change abstracts these methods and move them to

sync_allocations method is not shared between gre and vxlan type drivers
because they use different strategies and the targeted one is unclear.

Change-Id: I717b70eb718f7ae82f1bc0c81543b0c438e8bddd
9 years ago
Cedric Brandily bc4965080c Remove SELECT FOR UPDATE use in ML2 tunnel driver add_endpoint
SELECT FOR UPDATE expression, which is triggered with the use of the
SQLAlchemy Query object's with_lockmode('update') method, is
detrimental to performance and scalability of the database
performance code in Neutron due to the lock contention it produces.

SELECT FOR UPDATE can be entirely avoided in add_endpoint methods
with the use of single-shot SELECT and INSERT expressions and the
correction of VxlanEndpoint primary key: indeed previously it was not
possible to create multiple endpoints with the same ip, now the model
primary key constraint ensures it.

Change-Id: Id69fbc15c8f51b4b275cd742312e6ff6802d8c0f
Partial-Bug: #1330562
9 years ago
Cedric Brandily c9fd72a8e1 Add partial specs support in ML2 for gre/vxlan provider networks
ML2 provider networks partial specs let admins choose some provider
network attributes and let neutron choose remaining attributes. This
change provides the implementation for GRE/VXLAN provider networks.

In practice, for GRE/VXLAN provider networks provider:segmentation_id
choice can be delegated to neutron, in such case neutron will try to
find a network in tenant network pools which respects provided
provider attributes.


Related to blueprint provider-network-partial-specs
Partial-Bug: #1330562

Change-Id: I720d7d04f6e3453145e888d9e4d9b5e381d0f7d4
9 years ago
Jenkins e693d5be17 Merge "Add partial specs support in ML2 for vlan provider networks" 9 years ago
Cedric Brandily b3202c3283 Add partial specs support in ML2 for vlan provider networks
ML2 provider networks partial specs let admins choose some provider
network attributes and let neutron choose remaining attributes. This
change provides the implementation for VLAN provider networks.

In practice, for VLAN provider networks provider:physical_network
and provider:segmentation_id choices can be delegated to neutron,
in such case neutron will try to find a network in tenant network
pools which respects provided provider attributes.


Related to blueprint provider-network-partial-specs
Partial-Bug: #1330562

Change-Id: I2c52c71167edaa153b2e04681273e2f1be8d03aa
9 years ago
Ann Kamyshnikova ffdfd3368d Add 'server_default' parameter
If a model mentions 'default' it is better to set 'server_default'
as well because this 'defaut' can be reached only via code. This
change adds 'server_default' parameter to models and in database,
so if someone will use SQL query it will also use default value.

For model Router added parameter server_default. In migration it was
added by this change

For model MeteringLabelRule added parameter server_default.
In migration it will be added by heal script due to problems
with Metering plugin and grenade.

Partial-Bug: #1295539

Change-Id: Iab6fb4eca73e4918c16dac2945fd8552ae2a0c73
9 years ago
Cedric Brandily dca3a8eafc Remove SELECT FOR UPDATE use in ML2 type driver release_segment
SELECT FOR UPDATE expression, which is triggered with the use of the
SQLAlchemy Query object's with_lockmode('update') method, is
detrimental to performance and scalability of the database
performance code in Neutron due to the lock contention it produces.

SELECT FOR UPDATE can be entirely avoided in release_segment methods
with the use of single-shot UPDATE and DELETE expressions, and this
patch clears a number of uses of SELECT FOR UPDATE by consolidating
multiple SQL expressions into one.

Partial-Bug: #1330562
Change-Id: I29ffcafc8d4d73ac1cb50c94df5da85514d47a3f
9 years ago
Jakub Libosvar 1a116d24a9 Fix H302 violations in plugins package
H302 violation is reported by flake8 when importing separated objects from
modules instead of importing the whole module.
e.g.   from package.module import function
is changed to
       from package import module

Change-Id: I83372124f4fba7b94bbfb4a56a0c0ef779ee237f
Partial-Bug: #1291032
9 years ago
lizheming 42683e1784 Fix bug:range() is not same in py3.x and py2.x
in python3.x, there is no method named xrange(),
it has replaced by range(),which is equal to xrange() in python2.x.
so we must fix this issue.we use six module to fix this issue.

Change-Id: I9cee3f0df2d8abbf6f66d7c5a3a0fed98df622e8
Closes-Bug: #1268439
9 years ago
Jenkins 5be0aa3e88 Merge "ml2: gre, vxlan type driver can leak segment_id" 10 years ago
Isaku Yamahata 4808f91eb2 ml2: gre, vxlan type driver can leak segment_id
When multiple region of segment_id is specified, segment_id can leak
(will be never reused) on deletion of network.

Change-Id: Ia27b7e2c0ff4ab58be304e54123a679a5e35ab00
Closes-Bug: #1260262
10 years ago
Justin Hammond 4cdccd69a4 Update common network type consts to same origin
This patch removes new definitions of common network type constants (TYPE_FLAT,
TYPE_LOCAL, etc.) and modifies uses of aforementioned constants to a common
place where constants are defined (neutron.plugins.common.constants). This
patch does not change values that are equal in value but different in name:
NETWORK_TYPE_FLAT vs TYPE_FLAT. A second changeset will be made to handle that

Unit tests were modified as well when they referred to the constant.

Finally, the ovs agent code refers to the OVS plugin constants directly and
these had to be changed as well. A TODO flag was put in that file due to use
of another plugin specific constant.

Network types that were only defined in a single plugin, such as mellanox's
infiniband (IB) network type was not carried over to the common constants file.

Fixes-bug: #1196170

Change-Id: Ib6bfc8275496a24bf247946d177c734b62ae44bb
10 years ago
mathieu-rohon c1b1af72e9 ML2 tunnel drivers validate provider networks correctly
there was a copy/paste error in GRE/VXlan type drivers implementation.
the segment wasn't returned while validating the provider network.

Change-Id: I7df6d2e714d09618644f935a9ed41354b62de9d0
Fixes: bug #1202244
10 years ago
Kyle Mestery 81e87775c8 Add VXLAN tunneling support for the ML2 plugin
Add an ML2 Type Driver for VXLAN networks to allow the creation
of VXLAN networks with the OVS agent.

Change-Id: I7fcc384cc44d5adc752510847d8ba5f46bbb79fb
Implements: blueprint ml2-vxlan
10 years ago
mathieu-rohon 6bdfccaf1b Add gre tunneling support for the ML2 plugin
This patch add the type_driver GRE to enable the creation of GRE
tunnels with the OVS agent.
No Endpoints ID are managed. Only Endpoint IP are stored in DB,
and this IP is proposed as an endpoint ID for the OVS agent.
It also fixes the bug 1201471.

Implements: blueprint ml2-gre
Change-Id: I1a33a4bd3ebc4c97eecf17a59ce16b8c2066ec66
10 years ago