Commit Graph

41 Commits (b79842f289f0c9b9fdce9d6cc689538ec61216e7)

Author SHA1 Message Date
Brian Haley b79842f289 Start enforcing E125 flake8 directive
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues.  Didn't think it was going to be
close to 100 files when I started.

Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
4 years ago
Boden R 957eca96f4 delete common_db_mixin
The functionality within neutron.db.common_db_mixin is available via
neutron-lib APIs. This patch removes common_db_mixin and updates any
uses of it to use neutron-lib instead.



Change-Id: I2388f90b37abb09408809dda8c21da551bcd94bb
4 years ago
Hongbin Lu 46913a69fd Use constant IP_VERSION_4/6 in functional tests
Change-Id: I62b5a37508838a42b03a39de02660b8cafc08c41
5 years ago
Brian Haley d00b7ddec5 Remove router_ids argument to auto_schedule_routers()
The router_ids argument to auto_schedule_routers() is
unused, and was marked for deprecation in Queens.

Change-Id: Ie97b1ad05e294b5fe763ae8d7319800eb16ea3dc
6 years ago
Nakul Dahiwade 7c7b2d75aa OVO for NetworkDhcpAgentBinding
This patch introduces and implements Oslo-Versioned Objects for

Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Co-Authored-By: Manjeet Singh Bhatia <>

Change-Id: Ie6220f8a1455ea721f0d9c7a1b58240cc5fde05a
6 years ago
Boden R 0a1405794f use service type constants from neutron_lib plugins
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.

Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
6 years ago
Jenkins e76fe6262a Merge "Eliminate mixin references in DVRResourceOperationHandler" 6 years ago
Henry Gessau b3c0d5f239 Eliminate lookup of "resource extend" funcs by name
By registering functions directly we cut off the dependency of the
"resource extend" functions on the plugin. This is a step towards
the goal of removing the CommonDbMixin mixin class.

Also, we register all "resource extend" functions at plugin create
(in __new__) instead of in the class definition (which caused the
hooks to be registered on import). This ensures the "resource
extend" functions are only registered for the plugins/mixins that
are actually used.

Note that decorators are used to register "resource extend" methods,
similar to the callback receiver decorators.

Related-Blueprint: neutron-lib

Change-Id: I128cfda773d5f9597df9cd61261fdc05f2a174aa
6 years ago
Kevin Benton 1405391b17 Eliminate mixin references in DVRResourceOperationHandler
This eliminates the mixin references in DVRResourceOperationHandler
to prepare for it to be used by the DVR service provider and to be
removed from the main L3 plugin mixin.

Related: blueprint multi-l3-backends
Change-Id: I5ab6b92b67e6d3b461fd118ef0a26d6f6b6992c4
6 years ago
fpxie d2976d46d0 Replace six.iteritems with dict.items(Part-1)
according to,
now we should avoid using six.iteritems and replace
it with dict.items.

Change-Id: I8753e80b34c0f86cf70aebc3bcbd3392ee933f62
Partial-Bug: #1680761
6 years ago
Boden R 9658b4b34e use neutron_lib's provider_net api-def
Neutron-lib 1.1.0 is now out and contains the provider
network API definition (as per commit [1]). This patch
moves neutron references over to the neutron-lib

- Consumers using the public constants within neutron's
providernet API extension must now use the values
from neutron-lib.

[1] cba0f9f0dd920b1f828c4bba3bd388d5b4eb9abf

Change-Id: I46390a159e93642901de87ea6604f2e7ffa03bad
6 years ago
Ankur Gupta 55090400ad OVO External Networks
This patch introduces and implements Olso-Versioned Objects for
network extension External Networks.

There were joined performed to order the fetching of external
networks by standard attribute which seems useless because,
in networks/services/auto_allocated/ while fetching
external networks it logs error when multiple networks are
returned. Expected default external network there is one, so
ordering does not make much sense.

Co-Authored-By: Manjeet Singh Bhatia <>
Co-Authored-By: Victor Morales <>

Change-Id: Iad609f72945b84df7881b43d1fdf9a188e5816bc
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
6 years ago
Armando Migliaccio ca751a1486 Spin off context module

Partially-implements: blueprint neutron-lib

Change-Id: I48cf45dc1b07035d952152eac2548a3bd9fc2832
6 years ago
Assaf Muller dd5aca38f9 Remove deprecated min_l3_agents_per_router
The option was deprecated [1] for removal in Newton
and is being removed in Ocata.

[1] Deprecated in patch with Gerrit Change-Id of:

DocImpact remove all references to the option.

Change-Id: I3a9195ff6fd18fad9f85cec03a632e7e52d954e7
Closes-Bug: #1555042
6 years ago
John Schwarz 3e4c0ae223 Revert "Add ALLOCATING state to routers"
This reverts commit 9c3c19f07c.

Following the merge of Ie98d5e3760cdb17450aea546f4b61f5ba14baf1c, the
creation of new router uses RouterL3AgentBinding and its' new
binding_index attribute to ensure correctness of the resources. As such,
the ALLOCATING state (which was used to do just that) is no longer
needed and can be removed.

Closes-Bug: #1609738
Change-Id: Ib04e08df13ef4e6b94bd588854a5795163e2a617
6 years ago
John Schwarz 1e195afaf1 L3 scheduler: add retry indicators for bind_router
Change I3447ea5bcb7c57365c6f50efe12a1671e86588b3 added a binding_index
to the RouterL3AgentBinding table. In certain (concurrent) cases, a row
with the same binding_index might be used twice, which will raise
DBDuplicateEntry. However, that change didn't retry on this case at all
code-paths, so this patch rectifies this issue.

Closes-Bug: #1633042
Change-Id: I228b0084a8e7c48e78bc2ea6a3fccf7437210e73
7 years ago
Manjeet Singh Bhatia 276015788a Relocate external network db models
This patch will relocate external network db models
and separate it from mixins for OVO implementation.

Change-Id: I83e2c7bbab3bd59cc9d2af71c66c0662c7724ecc
Partial-Bug: #1597913
7 years ago
John Schwarz b1ec8d523d Add binding_index to RouterL3AgentBinding
The patch proposes adding a new binding_index to the
RouterL3AgentBinding table, with an additional Unique Constraint that
enforces a single <router_id, binding_id> per router. This goes a long
way into fixing 2 issues:

1. When scheduling a non-HA router, we only use binding_index=1. This
   means that only a single row containing that router_id can be
   committed into the database. This in fact prevents over-scheduling of
   non-HA routers. Note that for the HA router case, the binding_index
   is simply copied from the L3HARouterAgentPortBinding (since they are
   always created together they should always match).

2. This sets the ground-work for a refactor of the l3 scheduler - by
   using this binding and db-based limitation, we can schedule a router
   to agents using the RouterL3AgentBinding, while postponing the
   creation of L3HARouterAgentPortBinding objects for the agents until
   they ask for it (using sync_routers). This will be a major
   improvement over todays "everything can create
   L3HARouterAgentPortBinding" way of things).

Closes-Bug: #1535557
Change-Id: I3447ea5bcb7c57365c6f50efe12a1671e86588b3
7 years ago
Gary Kotton 9f09f27c5d Fix deprecation warnings
Remove deprecation warnings for various constants
and exceptions that have moved to neutron_lib.

Fix miscellaneous other deprecations.

Uses constants instead of l3_constants when importing
neutron-lib constants.

Co-Authored By: Henry Gessau <>
Co-Authored By: Gary Kotton <>

Change-Id: Ib0e8ff5c3e23677c1009241a1818cbc8a3430c38
7 years ago
Jenkins e92579b3d5 Merge "DHCP Auto Scheduling for routed provider networks" 7 years ago
Aradhana Singh 996b84540b DHCP Auto Scheduling for routed provider networks
In routed provider networks, a network can be divided by segments and
each segment will have its own DHCP agent. AutoScheduling for networks
exists. This patchset is for enabling auto scheduling for routed

Change-Id: I7d9f01dbd6d9c216474ee47c968919c1826787f7
Partially-implements: blueprint routed-networks
7 years ago
Jenkins 5be941a4a1 Merge "Added test cases for DVR L3 schedulers." 7 years ago
Ihar Hrachyshka 2a23527a9a tests: stop using ml2 plugin full import paths in tests
Stevedore documentation suggest that full import paths are not supposed
to be user visible. Since unit tests emulate users when configuring
oslo.config, we better off relying on well known plugin aliases than
internal details.

For in-tree that may be not a big deal, but with it we set a bad example
for third parties that may later find their tests broken eg. when we
decide to move code around.


Change-Id: I7bd036ac3df7e7f4c678356d0a793e7d38599dda
7 years ago
Jakub Libosvar 766abb752a Make pep8 job great again
There is a bug in pep8, when 'select' used, it omits all default checks
and runs only those specified by 'select'.  We got hit by this issue
since I2d26534230ffe5d01aa0aab6ec902f81cfba774d was merged which lead to
almost no static checks in pep8 job.

Also note that off_by_default decorator has no effect for now because
factory in hacking is triggered after ignored checks are collected.
There will be a follow-up patch for that in order to make pep8 doing
its job quickly.


Related-Bug: 1594756
Change-Id: I8e27f40908e1bb4307cc7c893169a9d99f3433c4
7 years ago
tonytan4ever 63fc967a8f Refactor NetworkDhcpAgentBinding
This patch set is for breaking the circular dependency between
Agent/NetworkDhcpAgentBinding. The goal of this is to implement
Agent OVO

Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I3f2a8bcc6f8644e94e377dc916fba6743cb230bc
7 years ago
Henry Gessau 4148a347b3 Use constants from neutron-lib
With this we enable the deprecation warnings by default.

Related-Blueprint: neutron-lib

Change-Id: I5b9e53751dd164010e5bbeb15f534ac0fe2a5105
7 years ago
Kevin Benton 9c3c19f07c Add ALLOCATING state to routers
This patch adds a new ALLOCATING status to routers
to indicate that the routers are still being built on the
Neutron server. Any routers in this state are excluded in
router retrievals by the L3 agent since they are not yet
ready to be wired up.

This is necessary when a router is made up of several
distinct Neutron resources that cannot all be put
into a single transaction. This patch applies this new
state to HA routers while their internal HA ports and
networks are being created/deleted so the L3 HA agent
will never retrieve a partially formed HA router. It's
important to note that the ALLOCATING status carries over
until after the scheduling is done, which ensures that
routers that weren't fully scheduled will not be sent to
the agents.

An HA router is placed in this state only when it is being
created or converted to/from the HA state since this is
disruptive to the dataplane.

This patch also reverts the changes introduced in
Iadb5a69d4cbc2515fb112867c525676cadea002b since they will
be handled by the ALLOCATING logic instead.

Co-Authored-By: Ann Kamyshnikova <>
Co-Authored-By: John Schwarz <>

Closes-Bug: #1550886
Related-bug: #1499647
Change-Id: I22ff5a5a74527366da8f82982232d4e70e455570
7 years ago
Ritesh Anand b47bd6bce1 Added test cases for DVR L3 schedulers.
Test cases listed below are to run for Chance and Least-routers scheduler,
for both auto and explicit scheduling.
  1. Legacy router scheduled on dvr_snat agent.
  2. Distributed router not scheduled on legacy agent.
  3. Distributed router not scheduled on dvr agent.
  4. Distributed router scheduled on dvr_snat agent.
  5. Already hosted legacy router not scheduled on dvr agent.
  6. Already hosted legacy router not scheduled on dvr_snat agent.
  7. Already hosted distributed router not scheduled on legacy agent.
  8. Already hosted distributed router not scheduled on dvr agent.
  9. Already hosted distributed router not scheduled on dvr_snat agent.
  10. Already hosted legacy router not scheduled on additional dvr agent.
  11. Distributed router not scheduled if it is on a different external
      network than the dvr_snat agent.

Change-Id: I94d6654631b3cd66dfd469b1230c3ec20d0c0369
7 years ago
Assaf Muller 0267c6a5ac Make DHCP agent scheduler physical_network aware
Currently neutron DCHP scheduler assumes that that every server running
a dhcp-agent can reach every network. Typically the scheduler can
wrongly schedule a vlan network on a dhcp-agent that has no reachability
to the network it's supposed to serve (ex: network's physical_network
not supported).

Typically such usecase can append if:

* physical_networks are dedicated to a specific service and we don't
  want to mix dnsmasqs related to different services (for
  isolation/configuration purpose),
* physical_networks are dedicated to a specific rack (see example
  diagram, the rack interconnection can
  be handled outside of neutron or inside when routed-networks will be

This change makes the DHCP scheduler network reachability aware by
querying plugin's filter_hosts_with_network_access method.

This change provides an implementation for ML2 plugin delegating host
filtering to its mechanism drivers: it aggregates the filtering done by
each mechanism or disables filtering if any mechanism doesn't overload
default mechanism implementation[1] (for backward compatibility with
out-of-tree mechanisms). Every in-tree mechanism overloads the default
implementation: OVS/LB/SRIOV mechanisms use their agent mapping to filter
hosts, l2pop/test/logger ones return empty set (they provide to "L2

This change provides a default implementation[2] for other plugins
filtering nothing (for backward compatibility), they can overload it to
provide their own implementation.

Such host filtering has some limitations if a dhcp-agent is on a host
handled by multiple l2 mechanisms with one mechanism claiming network
reachability but not the one handling dhcp-agent ports. Indeed the
host is able to reach the network but not dhcp-agent ports! Such
limitation will be handled in a follow-up change using host+vif_type

[1] neutron.plugin.ml2.driver_api.MechanismDriver.\
[2] neutron.db.agents_db.AgentDbMixin.filter_hosts_with_network_access

Closes-Bug: #1478100
Co-Authored-By: Cedric Brandily <>
Change-Id: I0501d47404c8adbec4bccb84ac5980e045da68b3
7 years ago
Bhagyashri Shewale 88e899f7a0 Fix module's import order
Made corrections in import order for built-in, third party and
project specific modules as per OpenStack import standards [1].


Change-Id: I899deefd6ee4732d6c0afd17a5afbe42b0fa37ba
7 years ago
Mathieu Rohon 5d53dfb8d6 Avoid duplicating tenant check when creating resources
The check of the tenant done in the method _get_tenant_id_for_create()
is already did by the Neutron Controller in prepare_request_body(),
with a call to attributes.populate_tenant_id().
Moreover, when the Controller processes a "create" requests, it
will add the 'tenant_id' to the resource dict.
Thus, _get_tenant_id_for_create() can be deleted.
Calls to this method are replaced by the res['tenant_id'].

Changes have to be done in UT to explicitly add the tenant_id while
creating resources, since the UT framework is bypassing the controller code
that automatically adds the tenant_id to the resource.

Co-Authored-By: Hong Hui Xiao <>
Closes-Bug: #1513825
Change-Id: Icea06dc81344e1120bdf986a97a6b1094bbb765e
Depends-On: I31022e9230fc5404c6a94edabbb08d2b079c3a09
Depends-On: Iea3f014ef17a1e1b755cd2efe99afd1a36ebbc6a
Depends-On: I604602d023e0cbf7f6591149f914d73217d7a574
8 years ago
Hirofumi Ichihara 5700afb320 Add functional test for availability_zone support
This patch adds the functional tests for availability_zone support.

Change-Id: Ie7fe7cd357a952baacb95d4fc05a687c6465ad61
Co-Authored-By: IWAMOTO Toshihiro <>
Partially-implements: blueprint add-availability-zone
8 years ago
Hirofumi Ichihara 6e50027819 Add availability_zone support for network
This patch adds the availability_zone support for network.


Change-Id: I9259d9679c74d3b3658771290e920a7896631e62
Co-Authored-By: IWAMOTO Toshihiro <>
Partially-implements: blueprint add-availability-zone
8 years ago
Jenkins dc0944f2d4 Merge "Fix dhcp autoschedule test assertion logic" 8 years ago
Assaf Muller 5df1a72c95 Fix KeyError: 'L3_ROUTER_NAT' in l3 scheduler functional test
The issue was introduced by commit with change ID:

In Neutron-speak we define the 'core plugin' as something like ML2, while
service plugins examples are L3, VPNaaS, etc. The code was initializing
the L3 service plugin as the core plugin, which is unexpected behavior.
The code will now use the DB core plugin base class as the core plugin,
and not initialize the service plugins. The tests will manually and locally
instantiate a L3 service plugin instance and use it.

Change-Id: I2b7d2f6ccd5fe18e322e70b4e376db23d76b9092
Closes-Bug: #1478273
8 years ago
Darragh O'Reilly 7f152490bf Fix dhcp autoschedule test assertion logic
The functional test for auto_schedule_networks passes even if the
method does nothing. This patch changes the test so it checks that
each expected hosted network is in hosted_net_ids.

Change-Id: I42130993b1af3f3d028e355d30322853453f064f
Closes-Bug: #1478531
8 years ago
Ritesh Anand 0a8369cc69 Added functional tests for L3 schedulers.
Legacy mode agents considered.

Following test cases are added:
Common scenarios:
1. No routers scheduled if no agents are present
2. No routers scheduled if it is already hosted
3. No routers scheduled if all agents are down
4. Router scheduled to the agent if router is not yet hosted
5. Router scheduled to the agent even if it already hosts a router

Following scenario specific to least routers scheduler is added:
1. Router is scheduled to agent hosting least routers

For each of Chance and Least Router schedulers auto scheduling is also verified.

A total of 22 test cases are added.

Change-Id: Ic4df299bac83c80abf0890a81d2dd36d15993b33
8 years ago
Assaf Muller 801dedebbf Extenuate register_dhcp_agent code duplication in tests
Non-obvious changes:
* Change helpers.register_agent to use a slimmed down version
  of a plugin that knows how to register an agent. This allows
  the helper to be used with tests that do not register a core

Change-Id: Iefb1af676af6a984b01cdc1e9050541dffb5951a
8 years ago
Numan Siddique 791d57922b Fix the ImportErrors in l3 and dhcp scheduler functional tests
Change-Id: I5b8746d37173869f78a9c23834f10d630d2a36cd
Closes-bug: #1443480
8 years ago
Shivakumar M bc0b8ff8d5 DHCP Service LoadBalancing Scheduler
In this blueprint, we also propose to write a generic scheduler
framework which can be used to schedule a new resource on
selected least loaded agents.
Currently dhcp_load_type will be fetched from neutron.conf file
and corresponding load is obtained by the agent report state.
The obtained load will be populated in the "load" column of the
agents table.
During scheduling, agent will be selected based on sorting all
the agents of particular type based on load column.
Example dhcp_load_type is networks


Implements: blueprint dhcpservice-loadbalancing

Change-Id: I5ec8adf0c4336f885d603662223caa7694708876
Author: Shivakumar M <>
Co-Authored-By: Praveen Kumar SM <>
Co-Authored-By: Benjamin GRASSART <>
Co-Authored-By: Sourabh Patwardhan <>
8 years ago
Numan Siddique 690f879778 Improve test coverage of dhcp agent scheduling
This patch adds few unit test cases and functional test cases
for schedule() and auto_schedule_networks() of DHCP agent

Change-Id: I6ed7c48bcf5fc43d805c7898c7d6f019b6792e18
Closes-bug: #1331516
8 years ago