Commit Graph

75 Commits (99f4495c940011293e3cabbb590770dc1e7b6900)

Author SHA1 Message Date
Isaku Yamahata 0063aa33c6 pecan.jsonify v1.3 adjustment
From pecan v1.3, pecan.jsonify uses json.Encoder unconditionally so that
jsonifying mock fails.
Pecan v1.2 uses simplejson.Encoder as encoder which accidentally encodes
MagicMock as {} due to check of '_asdict' attributes.
Since pecan.jsonify uses __json__ magic method when it's defined,
add __json__ method to mocks to return {} as json encode.

Closes-bug: #1754978
Change-Id: Iac955a7de2a1b66b5b56a73f5cc8a4e75150f82e
5 years ago
Isaku Yamahata 1c36c268de tests/tools: add prefix "_" to CALLBACK_PRIORITY_SUPPORTED
As it's private variable in the module, add "_" prefix
This patch is follow up patch for

Change-Id: I80aa34a000d1c1bd87cfff77af8fca56befe2542
5 years ago
Isaku Yamahata 5a1934da8e unit test: unbreak test cases for callbacks.subscribe
patch set of [1] added callback priority so that a new argument, priority,
was added to subscribe method. Thus its signature was changed.
It caused to break some unit tests.[2] Those test cases check arguments passed
to subscribe method.

The fix to those test cases needs to be compatible with older version of
So add a helper method, get_subscribe_args, to produce argument to subscribe
by checking if neutron-lib supports callback priority or not. and apply it
to broken test cases.


Change-Id: I94066ff8a8dae5c5637478aa52883b3b451f2857
5 years ago
Van Hung Pham 9f486a2f33 Router to OVO
This patch introduces the Router model class to Oslo-Versioned Object
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db

Change-Id: I5e0bef20aed884db7694286a398e10faff3b057e
6 years ago
Boden R 9e67ba5052 use core resource api defs from lib
The core resource API definitions are in neutron-lib and are already
setup in a RESOURCES map, similar to neutron's global
RESOURCE_ATTRIBUTE_MAP. However, a number of consumers directly use
RESOURCE_ATTRIBUTE_MAP and moreover can perform some interesting
operations on it while mockin" for tests. For that reason this
patch proposes we phase in the use of neutron's RESOURCES map rather
than a rip and replace of RESOURCE_ATTRIBUTE_MAP.

This patch removes the API definitions for the core neutron resources
by updating neutron's global RESOURCE_ATTRIBUTE_MAP to reference lib's
RESOURCE map. In addition the AttributeMapMemento class is removed; it
no longer servers a purpose and neutron-lib's fixture should be used
in its place.

This patch should be safe to merge without worry of impacting consumers.
No one is using AttributeMapMemento [1] and the global map only changed
its reference value.


Change-Id: Ib04fddcbd4465074452b71b16befa4e33b27259e
6 years ago
Zuul b520b16d8a Merge "use plugin constants from neutron-lib" 6 years ago
Boden R 95f1e03446 use plugin constants from neutron-lib
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.


Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
6 years ago
Boden R 0822b0aef4 consume common constants from lib
The neutron-lib commit I360545b6ee4291547e0c5c8e668ad03d3efa4725 moved
the externally consumed globals from neutron.common.constants into lib.
With the exception of PROVISIONAL_IPV6_PD_PREFIX all other constants
in neutron.common.constants should only be used in neutron, and will
hopefully remain that way. External consumers needing access to other
common constants should move them into lib first.


Change-Id: Ie4bcffccf626a6e1de84af01f3487feb825f8b65
6 years ago
Brian Haley d0b68181a0 Remove deprecated get_random_mac()
get_random_mac() was marked for deprecation in
Queens, remove it and use the one from neutron-lib.

Change-Id: Ibcd3831a9ed419355a3d7969bd256b4b4bab6173
6 years ago
sindhudevale c761a08473 OVO for L3HARouter
This patch introduces and integrates OVO for L3 HA Router.

Co-Authored-By: Nguyen Phuong An <>
Co-Authored-By: Vu Cong Tuan <>
Change-Id: I3463921dec415dd073503ab9470588193d08ce87
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
6 years ago
Lujin 310cbdd250 FloatingIP to OVO
This patch introduces the FloatingIP model class to Oslo-Versioned Object.

Change-Id: I4257ded4a4b0ce0ae3f562cc74b55b671fe4af69
6 years ago
Boden R d63032b547 use APIExtensionDescriptor for plugins with defs in lib
A few plugins that have API defs in neutron-lib still inherit from
ExtensionDescriptor and thus return their def's values. This patch
changes them to inherit from APIExtensionDescriptor which handles
returning these values for you.

In addition neutron-lib's APIDefinitionFixture is used in the
AttributeMapMemento to ensure API def maps are handled accordingly.

Change-Id: Ibf2e8367be65b7bb15359d81286883540b566e92
6 years ago
Nguyen Phuong An 913c9e78b9 [log]: implement logging plugin
This patch introduces the logging api definition and initial
implementation of LoggingApiPlugin. The api definition code will
be removed after [1] has been merged on neutron lib.


Co-Authored-By: Yushiro FURUKAWA <>

Partially-implements: blueprint security-group-logging
Related-Bug: #1468366
Change-Id: Iace31506502de25da9dce5fcfdbfe2c726bea27f
6 years ago
Nguyen Phuong An ae791696b8 [log]: db models and migration rules
This patch includes db models and migration rules for initial
logging object. The implementation bases on logging api for
security group spec[1]


Partially-implements: blueprint security-group-logging
Related-Bug: #1468366

Change-Id: I3f91b3927c33021facc9eb9238555c0e06a918c0
6 years ago
Jenkins 84f9ad6bfe Merge "Replace six.iteritems with dict.items(Part-1)" 6 years ago
fpxie d2976d46d0 Replace six.iteritems with dict.items(Part-1)
according to,
now we should avoid using six.iteritems and replace
it with dict.items.

Change-Id: I8753e80b34c0f86cf70aebc3bcbd3392ee933f62
Partial-Bug: #1680761
6 years ago
Henry Gessau 844033a5d1 Eliminate lookup of model query hooks by name
By registering functions directly we cut off the dependency of the
query hooks on the plugin. This is a step towards the goal of
removing the CommonDbMixin mixin class.

Also, we register all query hooks at plugin create (in __new__)
instead of in the class definition (which caused the hooks to be
registered on import). This ensures the query hooks are only
registered for the plugins/mixins that are actually used.

Since the query hooks are decoupled from the plugin, we remove them
from the extension mixins (make them global in their module). This is
a step towards refactoring all extension mixins for removal.

Extra: In this patch we also remove the CommonDbMixinHooksFixture test
fixture and instead just clear out the hooks after each test.

Related-Blueprint: neutron-lib

Change-Id: Ib6c2134d29e1764de627c3355f6cdee789d6301e
6 years ago
Henry Gessau e7cd868c20 Decouple hook and func registration from CommonDbMixin
Move the model query hook registration and resource extend funcs
registration methods out of the CommonDbMixin class and make them
regular utility functions.

This is a step in refactoring the CommonDbMixin class.

Change-Id: Iec1bb7f7098c83640ae695fd7cf2f4736f414ad2
6 years ago
Trevor McCasland ae3b344be3 use neutron_lib's get_random_mac
Neutron-lib 1.1.0 is now out and contains the get_random_mac
definition[1]. This patch moves neutron references over to
the neutron-lib version.


[1] ee0f5b2ab27c828cfedb771735d237a968423da2

Change-Id: I28a2a1d85a85461f7a4344b86d18da7f68066c95
6 years ago
Sławek Kapłoński afab7c6701 Display deprecation warning in test only once
WarningsFixture class was set to display deprecation
warnings "alway" - which means every occurence of
a warning.
Since oslo.context 2.12.0 there was huge amount of such
warnings displayed. It causes functional tests fails due
to reach global timeout for tests.

Now deprecation warning should be disaplayed only on first
occurence of matching warning in test.

Change-Id: I91196a8457c25e6eb5561fb74fd5864de85a070a
Closes-Bug: 1660088
6 years ago
Jenkins 74fcf15631 Merge "Use weakrefs for common_db_mixin callbacks" 6 years ago
Anindita Das 7872508f9f [Live Migration] Extend ml2_port_binding table
Extended the Primary Key of ml2_port_binding table to include "hosts"
and added a new column named "status" to indicate the status of the
port similar to the ml2_distributed_port_binding table. Updated the
PortBinding object with the above changes.

Change-Id: Ie01f83b01ab673b9ae7e05b4eaed504ffc8db773
Co-Authored-By: Brian Stajkowski <>
Partial-Bug: #1580880
6 years ago
Kevin Benton faf80b950f Use weakrefs for common_db_mixin callbacks
This will prevent the common_db_mixin dictionary
extension functions and query hooks from stopping
the GC of plugins in tests and causing resource leaks.

Change-Id: I7576851a44abd14cbc337a3d3e28690c7316ec81
6 years ago
Jenkins 9ed660af9e Merge "OVO for Ipam(Allocation,Subnet,Pool)" 6 years ago
Kevin Benton e834986084 Allow 0 in port range for securitygrouprule object
We allow 0 in port_range_min and port_range_max for the
non TCP/UDP use cases (e.g. ICMP). This adjusts the securitygrouprule
OVO object to use a new PortRangeWith0Field to allow 0.

Change-Id: I00f7a91202ccfcad1b8edb8759983332c6e79791
6 years ago
Nakul Dahiwade 89f6dd3305 OVO for Ipam(Allocation,Subnet,Pool)
This patch introduces and integrates Oslo-Versioned Objects for
IpamAllocationPool,IpamAllocation and IpamSubnet model classes.

Change-Id: Ib7d86dd10e92b5e2baeef0e69620c8688859fe7d
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Co-Authored-By: Anindita Das <>
Co-Authored-By: Mohit Malik <>
Co-Authored-By: Victor Morales <>
6 years ago
Gary Kotton 71fac629e2 Use get_random_string from neutron-lib
Use the method provided by neutron-lib and delete the duplicated
neutron one.


Change-Id: I3a62d8f58862ab978102ac205b1359424e225eeb
7 years ago
Anindita Das 640618b6c3 OVO for VlanAllocation
This patch introduces OVO for VlanAllocation.
SegmentTypeDriver integration will be required during the integration of
this patch, therefore it will be handled in

Change-Id: I8ad7e74ec74bf91b2cdf9d8aed231074ef252e71
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
7 years ago
Victor Morales 96b8d9a9af Agent to OVO
This patch introduces Agent model class to Oslo Versioned Object.

Given the number of dependencies and methods that use the model class,
the integration patch will be created in separated patch.

Co-Authored-By: Tony Tan <>

Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I4145b7f59b58a55c1a76b91618097ff69aee1568
7 years ago
Dariusz Smigiel 6bb12ef128 Removed deprecated function import_modules_recursively
Change Ia8d91a1704c894bc1f6cf14e6cdd971fab255b62 moved function
import_modules_recursively from to
Please use appropriate function path.

Change-Id: I7a178ceff5bbc4b6f45cfc040232148090bce135
Related-Bug: #1612959
7 years ago
Gary Kotton 396abb8ccb neutron-lib: start using neutron-lib helpers
The following helpers are used:

Change-Id: I5947b473330fd29f8d4c1a08f03d007a52c8dfd9
7 years ago
Artur Korzeniewski 94ee8bc306 Introduce ovo objects for networks
Those objects are intentionally not integrated into the database code so
far. This is to quicken access to their definitions to implement
push-notifications for networks.

Segments are part of the network object. Since we need to reduce the
number of SQL queries executed per resource, and we want to include
segmentation information for networks, Network model was extended with
segments relationship that makes the information available on every
network resource fetch from the database. This change required some
change in test_dhcp_agent_scheduler to expire a session used in the
tests to avoid obsolete segment state to be reused to validate

This implementation of the object is not complete for the job of
updating the resource in database. For example, tags are not yet exposed
on the object; also attributes like availability_zones, or external
network attributes, are not covered. Those attributes are hopefully
needed on server side only, so until we adopt the object for server
side, it should be ok to live without them.

Another database related thing still missing in this patch is lack of
RBAC support for the object. To complete this support,
a real get_bound_tenant_ids should be put onto the object, instead of
the current no-op stub.

This patch also includes some rearrangements that simplified the work.
Specifically, all network related objects are consolidated in the module, instead of being scattered through the
code base. Also, some setup code from test_policy relevant to RBAC was
moved into the base test class so that it can be utilized by other RBAC
enabled objects, like network.

Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Partially-Implements: blueprint push-notifications

Co-Authored-By: Victor Morales <>
Change-Id: I5160d0ab9e8042c356229420739db0ce42842368
7 years ago
ChangBo Guo(gcb) 7913798a2c Use method get_ipv6_addr_by_EUI64 from oslo.utils
Oslo.utils provides same method get_ipv6_addr_by_EUI64, let's use
it and deprecate method get_ipv6_addr_by_EUI64 in neutron.

Closes-Bug: #1597213
Change-Id: If1eda1283b4405e13ccb60dab530ce4cc1b22948
7 years ago
Gary Kotton 9f09f27c5d Fix deprecation warnings
Remove deprecation warnings for various constants
and exceptions that have moved to neutron_lib.

Fix miscellaneous other deprecations.

Uses constants instead of l3_constants when importing
neutron-lib constants.

Co-Authored By: Henry Gessau <>
Co-Authored By: Gary Kotton <>

Change-Id: Ib0e8ff5c3e23677c1009241a1818cbc8a3430c38
7 years ago
Ihar Hrachyshka 5925364c9a Fixed neutron-db-manage without neutron/tests installed
With 7c0f189309 in tree, we made
neutron-db-manage require neutron/tests/ installed. Some
distributions, like RDO, may split the python package into core package
and the one that contains python code needed for testing only (anything
under neutron/tests/), and hence don't guarantee that all setups have
neutron.tests package available.

This fix moves the import_module_recursively function from into neutron.common.utils because it has non-testing
use cases. All existing cases where we use the function switched to the
new location. The old symbol still works, though triggers a deprecation
warning, and will be removed in the next cycle.

Change-Id: Ia8d91a1704c894bc1f6cf14e6cdd971fab255b62
Closes-Bug: #1612959
7 years ago
Ihar Hrachyshka cb64ad10ad Introduce ovo objects for security groups
Those objects are intentionally not integrated into the database code so
far. This is to quicken access to their definitions to implement
push-notifications for security groups and ports.

Expanded the list of possible values for IpProtocolEnumField to allow
port numbers in addition to symbolic names since security group API
supports both.

This patch also makes DeclarativeObject not override extra filters on
class creation and instead reuse the list of filters defined on the
class as an attribute. This allows to define it per class not per object
(in __init__).

Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Partially-Implements: blueprint push-notifications

Change-Id: Idcdc928d37d5d661428525700001154f50c3056e
Co-Authored-By: Sayali Lunkad <>
Co-Authored-By: Victor Morales <>
7 years ago
Jakub Libosvar 766abb752a Make pep8 job great again
There is a bug in pep8, when 'select' used, it omits all default checks
and runs only those specified by 'select'.  We got hit by this issue
since I2d26534230ffe5d01aa0aab6ec902f81cfba774d was merged which lead to
almost no static checks in pep8 job.

Also note that off_by_default decorator has no effect for now because
factory in hacking is triggered after ignored checks are collected.
There will be a follow-up patch for that in order to make pep8 doing
its job quickly.


Related-Bug: 1594756
Change-Id: I8e27f40908e1bb4307cc7c893169a9d99f3433c4
7 years ago
Jakub Libosvar 3ead5f959e objects: Add filter query hook to NeutronDbObject
Extensions add query hooks to db models. This patch adds support for
filtering hooks for objects, which means we're able to register new name
that can be used for filtering when getting an object from database.

Change-Id: I106a21efe42f848ee18cb8349fe89afd404d25b1
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
7 years ago
Dustin Lundquist ae613a0157 Use IPv[46]_BITS instead of IPV[46]_MAX_PREFIXLEN
Replace references to IPV[46]_MAX_PREFIXLEN with IPv[46]_BITS from
neutron-lib. Replace several integer literals used to represent IP
address sizes with these constants too.

Remove IPV4_MAX_PREFIXLEN and IPV6_MAX_PREFIXLEN from neutron constants
as they are no longer referenced:[46]_MAX_PREFIXLEN

Change-Id: I03e1405e71f08db9ac6e759258625139c28ecc89
7 years ago
Jenkins e0c2122222 Merge "Use unittest2 uniformly across Neutron" 7 years ago
Martin Hickey afaad48aa3 Use unittest2 uniformly across Neutron
unittest2 contains bug fixes to unittest for all versions of Python<3.5.
Move unittest references to unittest2.

Bumped hacking requirement to reflect the version in
openstack/requirements that is the minimal version that supports
off_by_default checks.

Change-Id: I2d26534230ffe5d01aa0aab6ec902f81cfba774d
Depends-On: I883223962426ed8e9cdf4e304dfed17ef0946c8a
7 years ago
Artur Korzeniewski 563fea3786 Introducing the Subnet OVO
This patch is introducing the following OVOs:
- DNSNameServer
- Route
- IPAllocation
- Subnet

Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I3fca197098f24f68ea2b2915851c68dc170e781c
7 years ago
Jenkins 6b2de646e8 Merge "Tests: correcting typo in get_random_cidr." 7 years ago
Artur Korzeniewski 6ffa41185e Tests: correcting typo in get_random_cidr.
Adding IPv6 test coverage for Neutron object

Change-Id: I912f88b9dabdd58d687e894405ae5dea2a6c24e0
7 years ago
rossella 45b9960fdd Port allowed address pairs to OVO
This patch introduces Oslo VersionedObject for the
port extension allowed address pairs.

Implements: blueprint adopt-oslo-versioned-objects-for-db

Change-Id: I8d03528f8f45f5f50fa467b39245a513a37c5d89
Co-Authored-By: Martin Hickey <>
7 years ago
Martin Hickey b8cd05273f Add custom OVO field type for MAC address
Partial-Bug: #1541928
Co-Authored-By: Ihar Hrachyshka <>
Change-Id: If791b25aa673dde32b41c799fad9dc36efc87d04
7 years ago
Victor Morales 408173360f objects: SubnetPool, SubnetPoolPrefix
This patch adds SubnetPool and SubnetPoolPrefix objects. Note: objects
are not integrated into database plugin code. This is left for a
follow-up patch.

Other changes:
- made modify_fields_to_db object instance method since some objects
  may need to access all fields set on the object;
- provided a new RangeConstrainedInteger type that validates that value
  is in the range;
- modified several base test cases to work with objects that modify
  other tables that are different from the object db model.

Co-Authored-By: Ihar Hrachyshka <>
Related-Bug: #1541928
Change-Id: I7cbc694ab00c05d0a020fffe4f73141c6ceff7e9
7 years ago
Ihar Hrachyshka 58ba6586b7 tests: register all objects before validating their hash versions
The test validates that all object types registered in the object
registry have expected object hashes. If they are not, it means that
object API is changed, and it is used as an indicator to reviewers that
object version conversion rules should be added to allow upgrading and
downgrading objects between older and the new versions.

The test was not importing all modules that contain objects though, so
if no other code executed by the current testr process before the hasher
validation test imported all objects to validate, the test would
misbehave, claiming some expected object types not registered at all.

To make sure all object types available in the tree are imported (and
registered) before the test case is executed, we need to import all
modules under neutron.objects. Instead of maintaining the list of
modules with objects to import somewhere, inspect the list of those
modules dynamically, assuming they are all located under
neutron/objects/ subtree.

Without the fix, the test may randomly fail depending on test case
order for the current process.

Note it's not an issue for QoS objects since they are implicitly
imported by rpc callbacks resource manager that is initialized in the
base test class. This becomes a problem when you start to introduce
objects that are not part of rpc callbacks list of supported resources.

Change-Id: Ice408faf10b75c508b9c5f5b7ab23b2fc3289eaa
7 years ago
Saisrikiran Mudigonda 487ad1dea4 Add custom SQLAlchemy type for MACAddress
Related-Bug: #277558
Change-Id: I5aed64b083fb848cb16bd34b1ca167a264531e03
7 years ago
Artur Korzeniewski 4f995c3cb0 Add custom SQLAlchemy type for CIDR
Change-Id: I8e67012179d9cc6d27a2fdf47477857609c4c856
7 years ago