Commit Graph

24 Commits (6c9a282bcd36d744d1ad753afd8e7a219b1189c4)

Author SHA1 Message Date
Terry Wilson 1eec265ad0 Update to support the ovdsbapp 0.4.0 API
idl_factory was removed in favor of just passing in an Idl instance
as an Idl doesn't start a connection until its .run() is called.

The try/excepts will be removed when the ovsdbapp 0.4.0 constraint
changes are merged.

Change-Id: Id22faa1f6179c2fdf8a136972d65f10749c9fc2e
6 years ago
Terry Wilson e6333593ae Use the ovsdbapp library
This patch uses the ovsdbapp Python library, which is the new
project based on the Neutron OVSDB API.

The CLI implementation of the OVSDB API remains in the Neutron

Neutron continues providing the (deprecated) ability to allow
the OVSDB API to be imported from Neutron.

The deleted tests exist in the ovsdbapp project. More will be
moved later, but many of the tests in the Neutron tree use
ovs_lib, which doesn't exist in ovsdbapp so those tests will
probably stay in the Neutron tree.

Closes-Bug: #1684277
Depends-On: I3d3535b1d6fe37c78a9399903b65bbd688b1c4b9
Change-Id: Ic8c7db0e80d0ad104242322d3f1f70cab8caab92
6 years ago
Jenkins c627b2da08 Merge "Port idlutils.get_schema_helper from ovsdbapp to neutron" 6 years ago
XieYingYun 61682538d2 Remove unnecessary setUp function in testcase
In testcase, setUp will be called automatically. This patch used to
remove setUp functions that do nothing. Besides, it will keep code clean.

Change-Id: Ib9586a4c8fef171f0e83508d588b45097fa1c687
6 years ago
Dong Jun f95d33ec6e Port idlutils.get_schema_helper from ovsdbapp to neutron
There is a fatal error in networking-ovn in some case, so I port
idlutils.get_schema_helper from ovsdbapp to neutron and add some unit
tests. Networking-ovn will pass try_add_manager=False to it.

Change-Id: I12461c96f7a2845b3155fd587fb1fcc84ecf85f0
Partial-Bug: #1674889
6 years ago
Alin Balutoiu 34714b28ff Fix Python 3 compatibility in idlutils
In python 2 dict.values() would return a list while in python 3
it returns a dict_values object. The function row_by_record
tries to fetch directly the first element of the list, which
will work on python 2 but will fail on python 3 with the error
"'dict_values' object does not support indexing".

This patch addresses this issue by converting the return value
of dict.values() to list which will return the same value on
python 2 and python 3.

Change-Id: I8e3b28a84e9e5ccaeaf2d79e4547f9941472651b
6 years ago
Jenkins a4b3e28324 Merge "Refactor OVSDB native lib to be more customizable" 6 years ago
Isaku Yamahata 7d42176853 ovsdb: don't erase existing ovsdb managers
The current existing agent erases already set ovsdb managers
entries. In some use cases, cloud admin sets ovsdb managers.
eg, for SDN controllers or monitoring purpose. Neutron agent
shouldn't unconditionally erase the existing ovsdb managers.

This patch implements a new api add_manager (along with
get_manager and remove_manager) to the ovsdb api which will
allow us to configure a manager on a switch without overriding
the existing managers.

Closes-Bug: #1614766
Change-Id: Ibf9bd02fac3070d166546cac478ef984e4e43f28
Co-Authored-By: sridhargaddam <>
Co-Authored-By: Terry Wilson <>
7 years ago
Terry Wilson 2013058e13 Refactor OVSDB native lib to be more customizable
This change, in a backward-compatible way, allows users of the
native OVSDB library to pass in their own Idl factory function
where they can return any subclass of the Idl, any schema, and
and register whatever tables/columns they want. This way, we
do not have to keep up with changes to the upstream OVSDB Idl

The Connection connection, schema_name, and idl_class arguments
are deprecated in favor of the idl_factory function.

Change-Id: I8133da63caa8937c4fce68eb2fbb73d36b894043
7 years ago
Boden R e8bd6424e9 Replace retrying with tenacity
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project. Tenacity is a fork of retrying, but has improved the
interface and extensibility (see [1] for more details). Our end
goal here is removing the retrying package from our requirements.

Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- Tenacity uses seconds rather than ms as retrying did.
- Tenacity has different kwargs for the decorator and
Retrying class itself.
- Tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg.
- Tenacity defines 'time.sleep' as a default value for a kwarg.
That said consumers who need to mock patch time.sleep
need to account for this via mocking of time.sleep before
tenacity is imported.
- For retries that check a result, tenacity will raise if the retried
function raises, whereas retrying retried on all exceptions.

This patch updates all usages of retrying with tenacity.
Unit tests will be included where applicable.

Note: This change is not newton critical so projects are welcome
to hold off on committing until post-newton. Ideally this change
will merge by the first part of Ocata so dependant functionality
can land and have time to solidify for Ocata.


Closes-Bug: #1635395

Change-Id: I1c0620894d07d58efbba5226b5244fec950354ca
7 years ago
Omer Anson 4264083599 Generate OVSDB schema helper in a separate method
Move the generation of the schema helper in OVSDB connection to a
different method

In the OVSDB connection class, move the code retrieving the schema
helper to a separate method. This will allow subclasses to override this
method, and retrieve the schema helper in a different manner, or provide
it to the Connection subclass in advance.

This use-case came up in Dragonflow, where we wanted to retrieve
specific columns within specific tables, and not the whole tables.

Change-Id: I634841a2402e5d9bcbf1983a17eee2bb1a6299fb
Related-Bug: 1612403
7 years ago
Jenkins 8bf83baa2a Merge "Allow to override Idl class in OVSDB Connection" 7 years ago
Omer Anson 070ee35b9c Allow to override Idl class in OVSDB Connection
Add an option to neutron.agent.ovsdb.native.connection.Connection to
override the Idl class that is created to communicate with OVSDB.

This is a feature that would help in Dragonflow[1], where the notify
method should be overriden before start() is called, in order to receive
all events in real-time, and not retroactively.

[1] d17ae9705a/dragonflow/ovsdb/ (L135)

Change-Id: I49da05f02a00352b1b1db863d244e97f9c148804
Closes-Bug: 1627615
7 years ago
Jenkins 605cdce696 Merge "Handle uuid references within an ovsdb transaction" 7 years ago
Terry Wilson 5c09536386 Enable OVSDB Connection unit tests for py3
The OVS Python library is available for Python 3 and these tests
now run fine under Python 3, so go ahead and run them.

Change-Id: I579e2d3d3e016befafc61332bcc9147aaa385fe0
7 years ago
Terry Wilson 1f4f806584 Handle uuid references within an ovsdb transaction
Enable the OVSDB API implementations to handle referencing a newly
created object within the same transaction. For vsctl, this is via
the --id=@name construct. For native, this is via storing the
created row as the result, then replacing the result when the
transaction completes. This uses an api.Command object passed as
part of a column value in a db_set/db_create operation as a
reference to that object in a transaction. For example, do:

with self.ovsdb.transaction() as txn:
    queue = txn.add(self.ovsdb.db_create("Queue", ...))
    qos = txn.add(self.ovsdb.db_create("QoS", queues={0: queue}))
    port = txn.add(self.ovsdb.db_set("Port", pname, ('qos', qos)))

instead of having to do 5 separate transactions to: create a
queue, find the queue, create the QoS entry, find the QoS entry,
and finally to update the port with the QoS entry.

Change-Id: I1781794958af1483dabc0f5d17f2df6fea828564
Closes-Bug: #1615105
7 years ago
Oleg Bondarev a5d19b41a5 Mock threading.Thread to prevent daemon creation by unit tests
calls Connection.start() which starts a daemon with a while True loop
full of mocks. mock._CallList of those mocks start to grow very
quick and finally eat all available memory.

Closes-Bug: #1595878
Change-Id: Ie053a2248925ce5bb960207c16c23b261d1d458c
7 years ago
Jenkins 4582e64552 Merge "functional: fix OVSFW failure with native OVSDB api" 7 years ago
Inessa Vasilevskaya bffc5f062c functional: fix OVSFW failure with native OVSDB api
A bunch of functional tests fail because of non implemented
x != [] operation in idlutils.condition_match() and
wrong condition passed to db_find() in OVSFW test.
This patch addresses the issue by implementing lists
comparison in native.idlutils and fixing the call to
db_find() in OVSFW test.

A functional test for OVSDB API's db_find() has been
added to ensure that querying a list column gives the same
result both with vsctl and native ovsdb_interface; unit
test for idlutils.condition_match() with corner cases has
been added as well.

Change-Id: Ia93fb925b8814210975904a453249f15f3646855
Closes-bug: #1578233
7 years ago
Jakub Libosvar 9cc3d60d56 ovsdb: Don't skip ovsdb test cases in py34 suite
Post-2.5 ovs Python library now supports Python 3 [1] and we require
2.6dev version for py34 job, so we're fine to run ovsdb tests with
Python 3.

[1] 99c8be3ee4

Change-Id: Id87223eea45d1343a05d8fdcde08f26fe5486a30
7 years ago
Inessa Vasilevskaya dc7e2b10e8 unit: fix ValueError on TransactionQueue init with py34
With native ovsdb_interface several py34 tests fail
with ValueError during ovs_lib.OVSBridge creation
because of unbuffered text I/O in TransactionQueue
This patch fixes the issue by switching to binary mode
in os.fdopen

Change-Id: I4158cc24f6b28a290a4a4d4bec6c2383c6bf12aa
Closes-Bug: #1580270
7 years ago
Jakub Libosvar 99f2224e6b Mock out poller in ovsdb unittests
The second thread running in ovsdb connection crashed due to not passing
file descriptor to poller. This patch mocks out the poller as it's not
used in the tests to avoid false tracebacks in the logs.

Change-Id: I62e87e3248d176e5db7fe57bfb4467176ba05392
Closes-Bug: 1583069
7 years ago
lzklibj 667ea2ac7c Allow OVSDB Connection to register certain tables
Current OVSDB Connection will register all tables with schema_helper.
It doesn't matter for most cases, but for implementation for bp
routed-networks in networking-ovn, we don't need all tables in OVN_
Southbound DB are registered. We only need a certain table named
Chassis can be registered.

This patch add a parameter for OVSDB Connection to allow it to
register certain tables, instead of all tables.

Change-Id: I79df60a08a7a6c555b07a9b566d7c09b97e2463c
Closes-Bug: #1568718
7 years ago
Ihar Hrachyshka 2e1b0ea403 ovsdb: attempt to enable connection_uri for native impl on startup
The port is in most cases disabled, so to use it, we should first enable
it, and it means that we should still rely on ovs-vsctl for that initial

Closes-Bug: #1468259
Change-Id: I097b1c441df1f7f1785b8744f27809617bb21c14
8 years ago