idl_factory was removed in favor of just passing in an Idl instance
as an Idl doesn't start a connection until its .run() is called.
The try/excepts will be removed when the ovsdbapp 0.4.0 constraint
changes are merged.
Change-Id: Id22faa1f6179c2fdf8a136972d65f10749c9fc2e
This patch uses the ovsdbapp Python library, which is the new
project based on the Neutron OVSDB API.
The CLI implementation of the OVSDB API remains in the Neutron
tree.
Neutron continues providing the (deprecated) ability to allow
the OVSDB API to be imported from Neutron.
The deleted tests exist in the ovsdbapp project. More will be
moved later, but many of the tests in the Neutron tree use
ovs_lib, which doesn't exist in ovsdbapp so those tests will
probably stay in the Neutron tree.
Closes-Bug: #1684277
Depends-On: I3d3535b1d6fe37c78a9399903b65bbd688b1c4b9
Change-Id: Ic8c7db0e80d0ad104242322d3f1f70cab8caab92
In testcase, setUp will be called automatically. This patch used to
remove setUp functions that do nothing. Besides, it will keep code clean.
Change-Id: Ib9586a4c8fef171f0e83508d588b45097fa1c687
There is a fatal error in networking-ovn in some case, so I port
idlutils.get_schema_helper from ovsdbapp to neutron and add some unit
tests. Networking-ovn will pass try_add_manager=False to it.
Change-Id: I12461c96f7a2845b3155fd587fb1fcc84ecf85f0
Partial-Bug: #1674889
In python 2 dict.values() would return a list while in python 3
it returns a dict_values object. The function row_by_record
tries to fetch directly the first element of the list, which
will work on python 2 but will fail on python 3 with the error
"'dict_values' object does not support indexing".
This patch addresses this issue by converting the return value
of dict.values() to list which will return the same value on
python 2 and python 3.
Change-Id: I8e3b28a84e9e5ccaeaf2d79e4547f9941472651b
The patch introduces new abstract method to the API abstract class. The
method is supposed to return a new Transaction object. Each
API object is capable of store one nested transaction which is returned
by context manager in case some transaction already exists.
As there are no projects in OpenStack that use inheritance directly from
API abstract class, it's safe to make this new create_transaction() abstract method.
Only projects that currenlty use ovsdb API are networking-ovn,
dragonflow and networking-l2gw. OVN and Dragonflow use only IDL
implementation and L2GW copies the code of API abstract class.
Closes-bug 1653517
Change-Id: I55dd417cae7ebbe0668ba5606949ce4ab045d251
The current existing agent erases already set ovsdb managers
entries. In some use cases, cloud admin sets ovsdb managers.
eg, for SDN controllers or monitoring purpose. Neutron agent
shouldn't unconditionally erase the existing ovsdb managers.
This patch implements a new api add_manager (along with
get_manager and remove_manager) to the ovsdb api which will
allow us to configure a manager on a switch without overriding
the existing managers.
Closes-Bug: #1614766
Change-Id: Ibf9bd02fac3070d166546cac478ef984e4e43f28
Co-Authored-By: sridhargaddam <sgaddam@redhat.com>
Co-Authored-By: Terry Wilson <twilson@redhat.com>
This change, in a backward-compatible way, allows users of the
native OVSDB library to pass in their own Idl factory function
where they can return any subclass of the Idl, any schema, and
and register whatever tables/columns they want. This way, we
do not have to keep up with changes to the upstream OVSDB Idl
library.
The Connection connection, schema_name, and idl_class arguments
are deprecated in favor of the idl_factory function.
Change-Id: I8133da63caa8937c4fce68eb2fbb73d36b894043
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project. Tenacity is a fork of retrying, but has improved the
interface and extensibility (see [1] for more details). Our end
goal here is removing the retrying package from our requirements.
Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- Tenacity uses seconds rather than ms as retrying did.
- Tenacity has different kwargs for the decorator and
Retrying class itself.
- Tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg.
- Tenacity defines 'time.sleep' as a default value for a kwarg.
That said consumers who need to mock patch time.sleep
need to account for this via mocking of time.sleep before
tenacity is imported.
- For retries that check a result, tenacity will raise if the retried
function raises, whereas retrying retried on all exceptions.
This patch updates all usages of retrying with tenacity.
Unit tests will be included where applicable.
Note: This change is not newton critical so projects are welcome
to hold off on committing until post-newton. Ideally this change
will merge by the first part of Ocata so dependant functionality
can land and have time to solidify for Ocata.
[1] https://github.com/jd/tenacity
Closes-Bug: #1635395
Change-Id: I1c0620894d07d58efbba5226b5244fec950354ca
Move the generation of the schema helper in OVSDB connection to a
different method
In the OVSDB connection class, move the code retrieving the schema
helper to a separate method. This will allow subclasses to override this
method, and retrieve the schema helper in a different manner, or provide
it to the Connection subclass in advance.
This use-case came up in Dragonflow, where we wanted to retrieve
specific columns within specific tables, and not the whole tables.
Change-Id: I634841a2402e5d9bcbf1983a17eee2bb1a6299fb
Related-Bug: 1612403
Add an option to neutron.agent.ovsdb.native.connection.Connection to
override the Idl class that is created to communicate with OVSDB.
This is a feature that would help in Dragonflow[1], where the notify
method should be overriden before start() is called, in order to receive
all events in real-time, and not retroactively.
[1] d17ae9705a/dragonflow/ovsdb/impl_idl.py (L135)
Change-Id: I49da05f02a00352b1b1db863d244e97f9c148804
Closes-Bug: 1627615
The OVS Python library is available for Python 3 and these tests
now run fine under Python 3, so go ahead and run them.
Change-Id: I579e2d3d3e016befafc61332bcc9147aaa385fe0
Enable the OVSDB API implementations to handle referencing a newly
created object within the same transaction. For vsctl, this is via
the --id=@name construct. For native, this is via storing the
created row as the result, then replacing the result when the
transaction completes. This uses an api.Command object passed as
part of a column value in a db_set/db_create operation as a
reference to that object in a transaction. For example, do:
with self.ovsdb.transaction() as txn:
queue = txn.add(self.ovsdb.db_create("Queue", ...))
qos = txn.add(self.ovsdb.db_create("QoS", queues={0: queue}))
port = txn.add(self.ovsdb.db_set("Port", pname, ('qos', qos)))
instead of having to do 5 separate transactions to: create a
queue, find the queue, create the QoS entry, find the QoS entry,
and finally to update the port with the QoS entry.
Change-Id: I1781794958af1483dabc0f5d17f2df6fea828564
Closes-Bug: #1615105
ovs-vsctl, unless --no-wait is passed, will wait until ovs-vswitchd
has reacted to a successful transaction. This patch implements
the same logic, waiting for next_cfg to be incremented and checking
that any added interfaces have actually been assigned ofports.
Closes-Bug: #1604816
Closes-Bug: #1604370
Related-Bug: #1604115
Change-Id: I638b82c13394f150c0bd23301285bd3375e66139
tests.unit.agent.ovsdb.native.test_connection.TestOVSNativeConnection
calls Connection.start() which starts a daemon with a while True loop
full of mocks. mock._CallList of those mocks start to grow very
quick and finally eat all available memory.
Closes-Bug: #1595878
Change-Id: Ie053a2248925ce5bb960207c16c23b261d1d458c
A bunch of functional tests fail because of non implemented
x != [] operation in idlutils.condition_match() and
wrong condition passed to db_find() in OVSFW test.
This patch addresses the issue by implementing lists
comparison in native.idlutils and fixing the call to
db_find() in OVSFW test.
A functional test for OVSDB API's db_find() has been
added to ensure that querying a list column gives the same
result both with vsctl and native ovsdb_interface; unit
test for idlutils.condition_match() with corner cases has
been added as well.
Change-Id: Ia93fb925b8814210975904a453249f15f3646855
Closes-bug: #1578233
Post-2.5 ovs Python library now supports Python 3 [1] and we require
2.6dev version for py34 job, so we're fine to run ovsdb tests with
Python 3.
[1] 99c8be3ee4
Change-Id: Id87223eea45d1343a05d8fdcde08f26fe5486a30
Poller.block() calls select() on defined file descriptors. This patch
adds timeout to select() so it doesn't get stuck in case no fd is ready.
Also timeout was added when reading transaction results from queue.
Closes-Bug: 1567668
Change-Id: I7dbddd01409430ce93d8c23f04f02c46fb2a68c4
With native ovsdb_interface several py34 tests fail
with ValueError during ovs_lib.OVSBridge creation
because of unbuffered text I/O in TransactionQueue
initialization.
This patch fixes the issue by switching to binary mode
in os.fdopen
Change-Id: I4158cc24f6b28a290a4a4d4bec6c2383c6bf12aa
Closes-Bug: #1580270
The second thread running in ovsdb connection crashed due to not passing
file descriptor to poller. This patch mocks out the poller as it's not
used in the tests to avoid false tracebacks in the logs.
Change-Id: I62e87e3248d176e5db7fe57bfb4467176ba05392
Closes-Bug: 1583069
Current OVSDB Connection will register all tables with schema_helper.
It doesn't matter for most cases, but for implementation for bp
routed-networks in networking-ovn, we don't need all tables in OVN_
Southbound DB are registered. We only need a certain table named
Chassis can be registered.
This patch add a parameter for OVSDB Connection to allow it to
register certain tables, instead of all tables.
Change-Id: I79df60a08a7a6c555b07a9b566d7c09b97e2463c
Closes-Bug: #1568718
The port is in most cases disabled, so to use it, we should first enable
it, and it means that we should still rely on ovs-vsctl for that initial
call.
Closes-Bug: #1468259
Change-Id: I097b1c441df1f7f1785b8744f27809617bb21c14
Terry Wilson