Using nova-api in the wild shows that we've got an additional
exception, that must be treated like a deadlock, so that we
can handle this error properly and retry the transaction.
Closes-Bug: #1394298
Change-Id: If75f2b5984979fe55ad04ccec8713989307c56ae
We can get 'PRIMARY KEY must be unique' error on some
platforms and versions of sqlite library while trying to
insert the row with the same primary key that already exist.
In this case oslo.db should raise DBDuplicateEntry error.
Add corresponding filter to _sqlite_dupe_key_error
Closes-Bug: #1386145
Change-Id: Ifafd6a8e0b613a31e596043071aef4d410a976f2
is_backend_avail() helper function calls _ensure_backenv_available()
method,which creates a SQLAlchemy engine and opens a test connection,
but doesn't call engine.dispose(). Depending on Python interpreter
version used, the connection in the pool may remain open for some
time (even though, we don't store a reference to an Engine instance).
Closes-Bug: #1393633
Co-Authored-By: Victor Sergeyev <vsergeyev@mirantis.com>
Change-Id: I0297af709ed18fed52308500c13054cb49adf401
Add pbr to the list of installation requirements so that it is installed
via pip before this library is installed, instead of with easy_install.
This avoids issues like Bug #1384919, and ensures that projects that use
this library as a dependency are properly installed.
Related-Bug: #1384919
Change-Id: I6c155370dbd01fe4748d5137bdf288e8d3e1a67e
Remove the specification in tox.ini that _ is a builtin so that
it will no longer assume that _ does not need to be imported.
This helps ensure that the _ from i18n is used.
Activating this check did not flag any violations.
Change-Id: Idc4d6259d4eb2db66e7122535bc42f8647049a06
The tests/sqlalchemy/test_exc_filters.py tests were asserting
the exact string format of SQLAlchemy exceptions.
SQLAlchemy is improving its exception format in version 1.0,
thereby breaking these tests. Instead of supporting multiple
string formats, change the assertions to compare specific
fields per the oslo.db.exceptions and sqlalchemy.exc APIs.
Change-Id: Ie98ac3ca82a1052c463ca9ca5d41e228935497b3
In python3.x the 'de' variable will be removed from the
scope after the except block exits (if it ever is entered)
so we need to use a different variable name to ensure that
it will not be deleted so we can use it later.
This avoids errors of the format:
UnboundLocalError: local variable 'de' referenced before
assignment
Change-Id: I9ea2bd6b3a8c392f8d27b0130dd6b7683fc44e7c
This change is the first part in a series of changes
that will allow for full flexibility in database usage
during tests. The first step is to add more facility
to the oslo.db.sqlalchemy.provision system, implementing
a dispatch system that allows flexibility in creation
and dropping of databases, as well as moving the awareness
of the "openstack_citest" convention into provisioning.
The OpportunisticFixture and OpportunisticTestCase now
fold into DbFixture and DbTestCase, which defers in a simple
way to provision.ProvisionedDatabase for all connectivity.
ProvisionedDatabase in turn decides based on the given
environment as to how an engine should be provisioned
for a given test.
Control of database connectivity remains via the
OS_TEST_DBAPI_ADMIN_CONNECTION environment variable. When not
set, connectivity defaults to sqlite://, plus those backends
found to be available using "opportunistic" naming conventions.
When the variable is present, it provides a semicolon-delimited
list of URLs, and only those URLs will be used for initial
connectivity.
Future changes will allow provisioning to hold onto a single
database engine per test run, as well as allow a single
test class to run in multiple backend scenarios (e.g. one test
against many different backends).
Change-Id: Ifc02505c4c8ebd4a1ca56e14f76c0989576875c3
Partial-Bug: #1339206
Currently, it's enforced in devstack through setting encoding in
connection URI. It would be beneficial if oslo.db handles that on its
own, so that we can
- remove those connection arguments from devstack;
- more importantly, not rely on users to set it in all affected
projects.
Oursql and MySQLdb also require use_unicode=0 to avoid performance drop.
Change-Id: I676c9c5e418905160e4ab647b8a0f2fb601e8962
Closes-Bug: 1340779
Add method that checks that foreign keys in models are synchonized
with foreign keys that are created in database.
Change-Id: I4a776da0f53a79218ed4b111cac33b37d264fa1b
This function is currently otherwise unused and does not
work correctly on SQLite, not only dropping the
target unique constraint but instead dropping all unique
constraints from the target table prior to SQLAlchemy
version 1.0 (unreleased). The tests miss that it isn't
working as well unless SQLAlchemy 1.0 is used, in which
case they fail.
Change-Id: Icaff621fba4df289247e96b87c439c62e91543d6
Closes-bug: #1377646
Use import_module instead of try_import so that any import exception
includes a full stack trace and the original error message.
Change-Id: I64449688f22a224b590807317472f26e64f377a3
Before return create_engine() tests the connectivity to a DB. At this
moment a DB might be unavailable for some reason, in which case
create_engine() retries a few times. When used with MySQL, we didn't
actually retry, but instead failed on the first connectivity error
when trying to get the value of sql_mode session variable.
Closes-Bug: #1376211
Co-Authored-By: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: I14e25cfe1ed51b0d51a94e491b7267f26e42d34e
The "reconnect" listener that occurs on transaction begin
is not sufficient for libraries that use engines, connections
and sessions in "autocommit" mode, where no transaction is begun.
So we move our "ping" from the begin phase to the "connect"
phase instead. We use a newer SQLAlchemy event "engine_connect"
so that we can make use of the Connection and all the disconnect/
invalidation facilities that are included, rather than using
the low-level pool listener which is somewhat legacy.
Since the event was added in SQLAlchemy 0.9.0, a compatibility
layer is added which emulates the mechanics of the
handle_error compatibility layer.
Change-Id: I719fe45e657e747e5f8d64f20059e2b61c79b63d
Closes-bug: #1374497
The contextlib.nested function doesn't exist on py3.x so
in order to retain similar functionality create an equivalent
that will perform good enough for oslo.db usage.
Change-Id: I326816c9301f1c625e1e2723d4d639fd81cc596c
Get a tox environment for py34/py33 that doesn't stop the existing
py26 and py27 to continue working with the adjusted test-requirements since
MySQL-python and sqlalchemy-migrate do not yet exist in 3.x.
Note that this does not fully make py34 work yet but it starts to work
in this direction of getting it to work (which will apparently require
quite a big more tweaking).
Change-Id: I024957a76f2de0bd406d14da5af6e4caf3d493e7
Six is used directly in oslo.db (and is not just a transitive
dependency) so it should be listed in the requirements file
just like other non-transitive dependencies.
Change-Id: I81d2d7d0db9142bace7ecf8882efbbcd605fb527
The DialectFunctionDispatcher.dispatch_for() decorator method
necessarily returns the dispatcher itself and not the
decorated function, so that the object can continue to be
re-used even if the function name is the same as that of the
dispatcher. In order to support a single function being
wrapped by the dispatcher multiple times with different
criteria, dispatch_for() will now check for the last function
wrapped and use that.
Change-Id: I331670d9b76ae30e7a666648e7e2d4c72641c9ff
Closes-Bug: #1373568