489 Commits

Author SHA1 Message Date
Jenkins
f7489fe52c Merge "Fix python3.x scoping issues with removed 'de' variable" 2014-11-19 18:53:25 +00:00
Jenkins
e682f3ed2a Merge "Ensure is_backend_avail() doesn't leave open connections" 2014-11-19 18:37:06 +00:00
Roman Podoliaka
2269848495 Handle Galera deadlock on SELECT FOR UPDATE
Using nova-api in the wild shows that we've got an additional
exception, that must be treated like a deadlock, so that we
can handle this error properly and retry the transaction.

Closes-Bug: #1394298

Change-Id: If75f2b5984979fe55ad04ccec8713989307c56ae
2014-11-19 19:46:04 +02:00
Oleksii Chuprykov
4b2058b8c1 Add exception filter for _sqlite_dupe_key_error
We can get 'PRIMARY KEY must be unique' error on some
platforms and versions of sqlite library while trying to
insert the row with the same primary key that already exist.
In this case oslo.db should raise DBDuplicateEntry error.
Add corresponding filter to _sqlite_dupe_key_error

Closes-Bug: #1386145

Change-Id: Ifafd6a8e0b613a31e596043071aef4d410a976f2
2014-11-19 16:02:42 +02:00
Victor Sergeyev
b6d363def0 Add info on how to run unit tests
Change-Id: I7178fb2ca6e17d7428a44ec2cefe0634bb134634
2014-11-19 15:21:20 +02:00
Jenkins
027bdcb303 Merge "Add pbr to installation requirements" 2014-11-19 13:15:58 +00:00
Roman Podoliaka
7f755bf6fe Ensure is_backend_avail() doesn't leave open connections
is_backend_avail() helper function calls _ensure_backenv_available()
method,which creates a SQLAlchemy engine and opens a test connection,
but doesn't call engine.dispose(). Depending on Python interpreter
version used, the connection in the pool may remain open for some
time (even though, we don't store a reference to an Engine instance).

Closes-Bug: #1393633

Co-Authored-By: Victor Sergeyev <vsergeyev@mirantis.com>
Change-Id: I0297af709ed18fed52308500c13054cb49adf401
2014-11-18 16:05:01 +02:00
OpenStack Proposal Bot
c54d3a95ed Updated from global requirements
Change-Id: Ic80266ad328243a91688bc573be7e49d026c4b1f
2014-11-18 11:36:12 +00:00
Jenkins
58cfd8262e Merge "Assert exceptions based on API, not string messages" 2014-11-14 17:25:39 +00:00
OpenStack Proposal Bot
1b0c2b18d2 Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: Ic68faa63ccf1ae0af395643e97c0fc472848ac3e
2014-11-10 06:02:03 +00:00
Jenkins
04235f93b7 Merge "Reorganize DbTestCase to use provisioning completely" 2014-11-08 14:37:25 +00:00
Jenkins
664a4e1e2f Merge "Activate pep8 check that _ is imported" 2014-11-08 14:13:52 +00:00
Doug Hellmann
20991774b8 Add pbr to installation requirements
Add pbr to the list of installation requirements so that it is installed
via pip before this library is installed, instead of with easy_install.
This avoids issues like Bug #1384919, and ensures that projects that use
this library as a dependency are properly installed.

Related-Bug: #1384919

Change-Id: I6c155370dbd01fe4748d5137bdf288e8d3e1a67e
2014-11-08 12:58:23 +00:00
OpenStack Proposal Bot
9aa02f499d Updated from global requirements
Change-Id: I83bd22b11ca85eee8fb3dedceef4d9ac00343c38
2014-11-05 08:34:54 +00:00
James Carey
766ff5ef9f Activate pep8 check that _ is imported
Remove the specification in tox.ini that _ is a builtin so that
it will no longer assume that _ does not need to be imported.

This helps ensure that the _ from i18n is used.

Activating this check did not flag any violations.

Change-Id: Idc4d6259d4eb2db66e7122535bc42f8647049a06
2014-10-31 20:39:59 +00:00
Mike Bayer
f99e1b5bf2 Assert exceptions based on API, not string messages
The tests/sqlalchemy/test_exc_filters.py tests were asserting
the exact string format of SQLAlchemy exceptions.
SQLAlchemy is improving its exception format in version 1.0,
thereby breaking these tests.  Instead of supporting multiple
string formats, change the assertions to compare specific
fields per the oslo.db.exceptions and sqlalchemy.exc APIs.

Change-Id: Ie98ac3ca82a1052c463ca9ca5d41e228935497b3
2014-10-29 19:11:50 -04:00
Joshua Harlow
135701b202 Fix python3.x scoping issues with removed 'de' variable
In python3.x the 'de' variable will be removed from the
scope after the except block exits (if it ever is entered)
so we need to use a different variable name to ensure that
it will not be deleted so we can use it later.

This avoids errors of the format:

UnboundLocalError: local variable 'de' referenced before
                   assignment

Change-Id: I9ea2bd6b3a8c392f8d27b0130dd6b7683fc44e7c
2014-10-27 21:21:10 -07:00
OpenStack Proposal Bot
490f6442be Updated from global requirements
Change-Id: Iaed3965166e01fcaf0e9da75ffea3426cda0750e
2014-10-27 12:20:59 +00:00
OpenStack Proposal Bot
8bb12c02d4 Updated from global requirements
Change-Id: I215a53a00948b6477ba9d493df8b0ea6ef6e6d83
2014-10-22 19:16:36 +00:00
Mike Bayer
4e19870fd4 Reorganize DbTestCase to use provisioning completely
This change is the first part in a series of changes
that will allow for full flexibility in database usage
during tests.  The first step is to add more facility
to the oslo.db.sqlalchemy.provision system, implementing
a dispatch system that allows flexibility in creation
and dropping of databases, as well as moving the awareness
of the "openstack_citest" convention into provisioning.
The OpportunisticFixture and OpportunisticTestCase now
fold into DbFixture and DbTestCase, which defers in a simple
way to provision.ProvisionedDatabase for all connectivity.
ProvisionedDatabase in turn decides based on the given
environment as to how an engine should be provisioned
for a given test.

Control of database connectivity remains via the
OS_TEST_DBAPI_ADMIN_CONNECTION environment variable.  When not
set, connectivity defaults to sqlite://, plus those backends
found to be available using "opportunistic" naming conventions.
When the variable is present, it provides a semicolon-delimited
list of URLs, and only those URLs will be used for initial
connectivity.

Future changes will allow provisioning to hold onto a single
database engine per test run, as well as allow a single
test class to run in multiple backend scenarios (e.g. one test
against many different backends).

Change-Id: Ifc02505c4c8ebd4a1ca56e14f76c0989576875c3
Partial-Bug: #1339206
2014-10-22 13:02:20 -04:00
Jenkins
8629ddf424 Merge "ModelsMigrationsSync: Add check for foreign keys" 2014-10-21 23:18:05 +00:00
Ihar Hrachyshka
2a6dbcd09f Set utf8 encoding for mysql and postgresql
Currently, it's enforced in devstack through setting encoding in
connection URI. It would be beneficial if oslo.db handles that on its
own, so that we can

- remove those connection arguments from devstack;
- more importantly, not rely on users to set it in all affected
  projects.

Oursql and MySQLdb also require use_unicode=0 to avoid performance drop.

Change-Id: I676c9c5e418905160e4ab647b8a0f2fb601e8962
Closes-Bug: 1340779
2014-10-17 12:05:19 +02:00
Ann Kamyshnikova
1b410567bc ModelsMigrationsSync: Add check for foreign keys
Add method that checks that foreign keys in models are synchonized
with foreign keys that are created in database.

Change-Id: I4a776da0f53a79218ed4b111cac33b37d264fa1b
2014-10-17 14:03:28 +04:00
Jenkins
848a00bce8 Merge "ModelsMigrationsSync:add correct server_default check for Enum" 2014-10-17 09:25:04 +00:00
Jenkins
f95cd72dd4 Merge "Remove extraneous vim editor configuration comments" 2014-10-14 14:00:16 +00:00
Jenkins
e66fa21039 Merge "Updated from global requirements" 2014-10-13 13:29:12 +00:00
Jenkins
b427eb4505 Merge "Improve error reporting for backend import failures" 2014-10-13 13:29:05 +00:00
OpenStack Proposal Bot
8fb696ebbd Updated from global requirements
Change-Id: Ia7f6ff57a5792d39608d6fcf4490c99a19ea8afb
2014-10-11 15:56:02 +00:00
Christian Berendt
ba4a881530 Remove extraneous vim editor configuration comments
Change-Id: Id8fb52db12695bff4488ea9e74b492a9dabbbccf
Partial-Bug: #1229324
2014-10-08 22:46:07 +02:00
Mike Bayer
33011a51b2 Remove utils.drop_unique_constraint()
This function is currently otherwise unused and does not
work correctly on SQLite, not only dropping the
target unique constraint but instead dropping all unique
constraints from the target table prior to SQLAlchemy
version 1.0 (unreleased).   The tests miss that it isn't
working as well unless SQLAlchemy 1.0 is used, in which
case they fail.

Change-Id: Icaff621fba4df289247e96b87c439c62e91543d6
Closes-bug: #1377646
2014-10-08 14:47:56 -04:00
Doug Hellmann
64f6062cb4 Improve error reporting for backend import failures
Use import_module instead of try_import so that any import exception
includes a full stack trace and the original error message.

Change-Id: I64449688f22a224b590807317472f26e64f377a3
2014-10-06 11:23:03 -04:00
Roman Podoliaka
01a54cc5cc Ensure create_engine() retries the initial connection test
Before return create_engine() tests the connectivity to a DB. At this
moment a DB might be unavailable for some reason, in which case
create_engine() retries a few times. When used with MySQL, we didn't
actually retry, but instead failed on the first connectivity error
when trying to get the value of sql_mode session variable.

Closes-Bug: #1376211

Co-Authored-By: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: I14e25cfe1ed51b0d51a94e491b7267f26e42d34e
2014-10-03 18:59:56 +03:00
OpenStack Proposal Bot
26ec2fcaa2 Imported Translations from Transifex
Change-Id: I93f686641085244cb30020c6c946fab9e51a2206
2014-10-03 06:10:10 +00:00
Jenkins
4b8b89fa03 Merge "Explicitly depend on six in requirements file" 2014-10-02 17:17:48 +00:00
Jenkins
f8f53cf217 Merge "Use fixture from oslo.config instead of oslo-incubator" 2014-10-02 17:07:01 +00:00
Jenkins
cea0887d29 Merge "Add history/changelog to docs" 2014-10-02 17:06:59 +00:00
Jenkins
abecb9ea1a Merge "Unwrap DialectFunctionDispatcher from itself." 2014-10-02 17:01:16 +00:00
Jenkins
a7b3522477 Merge "Create a nested helper function that will work on py3.x" 2014-10-02 16:11:45 +00:00
Jenkins
633eadae08 Merge "Start adding a environment for py34/py33" 2014-10-02 16:11:43 +00:00
Jenkins
39c63e7923 Merge "Use six.wraps instead of functools.wraps" 2014-10-02 16:11:09 +00:00
Jenkins
96f610bf72 Merge "Use the six provided iterator mix-in" 2014-10-02 16:11:01 +00:00
Jenkins
8fecda230a Merge "Move begin ping listener to a connect listener" 2014-09-30 03:33:01 +00:00
Davanum Srinivas
9129545a58 Use fixture from oslo.config instead of oslo-incubator
Change-Id: Ic80562c662960a946b3083f0509700e4809d628d
2014-09-29 07:32:21 -04:00
Mike Bayer
228531013b Move begin ping listener to a connect listener
The "reconnect" listener that occurs on transaction begin
is not sufficient for libraries that use engines, connections
and sessions in "autocommit" mode, where no transaction is begun.
So we move our "ping" from the begin phase to the "connect"
phase instead.   We use a newer SQLAlchemy event "engine_connect"
so that we can make use of the Connection and all the disconnect/
invalidation facilities that are included, rather than using
the low-level pool listener which is somewhat legacy.
Since the event was added in SQLAlchemy 0.9.0, a compatibility
layer is added which emulates the mechanics of the
handle_error compatibility layer.

Change-Id: I719fe45e657e747e5f8d64f20059e2b61c79b63d
Closes-bug: #1374497
2014-09-27 23:30:24 -04:00
Joshua Harlow
7f9f4f1b49 Create a nested helper function that will work on py3.x
The contextlib.nested function doesn't exist on py3.x so
in order to retain similar functionality create an equivalent
that will perform good enough for oslo.db usage.

Change-Id: I326816c9301f1c625e1e2723d4d639fd81cc596c
2014-09-26 21:04:37 +00:00
OpenStack Proposal Bot
b42d8f1c13 Imported Translations from Transifex
Change-Id: I293172684fcff520ae445836bb132bf1a919d127
2014-09-26 06:04:11 +00:00
Joshua Harlow
4fa3350945 Start adding a environment for py34/py33
Get a tox environment for py34/py33 that doesn't stop the existing
py26 and py27 to continue working with the adjusted test-requirements since
MySQL-python and sqlalchemy-migrate do not yet exist in 3.x.

Note that this does not fully make py34 work yet but it starts to work
in this direction of getting it to work (which will apparently require
quite a big more tweaking).

Change-Id: I024957a76f2de0bd406d14da5af6e4caf3d493e7
2014-09-25 13:05:09 -07:00
Joshua Harlow
b09ee9a22f Explicitly depend on six in requirements file
Six is used directly in oslo.db (and is not just a transitive
dependency) so it should be listed in the requirements file
just like other non-transitive dependencies.

Change-Id: I81d2d7d0db9142bace7ecf8882efbbcd605fb527
2014-09-24 19:55:23 -07:00
Mike Bayer
7a3e091c1c Unwrap DialectFunctionDispatcher from itself.
The DialectFunctionDispatcher.dispatch_for() decorator method
necessarily returns the dispatcher itself and not the
decorated function, so that the object can continue to be
re-used even if the function name is the same as that of the
dispatcher.   In order to support a single function being
wrapped by the dispatcher multiple times with different
criteria, dispatch_for() will now check for the last function
wrapped and use that.

Change-Id: I331670d9b76ae30e7a666648e7e2d4c72641c9ff
Closes-Bug: #1373568
2014-09-24 15:04:53 -04:00
Jenkins
3fdafcd17f Merge "Update help string to use database" 2014-09-24 14:35:50 +00:00