489 Commits

Author SHA1 Message Date
rossella
1b7c295d52 Retry query if db deadlock error is received
Add a decorator retry_on_deadlock(). All db.api functions marked
with this decorator will be retried if a DBDeadlock exception is
received.

Change-Id: I3488bfec67ec1c563292f1b61a7a9697f118809c
Closes-bug: 1348588
2015-01-14 09:56:02 +00:00
Jenkins
67c6f02bb9 Merge "Ensure DBConnectionError is raised on failed revalidate" 2015-01-13 15:13:04 +00:00
Mike Bayer
b1fc55c7ce Restore the check_foreign_keys() method.
This method was prematurely removed from oslo.db without
a deprecation phase, when Alembic added support for
foreign key autodetection.   oslo.db continues to use
alembic for this purpose, however the
ModelsMigrationsSync.check_foreign_keys() method is restored
directly for those projects which were calling into it
explicitly.

Change-Id: Id892567bd60d6b4b88765bbfe3cd5c5e75910b25
2015-01-12 17:26:23 -05:00
Mike Bayer
046e576467 Ensure DBConnectionError is raised on failed revalidate
The second call to connection.scalar() inside of _connect_ping_listener()
is not subject to the exc_filters system under SQLAlchemy 0.9 and earlier,
in the case that the attempt to revalidate within fails.  This causes
the exception to be propagated outwards as the original
OperationalError (assuming that's what the DBAPI raised, as is typical),
rather than it being wrapped again in DBConnectionError.

SQLAlchemy 1.0 has altered this behavior such that the handle_error()
listener is invoked correctly for revalidiation attempts (it was being
invoked before, but without the correct state), as well as
for initial connect attempts, as part of
https://bitbucket.org/zzzeek/sqlalchemy/issue/3266/.  For 0.9 and earlier,
we here backport this system into oslo/db/sqlalchemy/compat.py,
including that we redefine the connection pool's handler to re-throw
the original dbapi.Error instance directly, then move the handling
of this error in terms of SQLAlchemy wrapping and events into
the Engine and Connection.   The approach here works back to
SQLAlchemy 0.8, and is conditional based on whether or not
SQLAlchemy 1.0 is detected.

Change-Id: I455e957b043318a8500909c66cadde53228b52d6
2015-01-12 16:02:01 -05:00
Jenkins
98cdceb1ec Merge "Fix slowest test output after test run" 2015-01-12 17:21:02 +00:00
Jenkins
b84d921ac8 Merge "Make sure sort_key_attr is QueryableAttribute when query" 2015-01-12 17:17:24 +00:00
Matthew Treinish
9a510e85ce Fix slowest test output after test run
This commit fixes the output from pretty_tox.sh so that the testr
slowest output is gobbled up by the pipe into subunit-trace.

Change-Id: I4e23cb3c3773e58a297416c1f51996fef059330a
2015-01-11 16:55:30 -05:00
Jenkins
a6aefaa34b Merge "Updated from global requirements" 2015-01-11 21:09:43 +00:00
Jenkins
b5956cbaee Merge "Add pretty_tox wrapper script" 2015-01-11 21:06:14 +00:00
OpenStack Proposal Bot
2c3768c71f Updated from global requirements
Change-Id: I76dce91a86cd2e551064068f4312df78f66f0c33
2015-01-09 18:35:05 +00:00
Eli Qiao
bce8ed3042 Make sure sort_key_attr is QueryableAttribute when query
When doing query.order_by, sort_key_attr is get from model class,
we need to make sure sort_key_attr is really a QueryableAttribute
type instance before we do the query or it will cause errors.
This will prevent if there IS a function which name is same as sort_key.

Closes-Bug: 1405069
Change-Id: I8a3eb08ab3469ec08e05bfce754b664943d65c83
2015-01-09 06:12:06 +08:00
Mike Bayer
cfbe5c5634 Ensure mysql_sql_mode is set for MySQLOpportunisticTests
The default setting for oslo.db is TRADITIONAL for mysql_sql_mode;
this is set up in oslo_db/options.py as well as
in EngineFacade; however, the provisioning system currently does not
use these options or EngineFacade, and the default for
mysql_sql_mode in the base session.create_engine() is None.
For a MySQL/MariaDB database where this is not established in my.cnf,
tests will run without TRADITIONAL SQL mode and some tests will fail,
including some Nova migration tests that rely upon this.

This patch resolves that as well as the warning we see
in tests regarding the empty SQL mode.

Change-Id: I3acdc0fb30327f56a76ee299cc6bade7c5a7e312
2015-01-08 11:59:47 +02:00
Mike Bayer
efbb3887e0 Add pretty_tox wrapper script
This ports Nova/Neutron's'(and others) "pretty tox" script into tools/
and establishes it within tox.ini.   This causes a vanilla test run
to output the full package names of all tests, the index of the
subprocess it's being run within, as well as the total run time
and status of each test.  It also captures stdout/stderr and reports on that
as well; revealing that oslo.db's tests have a lot of deprecation
warnings in fact.

The display of this runner shows a lot more information than the
default testr/subunit thing which is somewhat useless, and for
those who are actually watching the test output, this is what
we'd most like to see.

Pretty Tox.  Pretty please, can we have?

Change-Id: I50d1c6e998425964dd9a5497c2bc1e9145be3120
2015-01-06 17:57:50 -05:00
Jenkins
98b434db7d Merge "Ensure PostgreSQL connection errors are wrapped" 2015-01-06 02:07:43 +00:00
Jenkins
d0c13d2a13 Merge "Fix PatchStacktraceTest test" 2015-01-05 13:05:26 +00:00
Victor Sergeyev
91b0199a4f Fix PatchStacktraceTest test
After we moved out from the namespaces, tests were moved to
oslo_db/tests. The code in oslo_db.sqlalchemy.session filters out
lines from modules under oslo_db, so the test filename does not
appear in the context for the error message.
Use patched traceback module in tests to fix it.

Closes-Bug: #1405376

Change-Id: Id6022c065dfe13351fca5e54baa343d6c96b0270
2015-01-05 09:31:48 +00:00
Roman Podoliaka
30433383c7 Ensure PostgreSQL connection errors are wrapped
Currently PostgreSQL connection errors are not wrapped with our
custom DBConnectionError and engine.connect() will raise
OperationalError instead.

Change-Id: Iefdb9a99ca0cbe982bf12cb7e1ac47996fc5a025
2015-01-05 11:28:54 +02:00
Ann Kamyshnikova
75b402be3b Remove check_foreign_keys from ModelsMigrationsSync
Alembic 0.7.1 contains checks of foreign keys so method
check_foreign_keys is not needed anymore.

Change-Id: I77a2bba2de08125b5a601de03c1d2d5c73fa33ee
2014-12-31 11:56:59 +03:00
Jenkins
44c9c574bd Merge "Move files out of the namespace package" 2014-12-30 14:28:46 +00:00
Doug Hellmann
7063585c60 Move files out of the namespace package
Move the public API out of oslo.db to oslo_db. Retain the ability to
import from the old namespace package for backwards compatibility for
this release cycle.

Blueprint: drop-namespace-packages

Change-Id: Ie96b482b9fbcb1d85203ad35bb65c1f43e912a44
2014-12-24 13:28:48 +02:00
OpenStack Proposal Bot
4a57952d9f Updated from global requirements
Change-Id: I24ac192cbd8b4edf8f5a0ee2f9b3613d46899deb
2014-12-18 01:27:30 +00:00
Doug Hellmann
c6ddb04eb1 Fix the link to the bug reporting site
Update the README to fix the link to the bug reporting site and to
provide a slightly more detailed description of the library.

Change-Id: I8896297bd8d7a64c72e17c0d9a985cb35991e41d
2014-12-15 11:48:23 -05:00
Jenkins
571433bfc4 Merge "Repair string-based disconnect filters for MySQL, DB2" 2014-12-12 12:48:22 +00:00
Jenkins
8d07501262 Merge "Upgrade exc_filters for 'engine' argument and connect behavior" 2014-12-12 12:46:32 +00:00
Mike Bayer
0265aa4e01 Repair string-based disconnect filters for MySQL, DB2
The two regexp-based filters in exc_filters.py->_is_db_connection_error()
were both incorrectly formed.  The tests for these filters would pass
because the fixture also set the is_disconnect flag to True, which
normally would be set by SQLAlchemy, and therefore the
_raise_operational_errors_directly_filter() or possibly
the _raise_for_remaining_DBAPIError() filters would catch this,
view the is_disconnect flag as True, and promote to a DBConnectionError.
However, the _is_db_connection_error() filters are intended to
work independently of whether this flag is set; so two new tests
are added which unset the flag, and ensure that the exception messages
as given are caught here.

Change-Id: I37ddd669b89669730ae1ff07c7bc7a6ba5705f67
2014-12-12 12:22:37 +02:00
Jenkins
52b03037a7 Merge "Updated from global requirements" 2014-12-11 10:27:40 +00:00
Joshua Harlow
b1af0f5919 Fix python3.x scoping issues with removed 'uee' variable
In python3.x the 'uee' variable will be removed from the
scope after the except block exits (if it ever is entered)
so we need to use a different variable name to ensure that
it will not be deleted so we can use it later.

Change-Id: Ia86a2bd1a46e57eaf47704eadc991fd0c9b08661
2014-12-11 11:06:36 +02:00
OpenStack Proposal Bot
c6b352e50d Updated from global requirements
Change-Id: I976f8b8a7b3d31c7046ed53f6564c58af408b1b2
2014-12-11 07:19:26 +00:00
Jenkins
ed200283c1 Merge "Fix TestConnectionUtils to py3x compatibility" 2014-12-10 16:54:14 +00:00
Jenkins
b60b3fe7bf Merge "Fix test_migrate_cli for py3" 2014-12-09 16:29:27 +00:00
Oleksii Chuprykov
9658b28959 Fix test_migrate_cli for py3
Use __lt__ insted of __cmp__ for py3 compatability.

Change-Id: I85d8eeb2cbb754b72b04c817f49d3a62f1473994
2014-12-09 13:52:54 +02:00
Victor Sergeyev
4c939b38eb Fix TestConnectionUtils to py3x compatibility
Tests in TestConnectionUtils class depends on database and DB connector,
so the simplest way to run them with python 3.X is to use same database
and connector for the both python versions. PostgreSQL and psycopg2 are
looks good for this.

Replaced ``mysql`` to ``postgresql`` in TestConnectionUtils.

Change-Id: Icb1d6324b58feed515e4eb885715cbf2195768cf
2014-12-09 12:22:47 +02:00
Jenkins
34caedc342 Merge "Fix nested() for py3" 2014-12-09 09:52:00 +00:00
OpenStack Proposal Bot
9c3477dae1 Updated from global requirements
Change-Id: I3443352b9ec847534cf4016dc540da769501593b
2014-12-08 16:23:18 +00:00
Mike Bayer
32e5c6096a Upgrade exc_filters for 'engine' argument and connect behavior
This patch applies upgrades to the sqlalchemy/exc_filters.py and
sqlalchemy/compat/handle_error.py compatibility layers to accommodate
new changes in SQLAlchemy 1.0.  SQLA 1.0 will now route errors that
occur upon connect through the handle_error() event, just like any other,
so that when 1.0 is present we no longer need to use
exc_filters.handle_connect_error; the method becomes a passthrough
as far as running the event handler.  Additionally, SQLAlchemy 1.0
has added the "engine" parameter to ExceptionContext, specifically
to suit the case when the initial connect has failed and there is
no Connection object; the compatibility layer here now emulates
this behavior for SQLAlchemy versions prior to 1.0.

Change-Id: I61714f3c32625a621eaba501d20346519b8b12c7
2014-12-06 15:58:22 -05:00
Jeremy Stanley
161bbb2831 Workflow documentation is now in infra-manual
Replace URLs for workflow documentation to appropriate parts of the
OpenStack Project Infrastructure Manual.

Change-Id: I318dea15e0d2f7ef22427fe22d8d57173e60e784
2014-12-05 03:30:38 +00:00
Jenkins
4ad43a4061 Merge "Imported Translations from Transifex" 2014-11-28 15:07:54 +00:00
Jenkins
29e229a053 Merge "Make test_models pass on py3" 2014-11-28 12:16:42 +00:00
Jenkins
3d5a9b134f Merge "Repair include_object to accommodate new objects" 2014-11-28 09:22:03 +00:00
OpenStack Proposal Bot
f740e3bd98 Imported Translations from Transifex
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I7e50cf461b9ff7344a00a63f35709667c9ec857b
2014-11-28 06:01:27 +00:00
Oleksii Chuprykov
86c136a084 Fix nested() for py3
nested() should return enter_context instead of
nothing.

Change-Id: I910eace453d07e2f9a12dd0ce0eca22f4d155472
2014-11-24 18:43:24 +02:00
Jenkins
b80fceaf40 Merge "Add info on how to run unit tests" 2014-11-21 18:43:15 +00:00
Oleksii Chuprykov
ca1ad56579 Make test_models pass on py3
Add a check that models.ModelBase is iterable.
Remove checking on next() method since models.modelBase
isn't actually iterator.
Remove unnecessary checks from TimestampMixinTest

Change-Id: Ibbde0e49bec25e1acddaf64f03083cf46481a581
2014-11-21 17:05:34 +02:00
Jenkins
8a37f67050 Merge "Add table name to foreign keys diff" 2014-11-21 11:19:56 +00:00
Jenkins
ff1b6ef15a Merge "Add exception filter for _sqlite_dupe_key_error" 2014-11-21 11:04:23 +00:00
Mike Bayer
549ba15e30 Repair include_object to accommodate new objects
The tests in tests/sqlalchemy/test_migrations.py which expect
to see changes in unique constraints fail due to the
assumption that the "include_object" feature of Alembic only
deals with the 'table' and 'column' constructs.   As Alembic
0.7.0 has added include_object support for 'index' and
'unique_constraint', this method should not be limiting
itself to not include objects that it's testing for.
Tests won't pass on Alembic 0.7.0 without this patch.

Change-Id: Ia71446bb8c0b248c6310534deb290524e3946987
2014-11-21 12:32:04 +02:00
Jenkins
8c478fb986 Merge "Updated from global requirements" 2014-11-20 22:46:34 +00:00
Jenkins
3363683679 Merge "Handle Galera deadlock on SELECT FOR UPDATE" 2014-11-20 22:44:37 +00:00
Oleksii Chuprykov
10e8d15a26 Add table name to foreign keys diff
Add table to diff that is generated by check_foreign_keys
function.

Change-Id: Ib771ea49e873dd87cccacf6ec7463d6e005372a3
2014-11-20 19:26:22 +02:00
OpenStack Proposal Bot
ddd11df196 Updated from global requirements
Change-Id: I8a909d181302ea1c898dcdf1ce13a9e4681217f8
2014-11-20 14:11:24 +00:00