489 Commits

Author SHA1 Message Date
OpenStack Proposal Bot
0928d7319a Updated from global requirements
Change-Id: I7e4d84dc6c45c1220766261363f2fabc5085fe31
2014-09-23 02:00:17 +00:00
Joshua Harlow
696f3c1289 Use six.wraps instead of functools.wraps
The six module provides a helper wraps function that correctly
tracks the original wrapped function in python 2.x and 3.x so
use it where we can so that we maintain better compatability with
3.x

Change-Id: Ifa994bf63153f58f0aa1f6203a963ac7312ac2b1
2014-09-22 11:59:15 -07:00
Andreas Jaeger
8fac4c7c0e Update help string to use database
Use "database" instead of "db" in help string as noticed during
review of https://review.openstack.org/#/c/120200/

Change-Id: I6e17c759ae101af23b7d57c13dfba26aa6feddb6
2014-09-22 16:07:29 +02:00
Jenkins
91e5c5eec9 Merge "Add run_cross_tests.sh script" 2014-09-22 06:01:19 +00:00
Jenkins
aaf82a4637 Merge "Add description for test_models_sync function" 2014-09-22 05:56:51 +00:00
Jenkins
50e39e1907 Merge "Use __qualname__ if we can" 2014-09-21 08:59:13 +00:00
Jenkins
2849f1bdca Merge "Fixup Fixtures Use in db test classes." 1.0.1 2014-09-20 09:14:52 +00:00
Joshua Harlow
fc8eb627a7 Use __qualname__ if we can
Due to pep-3155 functions and methods will now have more
uniform qualified names that we can and should use when
we can (mainly applicable to python 3.x).

Once https://review.openstack.org/#/c/122495/ goes in we
can just use that module instead to get similar functionality.

Change-Id: Iea9e74aca38aa79c9294fd6e786551e91143900c
2014-09-19 13:03:26 -07:00
Sean Dague
9fcf1dbf75 Fixup Fixtures Use in db test classes.
With Fixture objects and test cases it is important to upcall first
rather than later to allow base classes to configure their internal
test tracking state. Failing to do so allows bugs like this one to
occur - when failure occurs early, the state required to gather
environmental details about the failure is not prepared, and the
test machinery will raise secondary exceptions as it tries to do that.
This may even obscure the original error.

I've taken a fairly minimal approach here - I think it would be
possible and likely even desirable to get rid of the test classes
altogether - this module appears to have very little, if any, actual
test code - its all glue for managing the test database connections.
However I know zzzeek is working on a fairly large reorg in this
area and I don't want to conceptually clash with that - we can do
further cleanups later.

Co-Authored-By: Robert Collins <rbtcollins@hp.com>

Change-Id: Ic9f24aa2352d097a10be8420b701fa9f588b21b0
Closes-Bug: #1330763
2014-09-19 09:10:27 -07:00
Oleksii Chuprykov
6a664b9e8a Add description for test_models_sync function
Add description of the diff list that is generated in
test_modes_sync function for readability reason.

Change-Id: I3ecaf2da5a22b1835e0c6604b59f9cd1874747de
2014-09-19 10:30:21 +00:00
Joshua Harlow
8bc1fb79ba Use the six provided iterator mix-in
Instead of having a customized next() and __next__() methods
just use the mix-in that six provides and used it to avoid
having to declare our own next() compatability function.

Change-Id: I1a5fc9bd43783577af879476ca46afcc668185f5
2014-09-16 12:50:30 -07:00
Ann Kamyshnikova
436dfdc737 ModelsMigrationsSync:add correct server_default check for Enum
If Column which type is Enum has server_default it is compared
incorrectly for PostgreSQL.

For example, Column('test', sa.Enum('one', 'two', name='enum'),
                     server_default='one').
In model it has DefaultClause('one', for_update=False)) alembic
finds it different from server variant "'one'::enum".

Change-Id: Id30858f9c4c7a2bfb7f9313d47930845fe354a70
2014-09-15 15:48:09 +04:00
Jenkins
cc7ae9fecc Merge "Add a check for SQLite transactional state" 2014-09-15 10:18:01 +00:00
Jenkins
edbd03e668 Merge "Updated from global requirements" 2014-09-13 18:18:05 +00:00
Jenkins
517b6d4a19 Merge "Imported Translations from Transifex" 2014-09-13 13:02:18 +00:00
Jenkins
2205cd7177 Merge "Clean up documentation" 2014-09-13 12:38:44 +00:00
OpenStack Proposal Bot
c785beefe7 Updated from global requirements
Change-Id: Ia959fbf504268aa13e077cb2637fc14367deb247
2014-09-13 07:27:35 +00:00
OpenStack Proposal Bot
ac05c2a8cd Imported Translations from Transifex
Change-Id: Idb877c59f277ccda8068d83445e43550da235038
2014-09-12 06:00:37 +00:00
Jenkins
9977e8a79f Merge "Fix DBReferenceError on MySQL and SQLite" 2014-09-11 23:51:48 +00:00
Jenkins
d71a6ccbb2 Merge "Use single quotes for db schema sanity check" 2014-09-11 05:26:22 +00:00
Davanum Srinivas
2075074899 Add history/changelog to docs
Change-Id: I6f96f02f73ef0c217c6327172f391a06900c2cf9
2014-09-10 08:30:11 -04:00
Mike Bayer
57f499eae5 Add a check for SQLite transactional state
When we intercept SQLite transactions for BEGIN, check
a marker that we place there to indicate BEGIN has
already been called, which we then remove on commit or
rollback.  This is to resolve the issue of the fact
that we currently use the StaticPool implementation
with SQLite, which shares a single SQLite connection
for all requests; any API method which makes use
of multiple sessions at once (not a good idea, but
this is prevalent throughout nova, neutron) will
therefore share the same SQLite connection with
multiple SQLAlchemy Connection wrappers that are not
aware of the existing state.  While this practice
should be corrected, for now the marker, which is
local to the SQLite connection as stored by the
pool, will track when BEGIN is safe to call, or
has already been called.

Change-Id: I70d44104412bd99d8c7713eb6cd9ff0f80c5da34
Closes-Bug: #1367354
2014-09-09 13:13:00 -04:00
Victor Sergeyev
c9e5fdf3bc Add run_cross_tests.sh script
Script `run_cross_tests.sh` can be used for running unit tests in
other projects using the current package's code.

Change-Id: Iad636216cd73d2eeb70448fda789c0ad9d579b3a
Related-Bug: #1367354
2014-09-09 19:13:03 +03:00
Jenkins
8f7681946a Merge "Renaming in WalkVersionsMixin" 2014-09-09 05:02:20 +00:00
Jenkins
edf4c41e55 Merge "warn against sorting requirements" 2014-09-08 20:16:41 +00:00
Doug Hellmann
bdcfb4a4df Let oslotest manage the six.move setting for mox
This instruction to set up a move in six for the mox module conflicts
with the one in oslotest. We are trying to use mox3 everywhere, so
remove this instruction and let oslotest handle it.

Imported from the incubator change with the same ID.

Change-Id: I59d5799283233f8411044ddb15c8abfc8850014c
2014-09-07 14:59:41 -04:00
Julien Danjou
4ed63fac85 Fix DBReferenceError on MySQL and SQLite
It turns out the test was wrong as the error message used was not a good
one. Real world tests showed that, so I've updated the test and the code
to work.

Refactored tests to use real backends, fix error message for SQLite.

Co-Authored-By: Victor Sergeyev <vsergeyev@mirantis.com>

Change-Id: Ifa64478a9ac9ed0a3c317a8974615b770c85f313
2014-09-05 19:01:59 +03:00
Oleksii Chuprykov
49ae6c0446 Renaming in WalkVersionsMixin
Rename _walk_versions(), _migrate_up(), _migrate_down()
and mark them deprecated as they will be removed in few
releases. These functions will be the part of public API
as this class will be used for migrations in other
OpenStack projects, so there is no need to make them
'private'.

Change-Id: Ic9358445e60a0dd43a5900e8bda7b12f2bebf679
2014-09-05 15:32:50 +03:00
Doug Hellmann
cae12025a9 Clean up documentation
Switch to manually generated rst files for the API documentation so we
do not expose private parts of the library.

Fix formatting of usage.rst

Convert bullet list to section headers to fix rendering issues and make
the docs more readable.

Fix formatting of docstrings in classes exposed in the docs to eliminate
warnings/errors from Sphinx.

Add history.rst

Change-Id: I6f500775f801558f7c0c29f180b60f83a7150e02
2014-09-04 16:27:00 -04:00
Huai Jiang
fb577ecd6f Use single quotes for db schema sanity check
double quotes would cause failure when mysql running on ANSI sql_mode.

Change-Id: I891b61035ecb90b91a9ccdbec14dd567c575299b
Closes-Bug:1365019
2014-09-04 09:16:47 +08:00
Doug Hellmann
ab475162c8 warn against sorting requirements
Change-Id: I64ae9191863564e278a35d42ec9cd743a233028e
Addresses-Bug: #1365061
2014-09-03 14:20:03 -04:00
Ann Kamyshnikova
be6f9a0ff7 ModelsMigrationsSync:Override compare_server_default
This test showed an error with alembic 0.6.6:
For MySQL Integer server_default value is compared incorrectly.
For example, when model has DefaultClause('0', for_update=False)
alembic finds it different from server variant "'0'".
For PostgreSQL Sting server_default value is compared incorrectly.
For example, when model has DefaultClause('', for_update=False)
alembic finds it different from server variant "''::character varying".

Such error appear in implementation of ModelsMigrationsSync to Neutron
https://review.openstack.org/76520.

Change-Id: I9730da1c0790a88f81570c51c36cc0cbe0651a5a
2014-09-03 13:22:28 +04:00
Jenkins
9234bbbc84 Merge "Updated from global requirements" 2014-09-03 06:20:12 +00:00
Jenkins
720af896c8 Merge "Add doc8 to tox environment docs" 2014-09-02 17:07:02 +00:00
OpenStack Proposal Bot
e187a4c69d Updated from global requirements
Change-Id: Ic7d77a737d3e16f4f781baebe8ce57d04db04832
2014-09-02 09:59:38 +00:00
OpenStack Proposal Bot
cbf995ef87 Imported Translations from Transifex
Change-Id: Ic2108aab91b04e337f6a68e0275ec2806dc1a3e8
2014-09-02 06:05:53 +00:00
Jenkins
0e854f2c39 Merge "Use oslo.i18n" 2014-09-02 00:34:02 +00:00
Christian Berendt
f2f0960b15 Add doc8 to tox environment docs
Check reStructuredText documents for common style issues.

Change-Id: I10e61742e86d2d4781c5a00532c8e17d38a7d4b4
2014-09-01 11:40:08 +02:00
Andrey Kurilin
92d9a1c40f Use oslo.i18n
Module `gettextutils` from common code was graduated to oslo.i18n, so it
would be great if we reuse this library.

Change-Id: Ie5836b82723fc80c1d54e5e65eac8565cf905c5a
2014-09-01 11:48:21 +03:00
Mike Bayer
78fd290a89 Repair pysqlite transaction support
This patch applies the events discussed in
SQLAlchemy documentation at
http://docs.sqlalchemy.org/en/rel_0_9/dialects/sqlite.html#serializable-isolation-savepoints-transactional-ddl
which correct for pysqlite's modifications to SQLite's transactional
cycle.   We turn off pysqlite's automatic emit of BEGIN
as well as COMMIT before any DDL, and instead emit our own
BEGIN at the point where SQLAlchemy knows a transaction is to start.
A proof of concept is added to illustrate SAVEPOINT working
completely.

SAVEPOINT support will be needed in order to support
transactional testing with rollbacks inside of tests.

Change-Id: Ie39bd2287173327903523b66160653608d9e8652
2014-08-28 14:25:22 -04:00
Roman Vasilets
41dada16cd Extract logging setup into a separate function
Change-Id: I9f1761a5332ad3c4a8f0b76c6f46fc687fd61aa5
2014-08-26 13:32:17 +03:00
Jenkins
76c0676687 Merge "Updated from global requirements" 2014-08-24 16:20:57 +00:00
Jenkins
4c7fc3e685 Merge "Remove reliance on create_engine() from TestsExceptionFilter" 2014-08-23 08:25:45 +00:00
Jenkins
93f41a43ef Merge "Consolidate sqlite and mysql event listeners" 2014-08-23 08:25:42 +00:00
OpenStack Proposal Bot
7c57798457 Updated from global requirements
Change-Id: I834ba5e6a6d583370099b5c87d28680a67a802c1
2014-08-22 23:35:01 +00:00
Jenkins
6585271783 Merge "Added check to see whether oslotest is installed" 2014-08-22 21:42:05 +00:00
Mike Bayer
977869d553 Remove reliance on create_engine() from TestsExceptionFilter
As we are continuing to add functionality to create_engine()
which in some cases means that backend-specific SQL is
emitted on connect, the practice of repurposing a
SQLite or other engine to act like another one for the
purposes of a test is becoming less feasible.   Here,
TestsExceptionFilter is altered to not rely upon the full
session.create_engine() process for unit tests, instead
calling upon just those listeners being tested.

Change-Id: I0ece42f956c59eab7a6cb86419d8d359250d5e71
2014-08-21 17:51:18 -04:00
Mike Bayer
626d762cbd Consolidate sqlite and mysql event listeners
This patch refactors the SQLite and MySQL event listener
setup in session.py to occur inline within the
new _init_events() hooks for each of these backends.  This
is to improve code readability and reduce unneeded boilerplate.

In particular, _mysql_set_mode_callback() was implemented using
four functions, all private to session.py, unused anywhere
else and also not consistently prefixed with "_mysql".
These functions have been consolidated into a single event
listen plus a quick check of SQL mode.   In particular,
the system now emits the logging check for "sql_mode"
in all cases.

On the SQLite side, three functions, again not all prefixed
with _sqlite, are consolidated into a single event listening
function wihtin _init_events().

The test suite is altered in several ways.  The tests
for connect arguments no longer inject a sqlite engine
into session.create_engine(); now that MySQL does a particular
statement every time, this was no longer feasible, and in general
other setup functions might emit SQL also, so the
CreateEngineTest tests now test directly into session._init_connection_args.
The tests for MySQL modes and SQLite connection settings
move into being plain round-trip tests, reducing lots of
mocking logic and ultimately doing a more complete test against
a real backend.

Change-Id: I04ca3fbb1f46a64ed6552496ae995f77aa0b2315
2014-08-21 17:51:18 -04:00
Mike Bayer
5ab43eefa5 Use dialect dispatch for engine initiailization.
This patch makes use of the new DialectFunctionDispatcher
to remove all conditional logic from within session.create_engine().
The flags that result in various engine connect arguments as well
as subsequent event listeners now fall under the realm of dispatched
functions keyed to their target dialects.   Additionally, this
modernizes the SQLite foreign keys listener to use events.

A test suite is added which via mocks asserts that arguments
expected to be present as well as non-present for a variety of url situations
is added.

Change-Id: Iba5e77e1396eda46ac2c6d6cbca159cbfadf4d04
2014-08-21 17:51:18 -04:00
Vlad Okhrimenko
468347515d Add get_non_innodb_tables() to utils
Add a helper function to utils for detection of tables which aren't
using InnoDB storage engine in MySQL (something we want to enforce in
consuming projects).

Change-Id: I22889e23b025fc32fe76e4086cd3cd3d5bd0b771
2014-08-21 16:46:56 +03:00