diff --git a/TODO b/TODO index 8d65f56..d1f0fa4 100644 --- a/TODO +++ b/TODO @@ -19,3 +19,7 @@ make_update_script_for_model: - port to unittest2 - write documentation how to test all databases +- glossary +- managing output - logging +- required_dbs tests +- add story to changeset tutorial diff --git a/docs/api.rst b/docs/api.rst index 045c809..5d960e8 100644 --- a/docs/api.rst +++ b/docs/api.rst @@ -105,6 +105,8 @@ Module :mod:`migrate.versioning` :members: :synopsis: Database version and repository management +.. _versioning-api: + Module :mod:`api ` ------------------------------------------ diff --git a/docs/changelog.rst b/docs/changelog.rst index ad46a22..3e9348b 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -23,11 +23,11 @@ .. _backwards-06: -**Backward incompatible changes**: +.. warning:: **Backward incompatible changes**: -- :func:`api.test` and schema comparison functions now all accept `url` as first parameter and `repository` as second. -- python upgrade/downgrade scripts do not import `migrate_engine` magically, but recieve engine as the only parameter to function (eg. ``def upgrade(migrate_engine):``) -- :meth:`Column.alter ` does not accept `current_name` anymore, it extracts name from the old column. + - :func:`api.test` and schema comparison functions now all accept `url` as first parameter and `repository` as second. + - python upgrade/downgrade scripts do not import `migrate_engine` magically, but recieve engine as the only parameter to function (eg. ``def upgrade(migrate_engine):``) + - :meth:`Column.alter ` does not accept `current_name` anymore, it extracts name from the old column. 0.5.4 ----- diff --git a/docs/changeset.rst b/docs/changeset.rst index 5cf12a3..f466c82 100644 --- a/docs/changeset.rst +++ b/docs/changeset.rst @@ -1,14 +1,14 @@ .. _changeset-system: .. highlight:: python -****************** -Database changeset -****************** +************************** +Database schema migrations +************************** .. currentmodule:: migrate.changeset.schema Importing :mod:`migrate.changeset` adds some new methods to existing -SA objects, as well as creating functions of its own. Most operations +SQLAlchemy objects, as well as creating functions of its own. Most operations can be done either by a method or a function. Methods match SQLAlchemy's existing API and are more intuitive when the object is available; functions allow one to make changes when only the name of @@ -70,7 +70,7 @@ Given a standard SQLAlchemy table:: assert table.c.col3 is col .. warning:: - Since version ``0.6.0`` passing column into alter is deprecated. Pass in explicit parameters instead. + Since version ``0.6.0`` passing column into :meth:`ChangesetColumn.alter` is deprecated. Pass in explicit parameters instead. .. note:: diff --git a/docs/conf.py b/docs/conf.py index 20d48c8..a54c81b 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -47,16 +47,16 @@ master_doc = 'index' # General information about the project. project = u'SQLAlchemy Migrate' -copyright = u'2009, Evan Rosson, Jan Dittberner' +copyright = u'2009, Evan Rosson, Jan Dittberner, Domen Kožar' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. -version = '0.5.4' +version = '0.6.0' # The full version, including alpha/beta/rc tags. -release = '0.5.4' +release = '0.6.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/docs/download.rst b/docs/download.rst index a1ae702..7fb9ec3 100644 --- a/docs/download.rst +++ b/docs/download.rst @@ -2,9 +2,13 @@ Download -------- You can get the latest version of SQLAlchemy Migrate from the -`project's download page`_, the `cheese shop`_, or via easy_install_:: +`project's download page`_, the `cheese shop`_, pip_ or via easy_install_:: - easy_install sqlalchemy-migrate + easy_install sqlalchemy-migrate + +or:: + + pip install sqlalchemy-migrate You should now be able to use the *migrate* command from the command line:: @@ -17,6 +21,7 @@ display more information about each command. If you'd like to be notified when new versions of SQLAlchemy Migrate are released, subscribe to `migrate-announce`_. +.. _pip: http://addmenow.si .. _easy_install: http://peak.telecommunity.com/DevCenter/EasyInstall#installing-easy-install .. _sqlalchemy: http://www.sqlalchemy.org/download.html .. _`project's download page`: http://code.google.com/p/sqlalchemy-migrate/downloads/list @@ -26,17 +31,17 @@ are released, subscribe to `migrate-announce`_. Development ----------- -Migrate's Subversion_ repository is at -http://sqlalchemy-migrate.googlecode.com/svn/ +Migrate's Mercurial_ repository is located at `Google Code`_. To get the latest trunk:: - svn co http://sqlalchemy-migrate.googlecode.com/svn/trunk + hg clone http://sqlalchemy-migrate.googlecode.com/hg/ Patches should be submitted to the `issue tracker`_. -We use `buildbot`_ to help us run tests on all databases that migrate supports. +We use `hudson`_ Continuous Integration tool to help us run tests on all databases that migrate supports. -.. _subversion: http://subversion.tigris.org/ +.. _Mercurial: http://www.mercurial-scm.org/ +.. _Google Code: http://sqlalchemy-migrate.googlecode.com/hg/ .. _issue tracker: http://code.google.com/p/sqlalchemy-migrate/issues/list -.. _buildbot: http://buildbot.fubar.si +.. _hudson: http://hudson.fubar.si diff --git a/docs/index.rst b/docs/index.rst index b8c8e68..dcecd5d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -25,7 +25,7 @@ SQLAlchemy Migrate. Currently, sqlalchemy-migrate supports Python versions from 2.4 to 2.6. - SQLAlchemy >=0.5 is supported only. + SQLAlchemy Migrate 0.6.0 supports SQLAlchemy both 0.5.x and 0.6.x branches. .. warning:: @@ -44,34 +44,34 @@ Download and Development Dialect support --------------- -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ -| Operation / Dialect | :ref:`sqlite ` | :ref:`postgres ` | :ref:`mysql ` | :ref:`oracle ` | :ref:`firebird ` | mssql | -| | | | | | | | -+=========================================================+==========================+==============================+========================+===========================+===============================+=======+ -| :ref:`ALTER TABLE RENAME TABLE ` | yes | yes | yes | yes | no | | -| | | | | | | | -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ -| :ref:`ALTER TABLE RENAME COLUMN ` | yes | yes | yes | yes | yes | | -| | (workaround) [#1]_ | | | | | | -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ -| :ref:`ALTER TABLE ADD COLUMN ` | yes | yes | yes | yes | yes | | -| | (with limitations) [#2]_ | | | | | | -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ -| :ref:`ALTER TABLE DROP COLUMN ` | yes | yes | yes | yes | yes | | -| | (workaround) [#1]_ | | | | | | -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ -| :ref:`ALTER TABLE ALTER COLUMN ` | yes | yes | yes | yes | yes [#4]_ | | -| | (workaround) [#1]_ | | | (with limitations) [#3]_ | | | -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ -| :ref:`ALTER TABLE ADD CONSTRAINT ` | no | yes | yes | yes | yes | | -| | | | | | | | -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ -| :ref:`ALTER TABLE DROP CONSTRAINT `| no | yes | yes | yes | yes | | -| | | | | | | | -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ -| :ref:`RENAME INDEX ` | no | yes | no | yes | yes | | -| | | | | | | | -+---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+-------+ ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ +| Operation / Dialect | :ref:`sqlite ` | :ref:`postgres ` | :ref:`mysql ` | :ref:`oracle ` | :ref:`firebird ` | mssql | +| | | | | | | | ++=========================================================+==========================+==============================+========================+===========================+===============================+====================+ +| :ref:`ALTER TABLE RENAME TABLE ` | yes | yes | yes | yes | no | not supported | +| | | | | | | | ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ +| :ref:`ALTER TABLE RENAME COLUMN ` | yes | yes | yes | yes | yes | not supported | +| | (workaround) [#1]_ | | | | | | ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ +| :ref:`ALTER TABLE ADD COLUMN ` | yes | yes | yes | yes | yes | not supported | +| | (with limitations) [#2]_ | | | | | | ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ +| :ref:`ALTER TABLE DROP COLUMN ` | yes | yes | yes | yes | yes | not supported | +| | (workaround) [#1]_ | | | | | | ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ +| :ref:`ALTER TABLE ALTER COLUMN ` | yes | yes | yes | yes | yes [#4]_ | not supported | +| | (workaround) [#1]_ | | | (with limitations) [#3]_ | | | ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ +| :ref:`ALTER TABLE ADD CONSTRAINT ` | no | yes | yes | yes | yes | not supported | +| | | | | | | | ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ +| :ref:`ALTER TABLE DROP CONSTRAINT `| no | yes | yes | yes | yes | not supported | +| | | | | | | | ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ +| :ref:`RENAME INDEX ` | no | yes | no | yes | yes | not supported | +| | | | | | | | ++---------------------------------------------------------+--------------------------+------------------------------+------------------------+---------------------------+-------------------------------+--------------------+ .. [#1] Table is renamed to temporary table, new table is created followed by INSERT statements. @@ -82,10 +82,9 @@ Dialect support Documentation ------------- -SQLAlchemy is split into two parts, database schema versioning and -database changeset management. This is represented by two python -packages :mod:`migrate.versioning` and :mod:`migrate.changeset`. The -versioning API is available as the :ref:`migrate ` command. +SQLAlchemy Migrate is split into two parts, database schema versioning (:mod:`migrate.versioning`) and +database migration management (:mod:`migrate.changeset`). +The versioning API is available as the :ref:`migrate ` command. .. toctree:: diff --git a/docs/versioning.rst b/docs/versioning.rst index 5a729f7..a361306 100644 --- a/docs/versioning.rst +++ b/docs/versioning.rst @@ -1,6 +1,6 @@ .. _versioning-system: .. currentmodule:: migrate.versioning -.. highlight:: bash +.. highlight:: console *********************************** Database schema versioning workflow @@ -22,36 +22,39 @@ Create a change repository -------------------------- To begin, we'll need to create a *repository* for our -project. Repositories are associated with a single database schema, -and store collections of change scripts to manage that schema. The -scripts in a repository may be applied to any number of databases. +project. -Repositories each have a name. This name is used to identify the -repository we're working with. - -All work with repositories is done using the migrate command. Let's +All work with repositories is done using the :ref:`migrate ` command. Let's create our project's repository:: $ migrate create my_repository "Example project" This creates an initially empty repository relative to current directory at -my_repository/ named `Example project`. The repository directory -contains a sub directory versions that will store the schema versions, -a configuration file :file:`migrate.cfg` that contains -:ref:`repository configuration `, a -:file:`README` file containing information that the directory is an -sqlalchemy-migrate repository and a script :ref:`manage.py ` -that has the same functionality as the :ref:`migrate ` command but is -preconfigured with the repository. +my_repository/ named `Example project`. -Version-control a database +The repository directory +contains a sub directory :file:`versions` that will store the :ref:`schema versions `, +a configuration file :file:`migrate.cfg` that contains +:ref:`repository configuration ` and a script :ref:`manage.py ` +that has the same functionality as the :ref:`migrate ` command but is +preconfigured with repository specific parameters. + +.. note:: + + Repositories are associated with a single database schema, + and store collections of change scripts to manage that schema. The + scripts in a repository may be applied to any number of databases. + Each repository has an unique name. This name is used to identify the + repository we're working with. + + +Version control a database -------------------------- -Next, we need to create a database and declare it to be under version -control. Information on a database's version is stored in the database +Next we need to declare database to be under version control. +Information on a database's version is stored in the database itself; declaring a database to be under version control creates a -table, named 'migrate_version' by default, and associates it with your -repository. +table named **migrate_version** and associates it with your repository. The database is specified as a `SQLAlchemy database url`_. @@ -102,12 +105,17 @@ and use it to perform commands:: 0 The script manage.py was created. All commands we perform with it are -the same as those performed with the 'migrate' tool, using the +the same as those performed with the :ref:`migrate ` tool, using the repository and database connection entered above. The difference between the script :file:`manage.py` in the current directory and the script inside the repository is, that the one in the current directory has the database URL preconfigured. +.. note:: + + Parameters specified in manage.py should be the same as in :ref:`versioning api `. + Preconfigured parameter should just be omitted from :ref:`migrate ` command. + Making schema changes ===================== @@ -139,47 +147,57 @@ This creates an empty change script at :file:`my_repository/versions/001_Add_account_table.py`. Next, we'll edit this script to create our table. + Edit the change script ---------------------- -Our change script defines two functions, currently empty: -:func:`upgrade`` and :func:`downgrade`. We'll fill those in +Our change script predefines two functions, currently empty: +:func:`upgrade` and :func:`downgrade`. We'll fill those in .. code-block:: python - from sqlalchemy import * - from migrate import * + from sqlalchemy import * + from migrate import * + + meta = MetaData() + + account = Table('account', meta, + Column('id', Integer, primary_key=True), + Column('login', String(40)), + Column('passwd', String(40)), + ) - meta = MetaData() - account = Table('account', meta, - Column('id', Integer, primary_key=True), - Column('login', String(40)), - Column('passwd', String(40)), - ) - - def upgrade(migrate_engine): - meta.bind = migrate_engine - account.create() - - def downgrade(migrate_engine): - meta.bind = migrate_engine - account.drop() + def upgrade(migrate_engine): + meta.bind = migrate_engine + account.create() + + def downgrade(migrate_engine): + meta.bind = migrate_engine + account.drop() As you might have guessed, :func:`upgrade` upgrades the database to the next -version. This function should contain the changes we want to perform; -here, we're creating a table. :func:`downgrade` should reverse changes made +version. This function should contain the :ref:`schema changes` we want to perform +(in our example we're creating a table). + +:func:`downgrade` should reverse changes made by :func:`upgrade`. You'll need to write both functions for every change script. (Well, you don't *have* to write downgrade, but you won't be able to revert to an older version of the database or test your scripts without it.) -As you can see, **migrate_engine** is passed to both functions. -You should use this in your change scripts, rather -than creating your own engine. -You should be very careful about importing files from the rest of your -application, as your change scripts might break when your application -changes. More about `writing scripts with consistent behavior`_. +.. note:: + + As you can see, **migrate_engine** is passed to both functions. + You should use this in your change scripts, rather + than creating your own engine. + +.. warning:: + + You should be very careful about importing files from the rest of your + application, as your change scripts might break when your application + changes. More about `writing scripts with consistent behavior`_. + Test the change script ------------------------ @@ -189,7 +207,7 @@ script will run its :func:`upgrade` and :func:`downgrade` functions on a specifi database; you can ensure the script runs without error. You should be testing on a test database - if something goes wrong here, you'll need to correct it by hand. If the test is successful, the database should -appear unchanged after upgrade() and downgrade() run. +appear unchanged after :func:`upgrade` and :func:`downgrade` run. To test the script:: @@ -201,14 +219,14 @@ To test the script:: Our script runs on our database (``sqlite:///project.db``, as specified in manage.py) without any errors. -Our repository's version now is:: +Our repository's version is:: $ python manage.py version 1 .. warning:: - test command exectues actual script, be sure you are NOT doing this on production database. + test command executes actual script, be sure you are NOT doing this on production database. Upgrade the database @@ -254,10 +272,11 @@ every time, despite any changes to your app's source code. You don't want your change scripts' behavior changing when your source code does. -**Consider the following example of what can go wrong (i.e. what NOT to -do)**: +.. warning:: -Your application defines a table in the model.py file: + **Consider the following example of what NOT to do** + +Let's say your application defines a table in the :file:`model.py` file: .. code-block:: python @@ -284,10 +303,13 @@ Your application defines a table in the model.py file: model.table.drop() This runs successfully the first time. But what happens if we change -the table definition? +the table definition in :file:`model.py`? .. code-block:: python + from sqlalchemy import * + + meta = MetaData() table = Table('mytable', meta, Column('id', Integer, primary_key=True), Column('data', String(42)), @@ -309,6 +331,7 @@ We'll create a new column with a matching change script model.meta.bind = migrate_engine model.table.drop() + This appears to run fine when upgrading an existing database - but the first script's behavior changed! Running all our change scripts on a new database will result in an error - the first script creates the @@ -319,6 +342,9 @@ To avoid the above problem, you should copy-paste your table definition into each change script rather than importing parts of your application. +.. note:: Sometimes it is enough to just reflect tables with SQLAlchemy instead of copy-pasting - but remember, explicit is better than implicit! + + Writing for a specific database ------------------------------- @@ -340,9 +366,7 @@ Writings .sql scripts --------------------- You might prefer to write your change scripts in SQL, as .sql files, -rather than as Python scripts. SQLAlchemy-migrate can work with that - -.. code-block:: python +rather than as Python scripts. SQLAlchemy-migrate can work with that:: $ python manage.py version 1 @@ -389,10 +413,10 @@ and you have a project management script that looks like you have first two slots filed, and command line usage would look like:: # preview Python script - migrate downgrade 2 --preview_py + $ migrate downgrade 2 --preview_py # downgrade to version 2 - migrate downgrade 2 + $ migrate downgrade 2 .. versionchanged:: 0.5.4 Command line parsing refactored: positional parameters usage @@ -442,6 +466,7 @@ For example, the following commands are similar: .. _repository_configuration: + Experimental commands ===================== @@ -470,6 +495,7 @@ As this sections headline says: These features are EXPERIMENTAL. Take the necessary arguments to the commands from the output of ``migrate help ``. + Repository configuration ======================== diff --git a/migrate/__init__.py b/migrate/__init__.py index 8bc84da..c8cf317 100644 --- a/migrate/__init__.py +++ b/migrate/__init__.py @@ -5,5 +5,9 @@ using Python. """ +from migrate.versioning import * +from migrate.changeset import * + + class MigrateDeprecationWarning(DeprecationWarning): """Warning for deprecated features in Migrate"""