openstack/deb-python-colander
Code Issues Proposed changes

- Get rid of `pserialize in favor of making serialize` always

Browse Source 
partially serialize.

- Get rid of ``pdeserialize``: it existed only for symmetry.  We'll
  add something like it back later if we need it.
This commit is contained in:
Chris McDonough
2010-04-03 20:16:31 +00:00
parent a3eb5928f9
commit 62f6252cb2
4 changed files with 59 additions and 208 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
CHANGES.txt
View File

@@ -6,6 +6,12 @@ Next release
- Fix bug in serialization of non-Unicode values in the ``String`` class. - Fix bug in serialization of non-Unicode values in the ``String`` class.
- Get rid of ``pserialize`` in favor of making ``serialize`` always
partially serialize.
- Get rid of ``pdeserialize``: it existed only for symmetry. We'll
add something like it back later if we need it.
0.5.1 (2010-04-02) 0.5.1 (2010-04-02)
------------------ ------------------

90
colander/__init__.py
View File

@@ -189,15 +189,7 @@ class OneOf(object):
raise Invalid(node, '"%s" is not one of %s' % ( raise Invalid(node, '"%s" is not one of %s' % (
value, ', '.join(['%s' % x for x in self.values]))) value, ', '.join(['%s' % x for x in self.values])))
class Type(object): class Mapping(object):
""" Abstract base class for types (only for convenience) """
def pserialize(self, node, value):
return self.serialize(node, value)
def pdeserialize(self, node, value):
return self.deserialize(node, value)
class Mapping(Type):
""" A type which represents a mapping of names to nodes. """ A type which represents a mapping of names to nodes.
The subnodes of the :class:`colander.SchemaNode` that wraps The subnodes of the :class:`colander.SchemaNode` that wraps
@@ -209,9 +201,9 @@ class Mapping(Type):
control the behavior after construction. control the behavior after construction.
unknown unknown
``unknown`` controls the behavior of this type when an unknown ``unknown`` controls the behavior of this type when an
key is encountered in the value passed to the ``serialize`` or unknown key is encountered in the value passed to the
``deserialize`` methods of this instance. The potential ``deserialize`` method of this instance. The potential
values of ``unknown`` are: values of ``unknown`` are:
- ``ignore`` means that keys that are not present in the schema - ``ignore`` means that keys that are not present in the schema
@@ -222,30 +214,23 @@ class Mapping(Type):
raised when unknown keys are present during deserialization. raised when unknown keys are present during deserialization.
- ``preserve`` will preserve the 'raw' unknown keys and values - ``preserve`` will preserve the 'raw' unknown keys and values
in the returned data structure. in the returned data structure during deserialization.
Default: ``ignore``. Default: ``ignore``.
Note that the ``pserialize`` and ``pdeserialize`` methods of
this type will override this behavior, behaving as if
``unknown`` is ``ignore`` for the duration of those method calls.
partial partial
``partial`` controls the behavior of this type when a ``partial`` controls the behavior of this type when a
schema-expected key is missing from the value passed to the schema-expected key is missing from the value passed to the
``serialize`` and ``deserialize`` methods of this instance. ``deserialize`` method of this instance.
During serialization and deserialization, when ``partial`` is
``False``, a :exc:`colander.Invalid` exception will be raised During deserialization, when ``partial`` is ``False``, a
if the mapping value does not contain a key specified by the :exc:`colander.Invalid` exception will be raised if the
schema node related to this mapping type. When ``partial`` is mapping value does not contain a key specified by the schema
``True``, no exception is raised and a partial mapping will node related to this mapping type. When ``partial`` is
be serialized/deserialized. ``True``, no exception is raised and a partial mapping will be
deserialized.
Default: ``False``. Default: ``False``.
Note that the ``pserialize`` and ``pdeserialize`` methods of
this type will override this behavior, behaving as if
``partial`` is ``True`` for the duration of those method calls.
""" """
def __init__(self, unknown='ignore', partial=False): def __init__(self, unknown='ignore', partial=False):
@@ -322,7 +307,8 @@ class Mapping(Type):
return subnode.serialize(subval) return subnode.serialize(subval)
def default_callback(subnode): def default_callback(subnode):
return subnode.serialize(subnode.default) return subnode.serialize(subnode.default)
return self._impl(node, value, callback, default_callback) return self._impl(node, value, callback, default_callback,
unknown='ignore', partial=True)
def deserialize(self, node, value): def deserialize(self, node, value):
def callback(subnode, subval): def callback(subnode, subval):
@@ -331,24 +317,6 @@ class Mapping(Type):
return subnode.default return subnode.default
return self._impl(node, value, callback, default_callback) return self._impl(node, value, callback, default_callback)
def pserialize(self, node, value):
def callback(subnode, subval):
return subnode.pserialize(subval)
def default_callback(subnode):
return subnode.serialize(subnode.default)
return self._impl(
node, value, callback, default_callback, unknown='ignore',
partial=True)
def pdeserialize(self, node, value):
def callback(subnode, subval):
return subnode.pdeserialize(subval)
def default_callback(subnode):
return subnode.default
return self._impl(
node, value, callback, default_callback, unknown='ignore',
partial=True)
class Positional(object): class Positional(object):
""" """
Marker abstract base class meaning 'this type has children which Marker abstract base class meaning 'this type has children which
@@ -357,7 +325,7 @@ class Positional(object):
creating a dictionary representation of an error tree. creating a dictionary representation of an error tree.
""" """
class Tuple(Type, Positional): class Tuple(Positional):
""" A type which represents a fixed-length sequence of nodes. """ A type which represents a fixed-length sequence of nodes.
The subnodes of the :class:`colander.SchemaNode` that wraps The subnodes of the :class:`colander.SchemaNode` that wraps
@@ -411,7 +379,7 @@ class Tuple(Type, Positional):
return subnode.serialize(subval) return subnode.serialize(subval)
return self._impl(node, value, callback) return self._impl(node, value, callback)
class Sequence(Type, Positional): class Sequence(Positional):
""" """
A type which represents a variable-length sequence of nodes, A type which represents a variable-length sequence of nodes,
all of which must be of the same type. all of which must be of the same type.
@@ -519,7 +487,7 @@ Seq = Sequence
default_encoding = 'utf-8' default_encoding = 'utf-8'
class String(Type): class String(object):
""" A type representing a Unicode string. """ A type representing a Unicode string.
This type constructor accepts a single argument ``encoding``, This type constructor accepts a single argument ``encoding``,
@@ -568,7 +536,7 @@ class String(Type):
Str = String Str = String
class Integer(Type): class Integer(object):
""" A type representing an integer. """ A type representing an integer.
The subnodes of the :class:`colander.SchemaNode` that wraps The subnodes of the :class:`colander.SchemaNode` that wraps
@@ -592,7 +560,7 @@ class Integer(Type):
Int = Integer Int = Integer
class Float(Type): class Float(object):
""" A type representing a float. """ A type representing a float.
The subnodes of the :class:`colander.SchemaNode` that wraps The subnodes of the :class:`colander.SchemaNode` that wraps
@@ -616,7 +584,7 @@ class Float(Type):
Int = Integer Int = Integer
class Boolean(Type): class Boolean(object):
""" A type representing a boolean object. """ A type representing a boolean object.
During deserialization, a value in the set (``false``, ``0``) will During deserialization, a value in the set (``false``, ``0``) will
@@ -649,7 +617,7 @@ class Boolean(Type):
Bool = Boolean Bool = Boolean
class GlobalObject(Type): class GlobalObject(object):
""" A type representing an importable Python object. This type """ A type representing an importable Python object. This type
serializes 'global' Python objects (objects which can be imported) serializes 'global' Python objects (objects which can be imported)
to dotted Python names. to dotted Python names.
@@ -756,7 +724,7 @@ class GlobalObject(Type):
except AttributeError: except AttributeError:
raise Invalid(node, '%r has no __name__' % value) raise Invalid(node, '%r has no __name__' % value)
class DateTime(Type): class DateTime(object):
""" A type representing a Python ``datetime.datetime`` object. """ A type representing a Python ``datetime.datetime`` object.
This type serializes python ``datetime.datetime`` objects to a This type serializes python ``datetime.datetime`` objects to a
@@ -822,7 +790,7 @@ class DateTime(Type):
'exc':e}) 'exc':e})
return result return result
class Date(Type): class Date(object):
""" A type representing a Python ``datetime.date`` object. """ A type representing a Python ``datetime.date`` object.
This type serializes python ``datetime.date`` objects to a This type serializes python ``datetime.date`` objects to a
@@ -958,16 +926,6 @@ class SchemaNode(object):
this node.""" this node."""
return self.typ.serialize(self, value) return self.typ.serialize(self, value)
def pserialize(self, value):
""" Partially serialize the value based on the schema
represented by this node. """
return self.typ.pserialize(self, value)
def pdeserialize(self, value):
""" Partially deserialize the value based on the schema
represented by this node. """
return self.typ.pdeserialize(self, value)
def add(self, node): def add(self, node):
""" Add a subnode to this node. """ """ Add a subnode to this node. """
self.children.append(node) self.children.append(node)

114
colander/tests.py
View File

@@ -211,23 +211,6 @@ class TestOneOf(unittest.TestCase):
e = invalid_exc(validator, None, None) e = invalid_exc(validator, None, None)
self.assertEqual(e.msg, '"None" is not one of 1, 2') self.assertEqual(e.msg, '"None" is not one of 1, 2')
class TestType(unittest.TestCase):
def _makeOne(self):
from colander import Type
return Type()
def test_pserialize(self):
typ = self._makeOne()
typ.serialize = lambda *arg: arg
result = typ.pserialize(None, None)
self.assertEqual(result, (None, None))
def test_pdeserialize(self):
typ = self._makeOne()
typ.deserialize = lambda *arg: arg
result = typ.pdeserialize(None, None)
self.assertEqual(result, (None, None))
class TestMapping(unittest.TestCase): class TestMapping(unittest.TestCase):
def _makeOne(self, *arg, **kw): def _makeOne(self, *arg, **kw):
from colander import Mapping from colander import Mapping
@@ -339,101 +322,35 @@ class TestMapping(unittest.TestCase):
result = typ.serialize(node, {'a':1}) result = typ.serialize(node, {'a':1})
self.assertEqual(result, {'a':1}) self.assertEqual(result, {'a':1})
def test_serialize_unknown_raise(self):
node = DummySchemaNode(None)
node.children = [DummySchemaNode(None, name='a')]
typ = self._makeOne('raise')
e = invalid_exc(typ.serialize, node, {'a':1, 'b':2})
self.assertEqual(e.msg, "Unrecognized keys in mapping: {'b': 2}")
def test_serialize_unknown_preserve(self):
node = DummySchemaNode(None)
node.children = [DummySchemaNode(None, name='a')]
typ = self._makeOne(unknown='preserve')
result = typ.serialize(node, {'a':1, 'b':2})
self.assertEqual(result, {'a':1, 'b':2})
def test_serialize_subnodes_raise(self):
node = DummySchemaNode(None)
node.children = [
DummySchemaNode(None, name='a', exc='Wrong 2'),
DummySchemaNode(None, name='b', exc='Wrong 2'),
]
typ = self._makeOne()
e = invalid_exc(typ.serialize, node, {'a':1, 'b':2})
self.assertEqual(e.msg, None)
self.assertEqual(len(e.children), 2)
def test_serialize_subnode_missing_default(self):
node = DummySchemaNode(None)
node.children = [
DummySchemaNode(None, name='a'),
DummySchemaNode(None, name='b', default='abc'),
]
typ = self._makeOne()
result = typ.serialize(node, {'a':1})
self.assertEqual(result, {'a':1, 'b':'abc'})
def test_serialize_subnode_missing_nodefault(self):
node = DummySchemaNode(None)
node.children = [
DummySchemaNode(None, name='a'),
DummySchemaNode(None, name='b'),
]
typ = self._makeOne()
e = invalid_exc(typ.serialize, node, {'a':1})
self.assertEqual(e.children[0].msg, '"b" is required but missing')
def test_serialize_subnode_partial(self): def test_serialize_subnode_partial(self):
node = DummySchemaNode(None) node = DummySchemaNode(None)
node.children = [ node.children = [
DummySchemaNode(None, name='a'), DummySchemaNode(None, name='a'),
DummySchemaNode(None, name='b'), DummySchemaNode(None, name='b'),
] ]
typ = self._makeOne(partial=True) typ = self._makeOne()
result = typ.serialize(node, {'a':1}) result = typ.serialize(node, {'a':1})
self.assertEqual(result, {'a':1}) self.assertEqual(result, {'a':1})
def test_pserialize(self): def test_serialize_partial_with_default(self):
node = DummySchemaNode(None)
node.children = [
DummySchemaNode(None, name='a'),
DummySchemaNode(None, name='b'),
]
typ = self._makeOne()
result = typ.pserialize(node, {'a':1})
self.assertEqual(result, {'a':1})
def test_pserialize_using_default(self):
node = DummySchemaNode(None) node = DummySchemaNode(None)
node.children = [ node.children = [
DummySchemaNode(None, name='a'), DummySchemaNode(None, name='a'),
DummySchemaNode(None, name='b', default='abc'), DummySchemaNode(None, name='b', default='abc'),
] ]
typ = self._makeOne() typ = self._makeOne()
result = typ.pserialize(node, {'a':1}) result = typ.serialize(node, {'a':1})
self.assertEqual(result, {'a':1, 'b':'abc'}) self.assertEqual(result, {'a':1, 'b':'abc'})
def test_pdeserialize(self): def test_serialize_with_unknown(self):
node = DummySchemaNode(None) node = DummySchemaNode(None)
node.children = [ node.children = [
DummySchemaNode(None, name='a'), DummySchemaNode(None, name='a'),
DummySchemaNode(None, name='b'),
] ]
typ = self._makeOne() typ = self._makeOne()
result = typ.pdeserialize(node, {'a':1}) result = typ.serialize(node, {'a':1, 'b':2})
self.assertEqual(result, {'a':1}) self.assertEqual(result, {'a':1})
def test_pdeserialize_using_default(self):
node = DummySchemaNode(None)
node.children = [
DummySchemaNode(None, name='a'),
DummySchemaNode(None, name='b', default='abc'),
]
typ = self._makeOne()
result = typ.pdeserialize(node, {'a':1})
self.assertEqual(result, {'a':1, 'b':'abc'})
class TestTuple(unittest.TestCase): class TestTuple(unittest.TestCase):
def _makeOne(self): def _makeOne(self):
from colander import Tuple from colander import Tuple
@@ -1207,24 +1124,12 @@ class TestSchemaNode(unittest.TestCase):
e = invalid_exc(node.deserialize, 1) e = invalid_exc(node.deserialize, 1)
self.assertEqual(e.msg, 'Wrong') self.assertEqual(e.msg, 'Wrong')
def test_pdeserialize(self):
typ = DummyType()
node = self._makeOne(typ)
result = node.pdeserialize(1)
self.assertEqual(result, 1)
def test_serialize(self): def test_serialize(self):
typ = DummyType() typ = DummyType()
node = self._makeOne(typ) node = self._makeOne(typ)
result = node.serialize(1) result = node.serialize(1)
self.assertEqual(result, 1) self.assertEqual(result, 1)
def test_pserialize(self):
typ = DummyType()
node = self._makeOne(typ)
result = node.pserialize(1)
self.assertEqual(result, 1)
def test_add(self): def test_add(self):
node = self._makeOne(None) node = self._makeOne(None)
node.add(1) node.add(1)
@@ -1471,9 +1376,6 @@ class DummySchemaNode(object):
raise Invalid(self, self.exc) raise Invalid(self, self.exc)
return val return val
pserialize = serialize
pdeserialize = deserialize
class DummyValidator(object): class DummyValidator(object):
def __init__(self, msg=None): def __init__(self, msg=None):
self.msg = msg self.msg = msg
@@ -1496,9 +1398,3 @@ class DummyType(object):
def deserialize(self, node, value): def deserialize(self, node, value):
return value return value
def pserialize(self, node, value):
return value
def pdeserialize(self, node, value):
return value

57
docs/index.rst
View File

@@ -345,10 +345,10 @@ Serialization
Serializing a data structure is obviously the inverse operation from Serializing a data structure is obviously the inverse operation from
deserializing a data structure. The ``serialize`` method of a schema deserializing a data structure. The ``serialize`` method of a schema
performs serialization of application data. If you pass the performs serialization of application data (aka an ``appstruct``). If
``serialize`` method data that can be understood by the schema types you pass the ``serialize`` method data that can be understood by the
in the schema you're calling it against, you will be returned a data schema types in the schema you're calling it against, you will be
structure of serialized values. returned a data structure of serialized values.
For example, given the following schema: For example, given the following schema:
@@ -362,9 +362,8 @@ For example, given the following schema:
age = colander.SchemaNode(colander.Int(), age = colander.SchemaNode(colander.Int(),
validator=colander.Range(0, 200)) validator=colander.Range(0, 200))
If we try to serialize partial data using the ``serialize`` method of We can serialize a matching data structure:
the schema:
.. code-block:: python .. code-block:: python
:linenos: :linenos:
@@ -375,15 +374,18 @@ the schema:
The value for ``deserialized`` above will be ``{'age':'20', The value for ``deserialized`` above will be ``{'age':'20',
'name':'Bob'}`` (note the integer has become a string). 'name':'Bob'}`` (note the integer has become a string).
Note that validation of values happens during serialization, just as Serialization and deserialization are not completely symmetric,
it does during deserialization. however. Although schema-driven data conversion happens during
serialization, and defaults are injected as necessary, the default
:mod:`colander` types are defined in such a way that the validation of
values and structural validation does *not* happen as it does during
deserialization. For example, the ``required`` argument of a schema
is typically ignored, none of the validators associated with the
schema or any of is nodes is invoked.
Schema nodes also define a ``pserialize`` method, which can be used to This usually means you may "partially" serialize a data structure
"partially" serialize data. This is most useful when you want to where some of the values are missing. If we try to serialize partial
serialize a data structure where some of the values are missing. data using the ``serialize`` method of the schema:
For example, if we try to serialize partial data using the
``serialize`` method of the schema we defined above:
.. code-block:: python .. code-block:: python
:linenos: :linenos:
@@ -392,25 +394,14 @@ For example, if we try to serialize partial data using the
schema = Person() schema = Person()
deserialized = schema.serialize(data) deserialized = schema.serialize(data)
When we attempt to invoke ``serialize``, an :exc:`colander.Invalid` The value for ``deserialized`` above will be ``{'age':'20'}`` (note
error will be raised, because we did not include the ``name`` the integer has become a string). Above, even though we did not
attribute in our data. include the ``name`` attribute in the data we fed to ``serialize``, an
error is *not* raised.
To serialize with data representing only a part of the schema, use the The corollary: it is the responsibility of the developer to ensure he
``pserialize`` method: serializes "the right" data; :mod:`colander` will not raise an error
when asked to serialize something that is partially nonsense.
.. code-block:: python
:linenos:
data = {'age':20}
schema = Person()
deserialized = schema.pserialize(data)
No error is raised, and the value for ``deserialized`` above will be
``{'age':'20'}`` (note the integer has become a string).
A ``pdeserialize`` method also exists, which is a mirror image of
``pserialize`` for deserialization.
Defining A Schema Imperatively Defining A Schema Imperatively
------------------------------ ------------------------------