You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
383 lines
12 KiB
ReStructuredText
383 lines
12 KiB
ReStructuredText
Schema validation just got Pythonic
|
|
===============================================================================
|
|
|
|
**schema** is a library for validating Python data structures, such as those
|
|
obtained from config-files, forms, external services or command-line
|
|
parsing, converted from JSON/YAML (or something else) to Python data-types.
|
|
|
|
|
|
.. image:: https://secure.travis-ci.org/keleshev/schema.png?branch=master
|
|
:target: https://travis-ci.org/keleshev/schema
|
|
|
|
.. image:: https://img.shields.io/codecov/c/github/keleshev/schema.svg
|
|
:target: http://codecov.io/github/keleshev/schema
|
|
|
|
Example
|
|
----------------------------------------------------------------------------
|
|
|
|
Here is a quick example to get a feeling of **schema**, validating a list of
|
|
entries with personal information:
|
|
|
|
.. code:: python
|
|
|
|
>>> from schema import Schema, And, Use, Optional
|
|
|
|
>>> schema = Schema([{'name': And(str, len),
|
|
... 'age': And(Use(int), lambda n: 18 <= n <= 99),
|
|
... Optional('sex'): And(str, Use(str.lower),
|
|
... lambda s: s in ('male', 'female'))}])
|
|
|
|
>>> data = [{'name': 'Sue', 'age': '28', 'sex': 'FEMALE'},
|
|
... {'name': 'Sam', 'age': '42'},
|
|
... {'name': 'Sacha', 'age': '20', 'sex': 'Male'}]
|
|
|
|
>>> validated = schema.validate(data)
|
|
|
|
>>> assert validated == [{'name': 'Sue', 'age': 28, 'sex': 'female'},
|
|
... {'name': 'Sam', 'age': 42},
|
|
... {'name': 'Sacha', 'age' : 20, 'sex': 'male'}]
|
|
|
|
|
|
If data is valid, ``Schema.validate`` will return the validated data
|
|
(optionally converted with `Use` calls, see below).
|
|
|
|
If data is invalid, ``Schema`` will raise ``SchemaError`` exception.
|
|
|
|
|
|
Installation
|
|
-------------------------------------------------------------------------------
|
|
|
|
Use `pip <http://pip-installer.org>`_ or easy_install::
|
|
|
|
pip install schema
|
|
|
|
Alternatively, you can just drop ``schema.py`` file into your project—it is
|
|
self-contained.
|
|
|
|
- **schema** is tested with Python 2.6, 2.7, 3.2, 3.3 and PyPy.
|
|
- **schema** follows `semantic versioning <http://semver.org>`_.
|
|
|
|
How ``Schema`` validates data
|
|
-------------------------------------------------------------------------------
|
|
|
|
Types
|
|
~~~~~
|
|
|
|
If ``Schema(...)`` encounters a type (such as ``int``, ``str``, ``object``,
|
|
etc.), it will check if the corresponding piece of data is an instance of that type,
|
|
otherwise it will raise ``SchemaError``.
|
|
|
|
.. code:: python
|
|
|
|
>>> from schema import Schema
|
|
|
|
>>> Schema(int).validate(123)
|
|
123
|
|
|
|
>>> Schema(int).validate('123')
|
|
Traceback (most recent call last):
|
|
...
|
|
SchemaUnexpectedTypeError: '123' should be instance of 'int'
|
|
|
|
>>> Schema(object).validate('hai')
|
|
'hai'
|
|
|
|
Callables
|
|
~~~~~~~~~
|
|
|
|
If ``Schema(...)`` encounters a callable (function, class, or object with
|
|
``__call__`` method) it will call it, and if its return value evaluates to
|
|
``True`` it will continue validating, else—it will raise ``SchemaError``.
|
|
|
|
.. code:: python
|
|
|
|
>>> import os
|
|
|
|
>>> Schema(os.path.exists).validate('./')
|
|
'./'
|
|
|
|
>>> Schema(os.path.exists).validate('./non-existent/')
|
|
Traceback (most recent call last):
|
|
...
|
|
SchemaError: exists('./non-existent/') should evaluate to True
|
|
|
|
>>> Schema(lambda n: n > 0).validate(123)
|
|
123
|
|
|
|
>>> Schema(lambda n: n > 0).validate(-12)
|
|
Traceback (most recent call last):
|
|
...
|
|
SchemaError: <lambda>(-12) should evaluate to True
|
|
|
|
"Validatables"
|
|
~~~~~~~~~~~~~~
|
|
|
|
If ``Schema(...)`` encounters an object with method ``validate`` it will run
|
|
this method on corresponding data as ``data = obj.validate(data)``. This method
|
|
may raise ``SchemaError`` exception, which will tell ``Schema`` that that piece
|
|
of data is invalid, otherwise—it will continue validating.
|
|
|
|
An example of "validatable" is ``Regex``, that tries to match a string or a
|
|
buffer with the given regular expression (itself as a string, buffer or
|
|
compiled regex ``SRE_Pattern``):
|
|
|
|
.. code:: python
|
|
|
|
>>> from schema import Regex
|
|
>>> import re
|
|
|
|
>>> Regex(r'^foo').validate('foobar')
|
|
'foobar'
|
|
|
|
>>> Regex(r'^[A-Z]+$', flags=re.I).validate('those-dashes-dont-match')
|
|
Traceback (most recent call last):
|
|
...
|
|
SchemaError: Regex('^[A-Z]+$', flags=re.IGNORECASE) does not match 'those-dashes-dont-match'
|
|
|
|
For a more general case, you can use ``Use`` for creating such objects.
|
|
``Use`` helps to use a function or type to convert a value while validating it:
|
|
|
|
.. code:: python
|
|
|
|
>>> from schema import Use
|
|
|
|
>>> Schema(Use(int)).validate('123')
|
|
123
|
|
|
|
>>> Schema(Use(lambda f: open(f, 'a'))).validate('LICENSE-MIT')
|
|
<open file 'LICENSE-MIT', mode 'a' at 0x...>
|
|
|
|
Dropping the details, ``Use`` is basically:
|
|
|
|
.. code:: python
|
|
|
|
class Use(object):
|
|
|
|
def __init__(self, callable_):
|
|
self._callable = callable_
|
|
|
|
def validate(self, data):
|
|
try:
|
|
return self._callable(data)
|
|
except Exception as e:
|
|
raise SchemaError('%r raised %r' % (self._callable.__name__, e))
|
|
|
|
Now you can write your own validation-aware classes and data types.
|
|
|
|
Lists, similar containers
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
If ``Schema(...)`` encounters an instance of ``list``, ``tuple``, ``set`` or
|
|
``frozenset``, it will validate contents of corresponding data container
|
|
against schemas listed inside that container:
|
|
|
|
|
|
.. code:: python
|
|
|
|
>>> Schema([1, 0]).validate([1, 1, 0, 1])
|
|
[1, 1, 0, 1]
|
|
|
|
>>> Schema((int, float)).validate((5, 7, 8, 'not int or float here'))
|
|
Traceback (most recent call last):
|
|
...
|
|
SchemaError: Or(<type 'int'>, <type 'float'>) did not validate 'not int or float here'
|
|
'not int or float here' should be instance of 'float'
|
|
|
|
Dictionaries
|
|
~~~~~~~~~~~~
|
|
|
|
If ``Schema(...)`` encounters an instance of ``dict``, it will validate data
|
|
key-value pairs:
|
|
|
|
.. code:: python
|
|
|
|
>>> d = Schema({'name': str,
|
|
... 'age': lambda n: 18 <= n <= 99}).validate({'name': 'Sue', 'age': 28})
|
|
|
|
>>> assert d == {'name': 'Sue', 'age': 28}
|
|
|
|
You can specify keys as schemas too:
|
|
|
|
.. code:: python
|
|
|
|
>>> schema = Schema({str: int, # string keys should have integer values
|
|
... int: None}) # int keys should be always None
|
|
|
|
>>> data = schema.validate({'key1': 1, 'key2': 2,
|
|
... 10: None, 20: None})
|
|
|
|
>>> schema.validate({'key1': 1,
|
|
... 10: 'not None here'})
|
|
Traceback (most recent call last):
|
|
...
|
|
SchemaError: Key '10' error:
|
|
None does not match 'not None here'
|
|
|
|
This is useful if you want to check certain key-values, but don't care
|
|
about other:
|
|
|
|
.. code:: python
|
|
|
|
>>> schema = Schema({'<id>': int,
|
|
... '<file>': Use(open),
|
|
... str: object}) # don't care about other str keys
|
|
|
|
>>> data = schema.validate({'<id>': 10,
|
|
... '<file>': 'README.rst',
|
|
... '--verbose': True})
|
|
|
|
You can mark a key as optional as follows:
|
|
|
|
.. code:: python
|
|
|
|
>>> from schema import Optional
|
|
>>> Schema({'name': str,
|
|
... Optional('occupation'): str}).validate({'name': 'Sam'})
|
|
{'name': 'Sam'}
|
|
|
|
``Optional`` keys can also carry a ``default``, to be used when no key in the
|
|
data matches:
|
|
|
|
.. code:: python
|
|
|
|
>>> from schema import Optional
|
|
>>> Schema({Optional('color', default='blue'): str,
|
|
... str: str}).validate({'texture': 'furry'}
|
|
... ) == {'color': 'blue', 'texture': 'furry'}
|
|
True
|
|
|
|
Defaults are used verbatim, not passed through any validators specified in the
|
|
value.
|
|
|
|
**schema** has classes ``And`` and ``Or`` that help validating several schemas
|
|
for the same data:
|
|
|
|
.. code:: python
|
|
|
|
>>> from schema import And, Or
|
|
|
|
>>> Schema({'age': And(int, lambda n: 0 < n < 99)}).validate({'age': 7})
|
|
{'age': 7}
|
|
|
|
>>> Schema({'password': And(str, lambda s: len(s) > 6)}).validate({'password': 'hai'})
|
|
Traceback (most recent call last):
|
|
...
|
|
SchemaError: Key 'password' error:
|
|
<lambda>('hai') should evaluate to True
|
|
|
|
>>> Schema(And(Or(int, float), lambda x: x > 0)).validate(3.1415)
|
|
3.1415
|
|
|
|
Extra Keys
|
|
~~~~~~~~~~
|
|
|
|
The ``Schema(...)`` parameter ``ignore_extra_keys`` causes validation to ignore extra keys in a dictionary, and also to not return them after validating.
|
|
|
|
.. code:: python
|
|
|
|
>>> schema = Schema({'name': str}, ignore_extra_keys=True)
|
|
>>> schema.validate({'name': 'Sam', 'age': '42'})
|
|
{'name': 'Sam'}
|
|
|
|
If you would like any extra keys returned, use ``object: object`` as one of the key/value pairs, which will match any key and any value.
|
|
Otherwise, extra keys will raise a ``SchemaError``.
|
|
|
|
User-friendly error reporting
|
|
-------------------------------------------------------------------------------
|
|
|
|
You can pass a keyword argument ``error`` to any of validatable classes
|
|
(such as ``Schema``, ``And``, ``Or``, ``Regex``, ``Use``) to report this error
|
|
instead of a built-in one.
|
|
|
|
.. code:: python
|
|
|
|
>>> Schema(Use(int, error='Invalid year')).validate('XVII')
|
|
Traceback (most recent call last):
|
|
...
|
|
SchemaError: Invalid year
|
|
|
|
You can see all errors that occurred by accessing exception's ``exc.autos``
|
|
for auto-generated error messages, and ``exc.errors`` for errors
|
|
which had ``error`` text passed to them.
|
|
|
|
You can exit with ``sys.exit(exc.code)`` if you want to show the messages
|
|
to the user without traceback. ``error`` messages are given precedence in that
|
|
case.
|
|
|
|
A JSON API example
|
|
-------------------------------------------------------------------------------
|
|
|
|
Here is a quick example: validation of
|
|
`create a gist <http://developer.github.com/v3/gists/>`_
|
|
request from github API.
|
|
|
|
.. code:: python
|
|
|
|
>>> gist = '''{"description": "the description for this gist",
|
|
... "public": true,
|
|
... "files": {
|
|
... "file1.txt": {"content": "String file contents"},
|
|
... "other.txt": {"content": "Another file contents"}}}'''
|
|
|
|
>>> from schema import Schema, And, Use, Optional
|
|
|
|
>>> import json
|
|
|
|
>>> gist_schema = Schema(And(Use(json.loads), # first convert from JSON
|
|
... # use basestring since json returns unicode
|
|
... {Optional('description'): basestring,
|
|
... 'public': bool,
|
|
... 'files': {basestring: {'content': basestring}}}))
|
|
|
|
>>> gist = gist_schema.validate(gist)
|
|
|
|
# gist:
|
|
{u'description': u'the description for this gist',
|
|
u'files': {u'file1.txt': {u'content': u'String file contents'},
|
|
u'other.txt': {u'content': u'Another file contents'}},
|
|
u'public': True}
|
|
|
|
Using **schema** with `docopt <http://github.com/docopt/docopt>`_
|
|
-------------------------------------------------------------------------------
|
|
|
|
Assume you are using **docopt** with the following usage-pattern:
|
|
|
|
Usage: my_program.py [--count=N] <path> <files>...
|
|
|
|
and you would like to validate that ``<files>`` are readable, and that
|
|
``<path>`` exists, and that ``--count`` is either integer from 0 to 5, or
|
|
``None``.
|
|
|
|
Assuming **docopt** returns the following dict:
|
|
|
|
.. code:: python
|
|
|
|
>>> args = {'<files>': ['LICENSE-MIT', 'setup.py'],
|
|
... '<path>': '../',
|
|
... '--count': '3'}
|
|
|
|
this is how you validate it using ``schema``:
|
|
|
|
.. code:: python
|
|
|
|
>>> from schema import Schema, And, Or, Use
|
|
>>> import os
|
|
|
|
>>> s = Schema({'<files>': [Use(open)],
|
|
... '<path>': os.path.exists,
|
|
... '--count': Or(None, And(Use(int), lambda n: 0 < n < 5))})
|
|
|
|
>>> args = s.validate(args)
|
|
|
|
>>> args['<files>']
|
|
[<open file 'LICENSE-MIT', mode 'r' at 0x...>, <open file 'setup.py', mode 'r' at 0x...>]
|
|
|
|
>>> args['<path>']
|
|
'../'
|
|
|
|
>>> args['--count']
|
|
3
|
|
|
|
As you can see, **schema** validated data successfully, opened files and
|
|
converted ``'3'`` to ``int``.
|