Add tutorial section on creating plugins

Expand and reorg the naming discussion.

Add a section on the mechanics of creating plugins,
with some tested example code.

Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>
This commit is contained in:
Doug Hellmann 2013-06-05 17:10:26 -04:00
parent 6acd86a666
commit fd4e42f0fe
13 changed files with 349 additions and 77 deletions

View File

@ -0,0 +1,130 @@
==================
Creating Plugins
==================
After a lot of trial and error, the easiest way I have found to define
an API is to follow these steps:
#. Use the `abc module`_ to create a base abstract class to define the
behaviors required of plugins of the API. Developers don't have to
subclass from the base class, but it provides a convenient way to
document the API, and using an abstract base class keeps you
honest.
#. Create plugins by subclassing the base class and implementing the
required methods.
#. Define a unique namespace for each API by combining the name of the
application (or library) and a name of the API. Keep it
shallow. For example, "cliff.formatters" or
"ceilometer.pollsters.compute".
Example Plugin Set
==================
The example program in this tutorial will create a plugin set with
several data formatters, like what might be used by a command line
program to prepare data to be printed to the console. Each formatter
will take as input a dictionary with string keys and built-in data
types as values. It will return as output an iterator that produces
the string with the data structure formatted based on the rules of the
specific formatter being used. The formatter's constructor lets the
caller specify the maximum width the output should have.
A Plugin Base Class
===================
Step 1 above is to define an abstract base class for the API that
needs to be implemented by each plugin.
.. literalinclude:: ../../../stevedore/example/base.py
:language: python
:linenos:
:prepend: # stevedore/example/base.py
The constructor is a concrete method because subclasses do not need to
override it, but the :func:`format` method does not do anything useful
because there is no "default" implementation available.
Concrete Plugins
================
The next step is to create a couple of plugin classes with concrete
implementations of :func:`format`. A simple example formatter produces
output with each variable name and value on a single line.
.. literalinclude:: ../../../stevedore/example/simple.py
:language: python
:linenos:
:prepend: # stevedore/example/simple.py
An alternate implementation produces a reStructuredText `field list`_.
.. literalinclude:: ../../../stevedore/example/fields.py
:language: python
:linenos:
:prepend: # stevedore/example/fields.py
There are plenty of other formatting options, but these two examples
will give us enough to work with to demonstrate registering and using
pluins.
Registering the Plugins
=======================
To use setuptools entry points, you must package your application or
library using setuptools. The build and packaging process generates
metadata which is available after installation to find the plugins
provided by each python distribution.
The entry points must be declared as belonging to a specific
namespace, so we need to pick one before going any further. These
plugins are formatters from the stevedore examples, so I will use the
namespace "stevedore.example.formatter". Now it is possible to provide
all of the necessary information in the packaging instructions:
.. literalinclude:: ../../../stevedore/example/setup.py
:language: python
:linenos:
:emphasize-lines: 38-44
:prepend: # stevedore/example/setup.py
The important lines are 38-44. The ``entry_points`` argument to
:func:`setup` is a dictionary mapping the namespace for the plugins to
a list of their definitions. Each item in the list should be a string
with ``name = module:importable`` where *name* is the user-visible
name for the plugin, *module* is the Python import reference for the
module, and *importable* is the name of something that can be imported
from inside the module.
.. literalinclude:: ../../../stevedore/example/setup.py
:language: python
:lines: 37-43
In this case, there are three plugins registered. The "simple" and
"field" plugins defined above, and a "plain" plugin, which is just an
alias for the simple plugin.
setuptools Metadata
===================
During the build, setuptools copies entry point definitions to a file
in the ".egg-info" directory for the package. For example, the file
for stevedore is located in ``stevedore.egg-info/entry_points.txt``:
::
[stevedore.example.formatter]
simple = stevedore.example.simple:Simple
field = stevedore.example.fields:FieldList
plain = stevedore.example.simple:Simple
[stevedore.test.extension]
t2 = stevedore.tests.test_extension:FauxExtension
t1 = stevedore.tests.test_extension:FauxExtension
:mod:`pkg_resources` uses the ``entry_points.txt`` file from all of
the installed packages on the import path to find plugins. You should
not modify these files, except by changing the list of entry points in
``setup.py``.
.. _abc module: http://docs.python.org/2/library/abc.html
.. _field list: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#field-lists

View File

@ -1,68 +0,0 @@
=====================================
Guidelines for Implementing Plugins
=====================================
Stevedore uses setuptools entry points to define and load plugins. An
entry point is standard way to refer to a named object defined inside
a Python module or package. The name can be a reference to any class,
function, or instance, as long as it is created when the containing
module is imported (i.e., it needs to be a module-level global).
Names and Namespaces
====================
Entry points are registered using a *name* in a *namespace*.
Entry point names are usually considered user-visible. For example,
they frequently appear in configuration files where a driver is being
enabled. Because they are public, names are typically as short as
possible while remaining descriptive. For example, database driver
plugin names might be "mysql", "postgresql", "sqlite", etc.
Namespaces, on the other hand, are an implementation detail, and while
they are known to developers they are not usually exposed to users.
The namespace naming syntax looks a lot like Python's package syntax
(``a.b.c``) but *namespaces do not correspond to Python packages*. It
can be convenient to use a package name as a namespace, but it's not
required at all. The main feature of entry points is that they can be
discovered *across* packages. That means that a plugin can be
developed and installed completely separately from the application
that uses it, as long as they agree on the namespace and API.
Each namespace is owned by the code that consumes the plugins and is
used to search for entry points. The entry point names are typically
owned by the plugin, but they can also be defined by the consuming
code for named hooks (see :class:`~stevedore.hook.HookManager`). The
names of entry points must be unique within a given distribution, but
are not necessarily unique in a namespace (again, for hook patterns).
Keeping it Simple
=================
After a lot of trial and error, the easiest way I have found to define
an API is to follow these steps:
1. Define a unique namespace for each API by combining the name of the
application (or library) and a name of the API. Keep it
shallow. For example, "cliff.formatters" or
"ceilometer.pollsters.compute".
2. Use the `abc module`_ to create a base abstract class to define the
behaviors required of plugins of the API.
3. Create plugins by subclassing the base class and implementing the
required methods.
Developers don't have to subclass from the base class, but it provides
a convenient way to document the API, and using an abstract base class
keeps you honest.
.. seealso::
* `abc module`_
* `Using setuptools entry points`_
* `Package Discovery and Resource Access using pkg_resources`_
* `Using Entry Points to Write Plugins | Pylons`_
.. _Using setuptools entry points: http://reinout.vanrees.org/weblog/2010/01/06/zest-releaser-entry-points.html
.. _Package Discovery and Resource Access using pkg_resources: http://pythonhosted.org/distribute/pkg_resources.html
.. _Using Entry Points to Write Plugins | Pylons: http://docs.pylonsproject.org/projects/pylons-webframework/en/latest/advanced_pylons/entry_points_and_plugins.html
.. _abc module: http://docs.python.org/2/library/abc.html

View File

@ -9,12 +9,19 @@ application.
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2
implementation naming
registration creating_plugins
loading loading
calling calling
testing testing
.. seealso:: .. seealso::
:doc:`/essays/pycon2013` * :doc:`/essays/pycon2013`
* `Using setuptools entry points`_
* `Package Discovery and Resource Access using pkg_resources`_
* `Using Entry Points to Write Plugins | Pylons`_
.. _Using setuptools entry points: http://reinout.vanrees.org/weblog/2010/01/06/zest-releaser-entry-points.html
.. _Package Discovery and Resource Access using pkg_resources: http://pythonhosted.org/distribute/pkg_resources.html
.. _Using Entry Points to Write Plugins | Pylons: http://docs.pylonsproject.org/projects/pylons-webframework/en/latest/advanced_pylons/entry_points_and_plugins.html

View File

@ -0,0 +1,38 @@
===============================
Guidelines for Naming Plugins
===============================
Stevedore uses setuptools entry points to define and load plugins. An
entry point is standard way to refer to a named object defined inside
a Python module or package. The name can be a reference to any class,
function, or instance, as long as it is created when the containing
module is imported (i.e., it needs to be a module-level global).
Names and Namespaces
====================
Entry points are registered using a *name* in a *namespace*.
Entry point names are usually considered user-visible. For example,
they frequently appear in configuration files where a driver is being
enabled. Because they are public, names are typically as short as
possible while remaining descriptive. For example, database driver
plugin names might be "mysql", "postgresql", "sqlite", etc.
Namespaces, on the other hand, are an implementation detail, and while
they are known to developers they are not usually exposed to users.
The namespace naming syntax looks a lot like Python's package syntax
(``a.b.c``) but *namespaces do not correspond to Python
packages*. Using a Python package name for an entry point namespace is
an easy way to ensure a unique name, but it's not required at all.
The main feature of entry points is that they can be discovered
*across* packages. That means that a plugin can be developed and
installed completely separately from the application that uses it, as
long as they agree on the namespace and API.
Each namespace is owned by the code that consumes the plugins and is
used to search for entry points. The entry point names are typically
owned by the plugin, but they can also be defined by the consuming
code for named hooks (see :class:`~stevedore.hook.HookManager`). The
names of entry points must be unique within a given distribution, but
are not necessarily unique in a namespace.

View File

@ -1,5 +0,0 @@
======================
Registering a Plugin
======================
.. describe basics of defining plugins with setuptools and entry points

View File

@ -52,6 +52,11 @@ setup(
include_package_data=True, include_package_data=True,
entry_points={ entry_points={
'stevedore.example.formatter': [
'simple = stevedore.example.simple:Simple',
'field = stevedore.example.fields:FieldList',
'plain = stevedore.example.simple:Simple',
],
'stevedore.test.extension': [ 'stevedore.test.extension': [
't1 = stevedore.tests.test_extension:FauxExtension', 't1 = stevedore.tests.test_extension:FauxExtension',
't2 = stevedore.tests.test_extension:FauxExtension', 't2 = stevedore.tests.test_extension:FauxExtension',

View File

21
stevedore/example/base.py Normal file
View File

@ -0,0 +1,21 @@
import abc
class FormatterBase(object):
"""Base class for example plugin used in the tutoral.
"""
__metaclass__ = abc.ABCMeta
def __init__(self, max_width=60):
self.max_width = max_width
@abc.abstractmethod
def format(self, data):
"""Format the data and return unicode text.
:param data: A dictionary with string keys and simple types as
values.
:type data: dict(str:?)
:returns: Iterable producing the formatted text.
"""

View File

@ -0,0 +1,36 @@
import textwrap
from stevedore.example import base
class FieldList(base.FormatterBase):
"""Format values as a reStructuredText field list.
For example::
: name1 : value
: name2 : value
: name3 : a long value
will be wrapped with
a hanging indent
"""
def format(self, data):
"""Format the data and return unicode text.
:param data: A dictionary with string keys and simple types as
values.
:type data: dict(str:?)
"""
for name, value in sorted(data.items()):
full_text = ': {name} : {value}'.format(
name=name,
value=value,
)
wrapped_text = textwrap.fill(
full_text,
initial_indent='',
subsequent_indent=' ',
width=self.max_width,
)
yield wrapped_text + '\n'

View File

@ -0,0 +1,46 @@
from setuptools import setup, find_packages
setup(
name='stevedore-examples',
version='1.0',
description='Demonstration package for stevedore',
author='Doug Hellmann',
author_email='doug.hellmann@dreamhost.com',
url='https://github.com/dreamhost/stevedore',
download_url='https://github.com/dreamhost/stevedore/tarball/master',
classifiers=['Development Status :: 3 - Alpha',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Intended Audience :: Developers',
'Environment :: Console',
],
platforms=['Any'],
scripts=[],
provides=['stevedore.examples',
],
packages=find_packages(),
include_package_data=True,
entry_points={
'stevedore.example.formatter': [
'simple = stevedore.example.simple:Simple',
'field = stevedore.example.fields:FieldList',
'plain = stevedore.example.simple:Simple',
],
},
zip_safe=False,
)

View File

@ -0,0 +1,20 @@
from stevedore.example import base
class Simple(base.FormatterBase):
"""A very basic formatter.
"""
def format(self, data):
"""Format the data and return unicode text.
:param data: A dictionary with string keys and simple types as
values.
:type data: dict(str:?)
"""
for name, value in sorted(data.items()):
line = '{name} = {value}\n'.format(
name=name,
value=value,
)
yield line

View File

@ -0,0 +1,27 @@
"""Tests for stevedore.exmaple.fields
"""
from stevedore.example import fields
def test_simple_items():
f = fields.FieldList(100)
text = ''.join(f.format({'a': 'A', 'b': 'B'}))
expected = '\n'.join([
': a : A',
': b : B',
'',
])
assert text == expected
def test_long_item():
f = fields.FieldList(25)
text = ''.join(f.format({'name': 'a value longer than the allowed width'}))
expected = '\n'.join([
': name : a value longer',
' than the allowed',
' width',
'',
])
assert text == expected

View File

@ -0,0 +1,15 @@
"""Tests for stevedore.exmaple.simple
"""
from stevedore.example import simple
def test_simple_items():
f = simple.Simple(100)
text = ''.join(f.format({'a': 'A', 'b': 'B'}))
expected = '\n'.join([
'a = A',
'b = B',
'',
])
assert text == expected