Add tutorial section on creating plugins
Expand and reorg the naming discussion. Add a section on the mechanics of creating plugins, with some tested example code. Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>
This commit is contained in:
parent
6acd86a666
commit
fd4e42f0fe
130
docs/source/tutorial/creating_plugins.rst
Normal file
130
docs/source/tutorial/creating_plugins.rst
Normal file
@ -0,0 +1,130 @@
|
||||
==================
|
||||
Creating Plugins
|
||||
==================
|
||||
|
||||
After a lot of trial and error, the easiest way I have found to define
|
||||
an API is to follow these steps:
|
||||
|
||||
#. Use the `abc module`_ to create a base abstract class to define the
|
||||
behaviors required of plugins of the API. Developers don't have to
|
||||
subclass from the base class, but it provides a convenient way to
|
||||
document the API, and using an abstract base class keeps you
|
||||
honest.
|
||||
#. Create plugins by subclassing the base class and implementing the
|
||||
required methods.
|
||||
#. Define a unique namespace for each API by combining the name of the
|
||||
application (or library) and a name of the API. Keep it
|
||||
shallow. For example, "cliff.formatters" or
|
||||
"ceilometer.pollsters.compute".
|
||||
|
||||
Example Plugin Set
|
||||
==================
|
||||
|
||||
The example program in this tutorial will create a plugin set with
|
||||
several data formatters, like what might be used by a command line
|
||||
program to prepare data to be printed to the console. Each formatter
|
||||
will take as input a dictionary with string keys and built-in data
|
||||
types as values. It will return as output an iterator that produces
|
||||
the string with the data structure formatted based on the rules of the
|
||||
specific formatter being used. The formatter's constructor lets the
|
||||
caller specify the maximum width the output should have.
|
||||
|
||||
A Plugin Base Class
|
||||
===================
|
||||
|
||||
Step 1 above is to define an abstract base class for the API that
|
||||
needs to be implemented by each plugin.
|
||||
|
||||
.. literalinclude:: ../../../stevedore/example/base.py
|
||||
:language: python
|
||||
:linenos:
|
||||
:prepend: # stevedore/example/base.py
|
||||
|
||||
The constructor is a concrete method because subclasses do not need to
|
||||
override it, but the :func:`format` method does not do anything useful
|
||||
because there is no "default" implementation available.
|
||||
|
||||
Concrete Plugins
|
||||
================
|
||||
|
||||
The next step is to create a couple of plugin classes with concrete
|
||||
implementations of :func:`format`. A simple example formatter produces
|
||||
output with each variable name and value on a single line.
|
||||
|
||||
.. literalinclude:: ../../../stevedore/example/simple.py
|
||||
:language: python
|
||||
:linenos:
|
||||
:prepend: # stevedore/example/simple.py
|
||||
|
||||
An alternate implementation produces a reStructuredText `field list`_.
|
||||
|
||||
.. literalinclude:: ../../../stevedore/example/fields.py
|
||||
:language: python
|
||||
:linenos:
|
||||
:prepend: # stevedore/example/fields.py
|
||||
|
||||
There are plenty of other formatting options, but these two examples
|
||||
will give us enough to work with to demonstrate registering and using
|
||||
pluins.
|
||||
|
||||
Registering the Plugins
|
||||
=======================
|
||||
|
||||
To use setuptools entry points, you must package your application or
|
||||
library using setuptools. The build and packaging process generates
|
||||
metadata which is available after installation to find the plugins
|
||||
provided by each python distribution.
|
||||
|
||||
The entry points must be declared as belonging to a specific
|
||||
namespace, so we need to pick one before going any further. These
|
||||
plugins are formatters from the stevedore examples, so I will use the
|
||||
namespace "stevedore.example.formatter". Now it is possible to provide
|
||||
all of the necessary information in the packaging instructions:
|
||||
|
||||
.. literalinclude:: ../../../stevedore/example/setup.py
|
||||
:language: python
|
||||
:linenos:
|
||||
:emphasize-lines: 38-44
|
||||
:prepend: # stevedore/example/setup.py
|
||||
|
||||
The important lines are 38-44. The ``entry_points`` argument to
|
||||
:func:`setup` is a dictionary mapping the namespace for the plugins to
|
||||
a list of their definitions. Each item in the list should be a string
|
||||
with ``name = module:importable`` where *name* is the user-visible
|
||||
name for the plugin, *module* is the Python import reference for the
|
||||
module, and *importable* is the name of something that can be imported
|
||||
from inside the module.
|
||||
|
||||
.. literalinclude:: ../../../stevedore/example/setup.py
|
||||
:language: python
|
||||
:lines: 37-43
|
||||
|
||||
In this case, there are three plugins registered. The "simple" and
|
||||
"field" plugins defined above, and a "plain" plugin, which is just an
|
||||
alias for the simple plugin.
|
||||
|
||||
setuptools Metadata
|
||||
===================
|
||||
|
||||
During the build, setuptools copies entry point definitions to a file
|
||||
in the ".egg-info" directory for the package. For example, the file
|
||||
for stevedore is located in ``stevedore.egg-info/entry_points.txt``:
|
||||
|
||||
::
|
||||
|
||||
[stevedore.example.formatter]
|
||||
simple = stevedore.example.simple:Simple
|
||||
field = stevedore.example.fields:FieldList
|
||||
plain = stevedore.example.simple:Simple
|
||||
|
||||
[stevedore.test.extension]
|
||||
t2 = stevedore.tests.test_extension:FauxExtension
|
||||
t1 = stevedore.tests.test_extension:FauxExtension
|
||||
|
||||
:mod:`pkg_resources` uses the ``entry_points.txt`` file from all of
|
||||
the installed packages on the import path to find plugins. You should
|
||||
not modify these files, except by changing the list of entry points in
|
||||
``setup.py``.
|
||||
|
||||
.. _abc module: http://docs.python.org/2/library/abc.html
|
||||
.. _field list: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#field-lists
|
@ -1,68 +0,0 @@
|
||||
=====================================
|
||||
Guidelines for Implementing Plugins
|
||||
=====================================
|
||||
|
||||
Stevedore uses setuptools entry points to define and load plugins. An
|
||||
entry point is standard way to refer to a named object defined inside
|
||||
a Python module or package. The name can be a reference to any class,
|
||||
function, or instance, as long as it is created when the containing
|
||||
module is imported (i.e., it needs to be a module-level global).
|
||||
|
||||
Names and Namespaces
|
||||
====================
|
||||
|
||||
Entry points are registered using a *name* in a *namespace*.
|
||||
|
||||
Entry point names are usually considered user-visible. For example,
|
||||
they frequently appear in configuration files where a driver is being
|
||||
enabled. Because they are public, names are typically as short as
|
||||
possible while remaining descriptive. For example, database driver
|
||||
plugin names might be "mysql", "postgresql", "sqlite", etc.
|
||||
|
||||
Namespaces, on the other hand, are an implementation detail, and while
|
||||
they are known to developers they are not usually exposed to users.
|
||||
The namespace naming syntax looks a lot like Python's package syntax
|
||||
(``a.b.c``) but *namespaces do not correspond to Python packages*. It
|
||||
can be convenient to use a package name as a namespace, but it's not
|
||||
required at all. The main feature of entry points is that they can be
|
||||
discovered *across* packages. That means that a plugin can be
|
||||
developed and installed completely separately from the application
|
||||
that uses it, as long as they agree on the namespace and API.
|
||||
|
||||
Each namespace is owned by the code that consumes the plugins and is
|
||||
used to search for entry points. The entry point names are typically
|
||||
owned by the plugin, but they can also be defined by the consuming
|
||||
code for named hooks (see :class:`~stevedore.hook.HookManager`). The
|
||||
names of entry points must be unique within a given distribution, but
|
||||
are not necessarily unique in a namespace (again, for hook patterns).
|
||||
|
||||
Keeping it Simple
|
||||
=================
|
||||
|
||||
After a lot of trial and error, the easiest way I have found to define
|
||||
an API is to follow these steps:
|
||||
|
||||
1. Define a unique namespace for each API by combining the name of the
|
||||
application (or library) and a name of the API. Keep it
|
||||
shallow. For example, "cliff.formatters" or
|
||||
"ceilometer.pollsters.compute".
|
||||
2. Use the `abc module`_ to create a base abstract class to define the
|
||||
behaviors required of plugins of the API.
|
||||
3. Create plugins by subclassing the base class and implementing the
|
||||
required methods.
|
||||
|
||||
Developers don't have to subclass from the base class, but it provides
|
||||
a convenient way to document the API, and using an abstract base class
|
||||
keeps you honest.
|
||||
|
||||
.. seealso::
|
||||
|
||||
* `abc module`_
|
||||
* `Using setuptools entry points`_
|
||||
* `Package Discovery and Resource Access using pkg_resources`_
|
||||
* `Using Entry Points to Write Plugins | Pylons`_
|
||||
|
||||
.. _Using setuptools entry points: http://reinout.vanrees.org/weblog/2010/01/06/zest-releaser-entry-points.html
|
||||
.. _Package Discovery and Resource Access using pkg_resources: http://pythonhosted.org/distribute/pkg_resources.html
|
||||
.. _Using Entry Points to Write Plugins | Pylons: http://docs.pylonsproject.org/projects/pylons-webframework/en/latest/advanced_pylons/entry_points_and_plugins.html
|
||||
.. _abc module: http://docs.python.org/2/library/abc.html
|
@ -9,12 +9,19 @@ application.
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
implementation
|
||||
registration
|
||||
naming
|
||||
creating_plugins
|
||||
loading
|
||||
calling
|
||||
testing
|
||||
|
||||
.. seealso::
|
||||
.. seealso::
|
||||
|
||||
:doc:`/essays/pycon2013`
|
||||
* :doc:`/essays/pycon2013`
|
||||
* `Using setuptools entry points`_
|
||||
* `Package Discovery and Resource Access using pkg_resources`_
|
||||
* `Using Entry Points to Write Plugins | Pylons`_
|
||||
|
||||
.. _Using setuptools entry points: http://reinout.vanrees.org/weblog/2010/01/06/zest-releaser-entry-points.html
|
||||
.. _Package Discovery and Resource Access using pkg_resources: http://pythonhosted.org/distribute/pkg_resources.html
|
||||
.. _Using Entry Points to Write Plugins | Pylons: http://docs.pylonsproject.org/projects/pylons-webframework/en/latest/advanced_pylons/entry_points_and_plugins.html
|
||||
|
38
docs/source/tutorial/naming.rst
Normal file
38
docs/source/tutorial/naming.rst
Normal file
@ -0,0 +1,38 @@
|
||||
===============================
|
||||
Guidelines for Naming Plugins
|
||||
===============================
|
||||
|
||||
Stevedore uses setuptools entry points to define and load plugins. An
|
||||
entry point is standard way to refer to a named object defined inside
|
||||
a Python module or package. The name can be a reference to any class,
|
||||
function, or instance, as long as it is created when the containing
|
||||
module is imported (i.e., it needs to be a module-level global).
|
||||
|
||||
Names and Namespaces
|
||||
====================
|
||||
|
||||
Entry points are registered using a *name* in a *namespace*.
|
||||
|
||||
Entry point names are usually considered user-visible. For example,
|
||||
they frequently appear in configuration files where a driver is being
|
||||
enabled. Because they are public, names are typically as short as
|
||||
possible while remaining descriptive. For example, database driver
|
||||
plugin names might be "mysql", "postgresql", "sqlite", etc.
|
||||
|
||||
Namespaces, on the other hand, are an implementation detail, and while
|
||||
they are known to developers they are not usually exposed to users.
|
||||
The namespace naming syntax looks a lot like Python's package syntax
|
||||
(``a.b.c``) but *namespaces do not correspond to Python
|
||||
packages*. Using a Python package name for an entry point namespace is
|
||||
an easy way to ensure a unique name, but it's not required at all.
|
||||
The main feature of entry points is that they can be discovered
|
||||
*across* packages. That means that a plugin can be developed and
|
||||
installed completely separately from the application that uses it, as
|
||||
long as they agree on the namespace and API.
|
||||
|
||||
Each namespace is owned by the code that consumes the plugins and is
|
||||
used to search for entry points. The entry point names are typically
|
||||
owned by the plugin, but they can also be defined by the consuming
|
||||
code for named hooks (see :class:`~stevedore.hook.HookManager`). The
|
||||
names of entry points must be unique within a given distribution, but
|
||||
are not necessarily unique in a namespace.
|
@ -1,5 +0,0 @@
|
||||
======================
|
||||
Registering a Plugin
|
||||
======================
|
||||
|
||||
.. describe basics of defining plugins with setuptools and entry points
|
5
setup.py
5
setup.py
@ -52,6 +52,11 @@ setup(
|
||||
include_package_data=True,
|
||||
|
||||
entry_points={
|
||||
'stevedore.example.formatter': [
|
||||
'simple = stevedore.example.simple:Simple',
|
||||
'field = stevedore.example.fields:FieldList',
|
||||
'plain = stevedore.example.simple:Simple',
|
||||
],
|
||||
'stevedore.test.extension': [
|
||||
't1 = stevedore.tests.test_extension:FauxExtension',
|
||||
't2 = stevedore.tests.test_extension:FauxExtension',
|
||||
|
0
stevedore/example/__init__.py
Normal file
0
stevedore/example/__init__.py
Normal file
21
stevedore/example/base.py
Normal file
21
stevedore/example/base.py
Normal file
@ -0,0 +1,21 @@
|
||||
import abc
|
||||
|
||||
|
||||
class FormatterBase(object):
|
||||
"""Base class for example plugin used in the tutoral.
|
||||
"""
|
||||
|
||||
__metaclass__ = abc.ABCMeta
|
||||
|
||||
def __init__(self, max_width=60):
|
||||
self.max_width = max_width
|
||||
|
||||
@abc.abstractmethod
|
||||
def format(self, data):
|
||||
"""Format the data and return unicode text.
|
||||
|
||||
:param data: A dictionary with string keys and simple types as
|
||||
values.
|
||||
:type data: dict(str:?)
|
||||
:returns: Iterable producing the formatted text.
|
||||
"""
|
36
stevedore/example/fields.py
Normal file
36
stevedore/example/fields.py
Normal file
@ -0,0 +1,36 @@
|
||||
import textwrap
|
||||
|
||||
from stevedore.example import base
|
||||
|
||||
|
||||
class FieldList(base.FormatterBase):
|
||||
"""Format values as a reStructuredText field list.
|
||||
|
||||
For example::
|
||||
|
||||
: name1 : value
|
||||
: name2 : value
|
||||
: name3 : a long value
|
||||
will be wrapped with
|
||||
a hanging indent
|
||||
"""
|
||||
|
||||
def format(self, data):
|
||||
"""Format the data and return unicode text.
|
||||
|
||||
:param data: A dictionary with string keys and simple types as
|
||||
values.
|
||||
:type data: dict(str:?)
|
||||
"""
|
||||
for name, value in sorted(data.items()):
|
||||
full_text = ': {name} : {value}'.format(
|
||||
name=name,
|
||||
value=value,
|
||||
)
|
||||
wrapped_text = textwrap.fill(
|
||||
full_text,
|
||||
initial_indent='',
|
||||
subsequent_indent=' ',
|
||||
width=self.max_width,
|
||||
)
|
||||
yield wrapped_text + '\n'
|
46
stevedore/example/setup.py
Normal file
46
stevedore/example/setup.py
Normal file
@ -0,0 +1,46 @@
|
||||
from setuptools import setup, find_packages
|
||||
|
||||
setup(
|
||||
name='stevedore-examples',
|
||||
version='1.0',
|
||||
|
||||
description='Demonstration package for stevedore',
|
||||
|
||||
author='Doug Hellmann',
|
||||
author_email='doug.hellmann@dreamhost.com',
|
||||
|
||||
url='https://github.com/dreamhost/stevedore',
|
||||
download_url='https://github.com/dreamhost/stevedore/tarball/master',
|
||||
|
||||
classifiers=['Development Status :: 3 - Alpha',
|
||||
'License :: OSI Approved :: Apache Software License',
|
||||
'Programming Language :: Python',
|
||||
'Programming Language :: Python :: 2',
|
||||
'Programming Language :: Python :: 2.7',
|
||||
'Programming Language :: Python :: 3',
|
||||
'Programming Language :: Python :: 3.2',
|
||||
'Programming Language :: Python :: 3.3',
|
||||
'Intended Audience :: Developers',
|
||||
'Environment :: Console',
|
||||
],
|
||||
|
||||
platforms=['Any'],
|
||||
|
||||
scripts=[],
|
||||
|
||||
provides=['stevedore.examples',
|
||||
],
|
||||
|
||||
packages=find_packages(),
|
||||
include_package_data=True,
|
||||
|
||||
entry_points={
|
||||
'stevedore.example.formatter': [
|
||||
'simple = stevedore.example.simple:Simple',
|
||||
'field = stevedore.example.fields:FieldList',
|
||||
'plain = stevedore.example.simple:Simple',
|
||||
],
|
||||
},
|
||||
|
||||
zip_safe=False,
|
||||
)
|
20
stevedore/example/simple.py
Normal file
20
stevedore/example/simple.py
Normal file
@ -0,0 +1,20 @@
|
||||
from stevedore.example import base
|
||||
|
||||
|
||||
class Simple(base.FormatterBase):
|
||||
"""A very basic formatter.
|
||||
"""
|
||||
|
||||
def format(self, data):
|
||||
"""Format the data and return unicode text.
|
||||
|
||||
:param data: A dictionary with string keys and simple types as
|
||||
values.
|
||||
:type data: dict(str:?)
|
||||
"""
|
||||
for name, value in sorted(data.items()):
|
||||
line = '{name} = {value}\n'.format(
|
||||
name=name,
|
||||
value=value,
|
||||
)
|
||||
yield line
|
27
stevedore/tests/test_example_fields.py
Normal file
27
stevedore/tests/test_example_fields.py
Normal file
@ -0,0 +1,27 @@
|
||||
"""Tests for stevedore.exmaple.fields
|
||||
"""
|
||||
|
||||
from stevedore.example import fields
|
||||
|
||||
|
||||
def test_simple_items():
|
||||
f = fields.FieldList(100)
|
||||
text = ''.join(f.format({'a': 'A', 'b': 'B'}))
|
||||
expected = '\n'.join([
|
||||
': a : A',
|
||||
': b : B',
|
||||
'',
|
||||
])
|
||||
assert text == expected
|
||||
|
||||
|
||||
def test_long_item():
|
||||
f = fields.FieldList(25)
|
||||
text = ''.join(f.format({'name': 'a value longer than the allowed width'}))
|
||||
expected = '\n'.join([
|
||||
': name : a value longer',
|
||||
' than the allowed',
|
||||
' width',
|
||||
'',
|
||||
])
|
||||
assert text == expected
|
15
stevedore/tests/test_example_simple.py
Normal file
15
stevedore/tests/test_example_simple.py
Normal file
@ -0,0 +1,15 @@
|
||||
"""Tests for stevedore.exmaple.simple
|
||||
"""
|
||||
|
||||
from stevedore.example import simple
|
||||
|
||||
|
||||
def test_simple_items():
|
||||
f = simple.Simple(100)
|
||||
text = ''.join(f.format({'a': 'A', 'b': 'B'}))
|
||||
expected = '\n'.join([
|
||||
'a = A',
|
||||
'b = B',
|
||||
'',
|
||||
])
|
||||
assert text == expected
|
Loading…
Reference in New Issue
Block a user