Getting Started with :mod:`dip.plugins`
=======================================

A well designed dip application will be implemented as a set of components.
The more those components are decoupled from each other, the greater chance
there is of being able to reuse them in a different context.  Defining
components in terms of :term:`interfaces<interface>` is a major step in being
able to decouple them.  The use of
:ref:`default handlers<ref-computed-default-values>` makes it easy to provide
a default implementation of an interface while still allowing it to be
overridden with an alternative implementation.  However, that still requires
that one component is explictly aware of another component to use as a
default implementation.

The :mod:`dip.plugins` module provides mechanisms for making connections
between components while ensuring that the components themselves are completely
decoupled from each other.  A :term:`plugin` is an implementation of the
:class:`~dip.plugins.IPlugin` interface that makes those connections by
publishing objects, either as contributions to
:term:`extension points<extension point>` or as :term:`services<service>`.
Plugins are managed by a :term:`plugin manager`.  A plugin plays no part in an
application until it is enabled.

An extension point is a list of published objects, usually of a particular type
or implementing a particular interface.  Each extension point has a unique
string identifier.  All extension points defined by dip have identifiers
beginning with ``dip.``.  Plugins make contributions to extension points when
they are enabled.  A plugin can bind an extension point to an attribute of an
object.  A bound attribute will normally be a :class:`~dip.model.List` but can
be anything that has an ``append()`` method.  A contribution is a list of
objects.  When a contribution is made each object is appended to each attribute
that is bound to the extension point.  When an attribute is bound then any
previous contributions are appended to the attribute.

A service is an object that implements a particular interface.  Several plugins
may provide services that implement the same interface.  When a plugin requests
a service the plugin manager will choose which service is actually used.  The
plugin does not care about the particular service, its only concern is that it
has an object that implements the interface.

When a plugin is enabled it will create any services it provides, possibly
requesting other services to configure them with.  It will also make any 
contributions to extension points and possibly bind extension points to objects
it creates.


Standard Extension Points
-------------------------

dip refers to a number of standard extension points which plugins that
implement certain services are expected to support.

``dip.io.codecs``
    This must be supported by any plugin that offers a service implementing the
    :class:`dip.io.IIOManager` interface.  A contribution must implement the
    :class:`dip.io.ICodecsFactory` interface.

``dip.io.storage``
    This must be supported by any plugin that offers a service implementing the
    :class:`dip.io.IIOManager` interface.  A contribution must implement the
    :class:`dip.io.IStorageFactory` interface.

``dip.plugins.extension_points``
    This must be supported by a :class:`~dip.plugins.IPluginManager`
    implementation.  A contribution must implement the
    :class:`dip.plugins.IExtensionPoint` interface.

``dip.plugins.plugins``
    This must be supported by a :class:`~dip.plugins.IPluginManager`
    implementation.  A contribution must implement the
    :class:`dip.plugins.IPlugin` interface.

``dip.plugins.services``
    This must be supported by a :class:`~dip.plugins.IPluginManager`
    implementation.  A contribution must implement the
    :class:`dip.plugins.IService` interface.

``dip.shell.actions``
    This must be supported by any plugin that offers a service implementing the
    :class:`dip.shell.IShell` interface.  A contribution must implement the
    :class:`dip.ui.IAction` interface.

``dip.shell.objects``
    This must be supported by any plugin that offers a service implementing the
    :class:`dip.shell.IShellManager` interface.  A contribution must implement
    the :class:`dip.shell.IShellObjectFactory` interface.

``dip.shell.tools``
    This must be supported by any plugin that offers a service implementing the
    :class:`dip.shell.IShellManager` interface.  A contribution must implement
    the :class:`dip.shell.IToolFactory` interface.


Writing a Plugin
----------------

In this section we will describe the plugin for part of an application that
chooses a recipe from a list of available recipes based on the stock levels of
the different ingredients.  We assume that other plugins will contribute the
recipes and provide an object that implements the ``IStockLevels`` interface
with which our ``IRecipeChooser`` implementation can query the stock levels.

The following is the complete plugin for our recipe chooser service.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py

We will now walk through the code a section at a time.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :end-before: from recipes

The above line imports the required dip objects.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: dip.plugins
    :end-before: class RecipeChooserPlugin

The above lines import the required interfaces.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: from stock
    :end-before: The RecipeChooserPlugin

The above line defines the plugin object.  All plugins must implement the
:class:`~dip.plugins.IPlugin` interface.  The :class:`~dip.plugins.Plugin`
class is a convenience class for plugins that implements the
:class:`~dip.plugins.IPlugin` interface and makes contributions defined
declaratively using :class:`~dip.plugins.ContributionTo`.  It doesn't matter
what the name of the plugin class is.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: # The identifier
    :end-before: # The description

The above line defines the identifier of the plugin.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: # The description
    :end-before: # The contribution

The above line defines the description of the plugin.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: # The contribution
    :end-before: @recipe_chooser.default

The above line specifies that the value of the ``recipe_chooser`` typed
attribute will be contributed to the ``'dip.plugins.services'`` extension
point.  Contributing to this extension point is how services are registered
with the plugin manager.  Each contribution must be an implementation of the
:class:`~dip.plugins.IService` interface.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: recipe_chooser =
    :end-before: """ Create the

The above lines define the default handler for the ``recipe_chooser`` typed
attribute.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: """ Create the
    :end-before: stock_levels =

The above line imports the ``RecipeChooser`` class which is the implementation
of the ``IRecipeChooser`` interface we are going to provide.  Because this is
done in the default handler the class isn't imported (and the instance
subsequently created) until the plugin is actually enabled.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: from recipes.recipe_chooser
    :end-before: chooser =

The ``IRecipeChooser`` interface has an attribute called ``stock_levels`` which
is an instance of the ``IStockLevels`` interface.  We could rely on our
``RecipeChooser`` implementation to create a default value for ``stock_levels``
but that would mean that ``ReciperChooser`` would be coupled to the particular
``IStockLevels`` implementation, which we want to avoid.  Instead the above
line shows the plugin requesting an object that implements the ``IStockLevels``
interface.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: stock_levels =
    :end-before: self.bind_extension_point

The above line actually creates the ``RecipeChooser`` instance and gives it
the ``IStockLevels`` implementation we obtained from the call to
:meth:`~dip.plugins.Plugin.service`.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: chooser = Recipe
    :end-before: service =

The ``IRecipeChooser`` interface also has an attribute called ``recipes`` which
is a :class:`~dip.model.List` of ``IRecipe`` instances that is the list of
available recipes.  Again we want to keep our ``RecipeChooser`` implementation
decoupled from whatever object or objects provide the recipes.  We decide that
recipes will be added by other plugins making contributions to the
``'recipes.recipes'`` extension point.  The above line binds the extension
point to the ``recipes`` attribute of our ``RecipeChooser`` instance so that it
will be automatically updated when new recipes are contributed.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: self.bind_extension_point
    :end-before: return

Our ``RecipeChooser`` instance is now fully configured.  The above line uses
the default implementation of the :class:`~dip.plugins.IService` interface to
create the relationship between the service implementation and the interface
that it is published as implementing.

.. literalinclude:: /examples/plugins/recipes/plugins/recipe_chooser_plugin.py
    :start-after: service =
    :end-before: EOF

Finally, the above line returns the list of contributions to the
``'dip.plugins.services'`` extension point.


Registering a Plugin
--------------------

In a typical plugin based application most of the initialisation of the
application involves the same steps:

- create a plugin manager
- create and register each plugin
- enable all the registered plugins.

The following performs these steps for the ``RecipeChooserPlugin`` described in
the previous section.

.. literalinclude:: /examples/plugins/main.py

We will now walk through the code a section at a time.

.. literalinclude:: /examples/plugins/main.py
    :end-before: from recipes

The above line imports dip's default plugin manager class.  Of course you can
provide a sub-class of this if required, or any object that implements the
:class:`~dip.plugins.IPluginManager` interface.

.. literalinclude:: /examples/plugins/main.py
    :start-after: from dip.
    :end-before: # Create the plugin manager

The above line imports the plugin class.  It assumes that the ``recipes``
package has been structured as described in the next section.

.. literalinclude:: /examples/plugins/main.py
    :start-after: # Create the plugin manager
    :end-before: # Create the plugin.

The above line creates a plugin manager instance.

.. literalinclude:: /examples/plugins/main.py
    :start-after: # Create the plugin.
    :end-before: # Register

The above line creates an instance of the plugin.

.. literalinclude:: /examples/plugins/main.py
    :start-after: # Register
    :end-before: # Enable

The above line registers the plugin by contributing it to the
``'dip.plugins.plugins'`` extension point.

.. literalinclude:: /examples/plugins/main.py
    :start-after: # Enable
    :end-before: EOF

Finally, the above line enables all the plugins that have been registered but
have not already been enabled.


Structuring a Package to Support Plugins
----------------------------------------

When implementing a package it is important to think about how it should be
structured so that:

- an alternative service can be provided for an interface without creating
  an instance of the default service
- an alternative service can be provided for an interface by sub-classing the
  default service
- the default service can be configured
- plugins can be sub-classed.

We will now describe a package structure that meets these requirements and so
maximises the ability to reuse the contents of the package.

We decide that our ``recipes`` package will contain two plugins.  One will
contain our recipe chooser service.  The other will contain some example
recipes.  We could decide to put each plugin in a different package, but they
are logically related and, with the structure we will adopt, will be properly
decoupled from each other.  (However we would need to use separate packages if
we wanted to deploy them separately.)

    +---------+---------------------+--------------------------+
    | recipes | __init__.py         |                          |
    +---------+---------------------+--------------------------+
    |         | i_recipe.py         |                          |
    +---------+---------------------+--------------------------+
    |         | i_recipe_chooser.py |                          |
    +---------+---------------------+--------------------------+
    |         | recipe_chooser      | __init__.py              |
    +---------+---------------------+--------------------------+
    |         |                     | recipe_chooser.py        |
    +---------+---------------------+--------------------------+
    |         | recipes             | __init__.py              |
    +---------+---------------------+--------------------------+
    |         |                     | cheese_sandwich.py       |
    +---------+---------------------+--------------------------+
    |         |                     | frozen_pizza.py          |
    +---------+---------------------+--------------------------+
    |         | plugins             | __init__.py              |
    +---------+---------------------+--------------------------+
    |         |                     | recipe_chooser_plugin.py |
    +---------+---------------------+--------------------------+
    |         |                     | recipes_plugin.py        |
    +---------+---------------------+--------------------------+

We will now go through each of the above explaining what they contain and why.

:download:`recipes.__init__.py</examples/plugins/recipes/__init__.py>` imports
all of the objects that make up the ``recipe`` module's public API.
Specifically these are the ``IRecipe`` and ``IRecipeChooser`` interfaces.  Note
that importing this module does not cause any plugins to be loaded or object
implementations to be imported.

:download:`recipes.i_recipe.py</examples/plugins/recipes/i_recipe.py>` contains
the definition of the ``IRecipe`` interface.

:download:`recipes.i_recipe_chooser.py</examples/plugins/recipes/i_recipe_chooser.py>`
contains the definition of the ``IRecipeChooser`` interface.

:download:`recipes.recipe_chooser.__init__.py</examples/plugins/recipes/recipe_chooser/__init__.py>`
imports all of the objects that make up the ``recipes.recipe_chooser`` module's
public API.  Specifically this is the ``RecipeChooser`` class that is the
default recipe chooser service.  Keeping it separate from its corresponding
plugin definition means that it can be sub-classed without causing an unwanted
plugin definition to be loaded.

:download:`recipes.recipe_chooser.recipe_chooser.py</examples/plugins/recipes/recipe_chooser/recipe_chooser.py>`
contains the ``RecipeChooser`` class that is the default recipe chooser
service.

:download:`recipes.recipes.__init__.py</examples/plugins/recipes/recipes/__init__.py>`
imports all of the objects that make up the ``recipes.recipes`` module's
public API.

:download:`recipes.recipes.cheese_sandwich.py</examples/plugins/recipes/recipes/cheese_sandwich.py>`
contains the ``CheeseSandwich`` class.

:download:`recipes.recipes.frozen_pizza.py</examples/plugins/recipes/recipes/frozen_pizza.py>`
contains the ``FrozenPizza`` class.

:download:`recipes.plugins.__init__.py</examples/plugins/recipes/plugins/__init__.py>`
imports all of the objects that make up the ``recipes.plugins`` module's
public API.  Specifically these are the ``ReciperChooserPlugin`` and
``RecipesPlugin`` classes.

:download:`recipes.plugins.recipe_chooser_plugin.py</examples/plugins/recipes/plugins/recipe_chooser_plugin.py>`
contains the ``RecipeChooserPlugin`` plugin that creates the default recipe
chooser service in its default configuration.

:download:`recipes.plugins.recipes_plugin.py</examples/plugins/recipes/plugins/recipes_plugin.py>`
contains the ``RecipesPlugin`` plugin that contributes some example recipes.

The :mod:`dip.io` and :mod:`dip.shell` modules are examples within dip that
adopt this package structure.


Using Plugins with :mod:`dip.shell`
-----------------------------------

In the :mod:`dip.shell` tutorial we created and configured the shell explicitly
but suggested that in a more complex application a plugin based implementation
would be preferable.

For example, say you are implementing an application to be deployed throughout
your business.  As well as the generic functionality the head office and the
sales office will require additional functionality specific to their needs.  To
make it more complicated, the head office and sales office functionality is
being developed by two separate teams neither of which is developing the
generic functionality.

In order to simplify the development you want to decouple the different parts
of the application as much as possible.  Implementing the generic, head office
and sales office functionality as different plugins achieves this.  It allows
the different teams to develop, deploy, enhance and bug-fix their code
independently while giving the users a single application with which to work.

The following code is a more complete example of registering the different
plugins that make up the application.  It is this code that user would actually
run::

    import sys

    # Every PyQt4 GUI application needs a QApplication instance.
    from PyQt4.QtGui import QApplication

    app = QApplication(sys.argv, applicationName="Acme Inc. Management System")

    # Create a plugin manager.
    from dip.plugins import PluginManager

    plugin_manager = PluginManager()

    # Add the infrastructure plugins we need.
    from dip.io.plugins import FilesystemPlugin, IOManagerPlugin
    from dip.shell.plugins import QMainWindowShellPlugin, ShellManagerPlugin
    from dip.ui.plugins import QtToolkitPlugin

    plugin_manager.contribute('dip.plugins.plugins',
            QtToolkitPlugin(plugin_manager=plugin_manager))

    plugin_manager.contribute('dip.plugins.plugins',
            IOManagerPlugin(plugin_manager=plugin_manager))

    plugin_manager.contribute('dip.plugins.plugins',
            FilesystemPlugin(plugin_manager=plugin_manager))

    plugin_manager.contribute('dip.plugins.plugins',
            ShellManagerPlugin(plugin_manager=plugin_manager))

    plugin_manager.contribute('dip.plugins.plugins',
            QMainWindowShellPlugin(plugin_manager=plugin_manager))

    # Add the plugin implementing the generic application functionality.
    from acme.plugins.generic import GenericPlugin

    plugin_manager.contribute('dip.plugins.plugins',
            GenericPlugin(plugin_manager=plugin_manager))

    # Try to add the plugin implementing the head office functionality.
    try:
        from acme.plugins.head_office import HeadOfficePlugin

        plugin_manager.contribute('dip.plugins.plugins',
                HeadOfficePlugin(plugin_manager=plugin_manager))
    except ImportError:
        pass

    # Try to add the plugin implementing the sales office functionality.
    try:
        from acme.plugins.sales_office import SalesOfficePlugin

        plugin_manager.contribute('dip.plugins.plugins',
                SalesOfficePlugin(plugin_manager=plugin_manager))
    except ImportError:
        pass

    # Enable the plugins.
    plugin_manager.enable_plugins()

    # Get the shell.  This will be a QMainWindowShell because that is the only
    # implementation of IShell provided by any of the plugins we have enabled.
    from dip.shell import IShell

    shell = plugin_manager.service(IShell)

    # Display the shell and start the application.
    shell.widget.show()

    app.exec_()

While the above code successfully decouples the main areas of functionality it
still leaves a number of problems.

For example, we have hard coded the existence of the head office and sales
office plugins.  What happens if we are then told that the factory needs it's
own specific functionality that will be deployed in its own plugin?  We have to
change the above code - a simple change, but a change nevertheless.  A future
version of the :mod:`dip.plugins` module will support the automatic discovery
of plugins in order to deal with this.

The other major problem with the above code is that, while deployment is
relatively easy (because there are only two optional plugins), the plugins are
almost certain to contain too many separate business objects and tools.  This
makes it difficult to re-use those objects and tools individually.  Take, for
example, a sales forecast.  The sales office is likely to have a tool to allow
a forecast to be created, and the head office may have their own tool that
allows forecasts to be consolidated and analysed.  Clearly the sales forecast
business object would be better deployed as a separate plugin.  Following on
from this it is also a good idea to keep objects and the tools that operate on
those objects in separate plugins.

In summary, significant thought should be put into deciding how an
application's functionality is split between different plugins.  It is
guaranteed that somebody will add a new requirement at the most inconvenient
time that will challenge your current assumptions.  A successful design will be
able to accomodate such changes without requiring changes to core code.

Having established some best practice we will now, for the sake of this
example, ignore it by saying that the sales office plugin will contain an
implementation of a sales forecast object and a form-based tool for creating
and modifying a sales forecast.  The plugin definition is as follows::

    from dip.plugins import ContributionTo, Plugin

    class SalesOfficePlugin(Plugin):
        """ The SalesOfficePlugin is a plugin that contributes a sales forecast
        object and a tool to create and maintain it to a shell.
        """

        # The identifier of the plugin.
        id = 'acme.plugins.sales_office'

        # The description of the plugin.
        description = "The sales office plugin"

        # We will be contributing a shell object factory.
        objects = ContributionTo('dip.shell.objects')

        # We will be contributing a tool factory.
        tools = ContributionTo('dip.shell.tools')

        @objects.default
        def objects(self):
            """ Return the list of shell object factories to be added to the
            shell.
            """

            from acme.sales.sales_forecast_factory import SalesForecastFactory

            return [SalesForecastFactory()]

        @tools.default
        def tools(self):
            """ Return the list of tool factories to be added to the shell. """

            from dip.shell import ModelToolFactory

            from acme.sales import ISalesForecast

            return [
                    ModelToolFactory(name="Sales Forecast Editor",
                            model_type=ISalesForecast)]
