Python setuptools entry points

This is a quick tutorial on how to use Python setup tools entry points to elegantly distribute command line scripts as part of your program, and to enable your code to discover and use plugins in a principled fashion.

Installing scripts

Say your program supplies a python script dothis.py. If you have been using distutils, in your setup.py script you probably have something like

setup(...,
  scripts=['dothis.py']
     )

When I first used this it was like magic. This line would cause setup to place a executable wrapper script in /usr/local/bin that would point to your dothis.py script.

setuptools has a slightly different, but ultimately better, way to handle distributing commandline scripts. You may have to refactor your code a little bit to get this to work, but the refactoring improves the layout of your code. If your original script (like some of mine) had the pattern

# Your module code

if __name__ == "__main__":
  # Parse command line args 
  # Do a bunch of stuff

You need to refactor the stuff in main into a function – let’s call it cli():

# Your module code

def cli():  # Entry point for scripts
  # Parse command line args 
  # Do a bunch of stuff

if __name__ == "__main__":
  cli()

(In general this is proper practice, but folks have been known to ignore it)

In your setup script you should now discard the scripts line and instead use:

    ...
    entry_points={
           ....
      # Command line scripts
      'console_scripts': ['runme = dothis:cli']
    },
    ...

This has several advantages, mainly relating to playing nicely with both POSIX and Windows systems. There are also options for distributing Python GUI programs!

Plugins

Setuptools gives us a really nice way to implement a plugin system without us having to write any plugin manager that keeps track of where plugins are stored and how to import them (Relative import? Absolute path? Dotted path?). It also ensures that we and the plugin writers don’t have to worry about setting up some common space where plugin code has to be installed into to be discoverable by the main app.

What you do is simply settle on a context name for your plugin system. Say my.cool.plugins. It is important to choose a unique name, since the whole Python install will know about this name and we don’t want collisions. Having the name of your package somewhere is a good bet.

In your application code, you can use something like this to find and load a plugin module:

import pkg_resources
def _load_plugin(name, plugin_entry_point):
  for v in pkg_resources.iter_entry_points(plugin_entry_point, name):
    return v.load()
  raise ImportError('No plugin called "{:s}" has been registered.'.format(name))

Where ‘my.cool.plugin’ is the value you should pass for plugin_entry_point. This function will return the result of load() which is a module the same as if we used import by our own hand.

You can also discover all the available plugin modules:

def discover_all_plugins():
  return sorted([(v.name, v.module_name) for v in pkg_resources.iter_entry_points('my.cool.plugins')],
                cmp=lambda x, y: cmp(x[0], y[0]))

But how can we make the magic happen? How can we make a plugin discoverable this way? The magic lies in setup.py for the package supplying the package. Say the plugin code lies in a file called plugin_file.py under the directory my/plugin/code:

    ...
    entry_points={
           ....
      # Register the built in plugins
      'my.cool.plugins': ['fancy_name = my.plugin.code.plugin_file']
    },
    ...

Once you run python setup.py install on this, your application will be able to see the plugin.

Many thanks to Björn Pollex who encouraged me to look into setup tools entry points as a way to clean up how I distribute scripts and implement plugins.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s