Explaining Why Interfaces Are Great

Why?

Why use interfaces? Especially with Python's new ABCs, is there really a use for them?

Some of us Zope Interface fans — names withheld to protect the guilty, although you may feel free to unmask yourselves in the comments if you like — have expressed frustration that ABCs beat out interfaces for inclusion of the standard library.  However, I recently explored various mailing lists, Interfaces literature, and blogs, and haven't found a coherent description of why one would prefer interfaces over ABCs.  It's no surprise that Zope's interface package is poorly understood, given that nobody has even explained it!  In fact, PEP 3119 specifically says:

For now, I'll leave it to proponents of Interfaces to explain why Interfaces are better.

It seems that nobody has taken up the challenge.

I remember Jim Fulton trying to explain this to me many years ago, at a PyCon in Washington DC.  I definitely didn't understand it then.  I was reluctant to replace the crappy little interfaces system in Twisted at the time with something big and complicated-looking.   Luckily the other Twisted committers prevailed upon me, and Zope Interface has saved us from maintaining that misdesigned and semi-functional mess.

During that explanation, I remember that Jim kept saying that interfaces provided a model to "reason about intent".  At the time I didn't understand why you'd want to reason about intent in code.  Wouldn't the docstrings and the implementation specify the intent clearly enough?  Now, I can see exactly what he's talking about and I use the features he was referring to all the time.  I don't know how I'd write large Python programs without them.

Caveat

This isn't a rant against ABCs.  I think ABCs are mostly pretty good, certainly an improvement over what was (or rather, wasn't) there before.  ABCs provide things that Interfaces don't, like the new @abstractmethod and @abstractproperty decorators. Plus, one of the irritating things about using zope.interface is that the metadata about standard objects in zope.interface.common is not hooked up to anything: IMapping.providedBy({}) returns False.  ABCs will provide that metadata in the standard library, making zope.interface that much more useful once it has been upgraded to understand the declarations that the collections and numbers modules provide.

So, on to the main event: what do Zope Interfaces provide which makes them so great?

Clarity

Let's say we have an idea of something called a "vehicle".  We can represent it as one of two things: a real base class (Vehicle), an ABC (AVehicle) or an Interface (IVehicle).

There are a set of operations that interfaces and base-classes share.  We can ask, "is this thing I have a vehicle"?  In the base-class case we spell that if isinstance(something, Vehicle).  In the interfaces case, we say if IVehicle.providedBy(something).  We can ask, "will instances of this type be a vehicle?".  For an interface, we say if IVehicle.implementedBy(Something), and for a base class we say issubclass(Something, Vehicle).  With the new hooks provided by the ABCs in 2.6 and 3.0, these are almost equivalent.  With zope.interface, you can subclass InterfaceClass and write your own providedBy method.  With the ABC system, you subclass type and implement __instancecheck__.

However, there are some questions you can't quite cleanly ask of the ABC system.  For one thing, what does it really mean to be a Vehicle?  If you are looking at AVehicle, you can't tell the difference between implementation details and the specification of the interface.  You can use dir() and ignore a few of the usual suspects — __doc__, __module__, __name__, _abc_negative_cache_version — but what about the quirkier bits?  Metaclasses, inherited attributes, and so on?  There's probably some way to do it, but I certainly can't figure it out quickly enough to include in this article.  In other words, types have two jobs: they might be ABCs, or they might be types, or they might be both, and it's impossible to separate those responsibilities.

With an Interface, this question is a lot easier to ask.  For a quick look, list(IVehicle) will give a complete list of all the attributes expected of a vehicle, as strings.  If you want more detail, IVehicle.namesAndDescriptions() and Method.getSignatureInfo() will oblige.

Since the interface encapsulates only what an object is supposed to be, and no functionality of its own, it's possible for frameworks to inspect them and provide much nicer error messages when objects don't match their expectations.  zope.interface.verifyClass and zope.interface.verifyObject can tell you, both for error-reporting and unit-testing purposes, whether an object looks like a proper vehicle or not, without actually trying to drive it around.

Flexibility

At the most basic level, interfaces are more flexible because they are objects.  ABCs aren't objects, at least in the message-passing smalltalk sense; they are a collection of top-level functions and some rules about how those functions apply to types.  If you want to change the answer to isinstance(), you need to register a type by using ABCMeta.register or overriding __instancecheck__ on a real subclass of type.  If you want to change the answer to providedBy, for example for a unit test, all you need is an object with a providedBy method.

Of course, you can do it "for real" with an InterfaceClass, but you don't need to.  In other words, its semantics are those of a normal method call.

Interfaces aren't completely self-contained, of course: there are top-level functions that operate on interfaces, like verifyObject.  However, there's an interface to describe what is expected:

>>> from zope.interface.interfaces import Interface, IInterface
>>> IInterface.providedBy(Interface)
True
There's also the issue of who implements what.  For example, you might have a plug-in system which requires modules to implement some functionality.  Generally speaking, modules are instances of ModuleType, so specifying that all modules implement some type with an ABC is somewhat awkward.  With an interface, however, there is a specific facility for this: you put a moduleProvides(IVehicle) declaration at the top of your module.

In zope.interface, there is a very clear conceptual break between implements and provides.  A module may provide an interface — i.e. be an object which satisfies that interface at runtime — without there being any object that implements that interface — i.e. is a type whose instances automatically provide it.  This distinction comes in handy when avoiding certain things.  This distinction exists with ABCs; either you "are a subclass of" a type or you "are an instance of" a type, but the language around it is more awkward and vague, especially since you can be a "virtual instance" or "virtual subclass" now as well.

There's also the issue of dynamic proxies.  If you have a wrapper which provides security around another object, or transparent remote access to another object, or records method calls (and so on) the wrapper really wants to say that it provides the interfaces provided by the object it is wrapping, but the wrapper type does not implement those interfaces.  In other words, different instances of your wrapper class will actually provide different interfaces.  With zope.interface you can declare this via the directlyProvides declaration.  With ABCs, this is not generally possible because ABCMeta.register will only work on a type.

Adaptation

Let's say I have an object that provides IVehicle.  I want to display it somehow — and in today's web-centric world, that probably means "I want to generate some HTML".  How do I get from here to there?  ABCs don't provide an answer to that question.  Interfaces don't do that directly either, but they do provide a mechanism which allows you to provide an answer: you can adapt from one interface to another.

I'm not going to get into the intricacies of exactly how adaptation works in zope.interface, since it isn't important to understand most of the time.  Suffice it to say you can adapt based on specific hooks that are registered, based on the type an object is, or based on what interfaces it provides.

The gist of it is that you have some thing that you don't know what it is, and you want an object that provides IHTMLRenderer.  The way you express that intent is:

renderer = IHTMLRenderer(someObject)

If there are no rules for adapting an object like the one you have passed to an IHTMLRenderer, then you will get an exception - which is all that will happen, normally.  However, this point of separation between the contract that your code expects and the concrete type that your code ends up actually talking to can be very useful.

The larger Zope application server has a rich and complex set of tools for defining which adapter is appropriate in which context, but Twisted has a very simple interface to adaptation.  You simply register an adapter, which is a 1-argument callable that takes an object that conforms to some interface or is an instance of some class, and returns an object that provides another interface.  Here's how you do it:

import attr
from zope.interface import implementer
from twisted.python.components import registerAdapter

@implementer(IHTMLRenderer)
@attr.s
class VehicleRenderer(object):
    "Render a vehicle as HTML"
    vehicle = attr.ib(validator=attr.validator.provides(IVehicle))
    def renderHTML(self):
        return "<h1>A Great Vehicle %s (%s)</h1>" % (
                   self.vehicle.make.name,
                   self.vehicle.model.name)

registerAdapter(VehicleRenderer, IVehicle, IHTMLRenderer)

Now, whenever you do IHTMLRenderer(someVehicle), you'll get a VehicleRenderer(someVehicle).

Your code for rendering now doesn't need any special-case knowledge about particular types.  It is written to an interface, and it's very easy to figure out which one; it says "IHTMLRenderer" right there.  It's also easy to find implementors of that interface; just grep for "implementer.*IHTMLRenderer" or similar.  Or, use pydoctor and look at the "known implementations" section for the interface in question.

Conclusion

In a super-dynamic language like Python, you don't need a system for explicit abstract interfaces.  No compiler is going to shoot you for calling a 'foo' method.  But, formal interface definitions serve many purposes.  They can function as documentation, as a touchstone for code which wants to clearly report programming errors ("warning:  MyWidget claims to implement IWidget, but doesn't implement a 'doWidgetStuff' method"), and a mechanism for indirection when you know what contract your code wants but you don't know what implementation will necessarily satisfy it (adaptation).

Even with a standard library mechanism for doing some of these things, Zope Interface remains a highly useful library, and if you are working on a large Python system you should consider augmenting its organization and documentation with this excellent tool.