It's Always Sunny in Python

Friday February 27, 2009
Jonathan Lange suggests that you have three options when you have some code that needs testing.
  1. Give up
  2. Work hard to write the damn test
  3. Make your code testable.
In fact he starts with just the first two options and then reveals the third, but I suggest that there is an option number four, available only in Python (and other, similar dynamic languages):
  1. Cheat.
I agree with Jonathan that you should generally make your code more flexible, and thereby testable.  Any code which is friendly to being invoked as a library or as an independent unit of functionality can be tested, so you should endeavor to never write code that is unfriendly to being executed as just a normal method call.  This is especially true if you are writing new code in a properly test-driven manner.  Keeping in mind, of course, that one should not modify the SUT purely for the purpose of testing.

However, if one is (as I often find myself) adding test coverage to a grotty old system, written at a time or in a place where test-driven development was not the norm, one typically wants to establish test coverage before making any changes to the code or its design.  In such a situation, one may often find oneself in the undesirable position of needing to carefully modify some implementation code so that it can be tested, hoping that none of its untested interactions with other areas of the system will be broken as a result.  For example, you might encounter some paranoid and misguided Java code like this:

    // Startup.java
    private static final void emitLogMessage(final String message) {
        System.out.println(message);
    }

    public static final void startUp() {
        // ...
        emitLogMessage("Starting up!");
        // ...
    }

In this case, it's very difficult to get in the way of any part of this system.  Nothing is parameterized, everything is global, and the compiler won't even let you call one of these methods.  You really only have Jonathan's three options here, none of which are desirable.
  1. You can give up on testing this part of the system until you've covered other parts of the system.  In many cases this is the right thing to do, but it is often the lowest-level and most critical parts of a system which have calcified into this sort of untestable rubble.
  2. You can work hard to write the damn test.  There are a number of extremely subtle nuances of the Java runtime which you can take advantage of to make a lie of the "private" and "final" keywords.  You can load the code using a custom classloader, manipulate its bytecode, or invoke private methods using reflection.  This is ultimately the "right thing" to do, but it requires the development of a daunting skill-set which you would not otherwise need.
  3. You can make the code testable, changing it before you've properly tested the code which is already in use.  Ultimately this is what you want to get to anyway, but if the code is doing something subtle that you didn't test (and none of the rest of the system is tested yet) you might be (rightly) concerned that this could break something else.
In Python, the hard work is not so hard.  To start with, Python doesn't have the misfeatures of private and final.  It also doesn't have any baroque "reflection" constructs.  All you need to understand is attribute access.  So, if you have a similar Python file:
# startup.py
import sys

def emitLogMessage(message):
    sys.stdout.write("%s\n" % (message,))

def startUp():
    # ...
    emitLogMessage("Starting up!")
    # ...
Idiomatically, the situation looks just as hopeless.  Everything is global, and nothing is parameterized.  It's hard-coded.  However, if you look at it from the right angle, you will realize that you can't really code that "hard" in python.

What emitLogMessage is doing in this case is not making a fixed reference to the global sys module: it is simply accessing the sys attribute of the startup module.  So in fact, we can easily test it:
# test_startup.py
import sys
import startup
import unittest

class FakeSys(object):
    def __init__(self, test):
        self.test = test
    @property
    def stdout(self):
        return self
    def write(self, message):
        self.test.messages.append(message)

class StartupTest(unittest.TestCase):
    def setUp(self):
        self.messages = []
        startup.sys = FakeSys(self)

    def tearDown(self):
        startup.sys = sys

    def test_startupLogMessage(self):
        startup.startUp()
        self.assertEquals(self.messages, ["Starting up!\n"])
So testing startUp is a simple matter of replacing the sys object that it's talking to: which, in Python, is rarely more than a setattr() away.

I've taken care here to use only standard Python features.  This is, after all, theoretically possible in Java, it's just a heck of a lot harder, both to use and to understand — the learning curve is a big part of the problem.  However, if you're using Twisted, and willing to spend just a brief moment to learn about one of its testing features, you can save a few lines of code and opportunities for error:
import startup
from twisted.trial import unittest

class FakeSys(object):
    def __init__(self, test):
        self.test = test
    @property
    def stdout(self):
        return self
    def write(self, message):
        self.test.messages.append(message)

class StartupTest(unittest.TestCase):
    def setUp(self):
        self.messages = []
        self.patch(startup, "sys", FakeSys(self))

    def test_startupLogMessage(self):
        startup.startUp()
        self.assertEquals(self.messages, ["Starting up!\n"])
Please keep in mind that this is still not the best way to do things.  Use the front door first.  It's much better to use a stable, documented, supported API in your tests than to depend on an accident of implementation which should be able to change.  However, it is even worse to associate the feeling of testing with the feeling of being stuck, being unable to figure out how to dig yourself out of some hole that old, bad design has dug you into.

I'm writing this mostly for people who are new to test-driven development in Python and think that unit tests need to be a huge amount of extra work.  They don't.  If you ever find yourself struggling, unable to figure out how you could possibly write a test which would exercise some tangle of poorly-designed code, just remember: it's all just objects and methods, attributes and values.  You can replace anything with anything else with a pretty trivial amount of effort.  Of course you should try to figure out how to improve your design, but you should never think that you need to stop writing tests just because you used a global variable and you can't figure out what to replace it with.