Pytest is fast becoming the de facto standard for Python unit testing. One of its most distinctive features is that it allows (indeed, encourages) you to use plain old Python assert rather than having to use library-specific assertSomeLongChainOfConditions methods.
For example, instead of writing:
def test_generate_list(self): results = generate_list() self.assertListContains(results, 'foo')
You can write:
def test_generate_list(self): results = generate_list() assert 'foo' in results
Not only is there less to remember for the test writer, it’s easier to understand for the test reader.
Why didn’t the Python unittest library do this from the start? The obvious answer is that most unit test libraries take their inspiration from JUnit, and in Java an assert statement triggers an error that can’t be caught. But the same isn’t true in Python, where you can catch an AssertionError just the same as you can catch any other exception.
In fact, in Python’s unittest the assertion methods throw an AssertionError when a condition fails, so type-wise you handle the same exception from:
self.assertEquals(1, 2)
as from:
assert 1 == 2
Why did the unittest authors bother with all those extra methods? Surely it’s not just copying Java?
Thinking a little more, it becomes clear that assert doesn’t actually give us all we need. Suppose we have a test assertion that’s failing:
assert the_number_of_the_counting == 3
When this fails, all we get is an AssertionError(). At this point you want your unit test library to give some useful diagnostics, such as the true value of the_number_of_the_counting, but it doesn’t have any data to go on. There isn’t even any message:
try: assert the_number_of_the_counting == 3 except AssertionError as e: print("Message: " + e.message)
This just prints:
Message:
We seem to be out of options here. We can’t change what the assert statement does, because it’s built into the language. We can’t create our own replacement with a function because when you pass an expression to a function the function only gets to see the resultant value, it can’t see the whole expression. For example, if we have:
my_assert(1 == 2)
then the my_assert function just receives a False value, and can’t see the two numbers that are being compared. No matter what happens inside the function, the information is already lost.
How do other languages cope with this? Most of them just rely on explicit assertion methods like assertEquals, but C++ has an approach that’s worth thinking about.
In C++ (derived from the C language it grew out of) you can declare a macro, which defines an operation to be applied to the text of the program before any compilation happens. There’s an example of this in the Boost library. You write something like:
BOOST_TEST(a + b < 10);
And before the compiler compiles the code it’s processed by a rule that converts it into something like:
real_assert("a + b < 10", a + b < 10);
This is starting to get us somewhere, because the function now at least has enough information to log a decent error message. We can implement this function with something like:
void real_assert(string expression, bool value) { if (!value) { report_error("Expression " + expression + " wasn't satisfied"); } }
As an aside, this sort of text macro substitution is pretty uncommon outside of C and C++. It can do some useful things, but it’s pretty easy to make things unmaintainable and it’s often hard to achieve what you want.
In this particular case, the macro approach still imposes serious limits on what we can do. A really great test library, when faced with the failure of the expression a + b < 10 would print out not just the whole expression but the values of a and b, so that the developer can see which is wrong. This kind of thing just isn't possible with the compile-time string substitution.
Pytest has a really interesting solution to this problem, and I think it illustrates a key reason why Python is such an interesting language: user-defined libraries can do things that would need compiler modifications in other languages. I'll dig into this in the next part.