The world of testing has no shortage of terminology, and now that you know the difference between automated and manual testing, it’s time to go a level deeper.
Think of how you might test the lights on a car. You would turn on the lights (known as the test step) and go outside the car or ask a friend to check that the lights are on (known as the test assertion).
Testing multiple components is known as integration testing.
Think of all the things that need to work correctly in order for a simple task to give the right result.
These components are like the parts to your application, all of those classes, functions, and modules you’ve written.
A major challenge with integration testing is when an integration test doesn’t give the right result. It’s very hard to diagnose the issue without being able to isolate which part of the system is failing.
If the lights didn’t turn on, then maybe the bulbs are broken. Is the battery dead? What about the alternator? Is the car’s computer failing?
If you have a fancy modern car, it will tell you when your light bulbs have gone. It does this using a form of unit test.
A unit test is a smaller test, one that checks that a single component operates in the right way. A unit test helps you to isolate what is broken in your application and fix it faster.
You have just seen two types of tests:
- An integration test checks that components in your application operate with each other.
- A unit test checks a small component in your application.
You can write both integration tests and unit tests in Python. To write a unit test for the built-in function sum(), you would check the output of sum() against a known output.
For example, here’s how you check that the sum() of the numbers (1, 2, 3) equals 6:
>>>
>>> assert sum([1, 2, 3]) == 6, "Should be 6"
This will not output anything on the REPL because the values are correct.
If the result from sum() is incorrect, this will fail with an AssertionError and the message "Should be 6". Try an assertion statement again with the wrong values to see an AssertionError:
>>>
>>> assert sum([1, 1, 1]) == 6, "Should be 6"Traceback (most recent call last): File "<stdin>", line 1, in <module>AssertionError: Should be 6
In the REPL, you are seeing the raised AssertionError because the result of sum() does not match 6.
Instead of testing on the REPL, you’ll want to put this into a new Python file called test_sum.py and execute it again:
def test_sum(): assert sum([1, 2, 3]) == 6, "Should be 6"if __name__ == "__main__": test_sum() print("Everything passed")
Now you have written a test case, an assertion, and an entry point (the command line). You can now execute this at the command line:
$ python test_sum.pyEverything passed
You can see the successful result, Everything passed.
In Python, sum() accepts any iterable as its first argument. You tested with a list. Now test with a tuple as well. Create a new file called test_sum_2.py with the following code:
def test_sum(): assert sum([1, 2, 3]) == 6, "Should be 6"def test_sum_tuple(): assert sum((1, 2, 2)) == 6, "Should be 6"if __name__ == "__main__": test_sum() test_sum_tuple() print("Everything passed")
When you execute test_sum_2.py, the script will give an error because the sum() of (1, 2, 2) is 5, not 6.
Here you can see how a mistake in your code gives an error on the console with some information on where the error was and what the expected result was.
Writing tests in this way is okay for a simple check, but what if more than one fails? This is where test runners come in.
The test runner is a special application designed for running tests, checking the output, and giving you tools for debugging and diagnosing tests and applications.
There are many test runners available for Python. The one built into the Python standard library is called unittest.
The principles of unittest are easily portable to other frameworks. The three most popular test runners are:
unittestnose or nose2pytest
Choosing the best test runner for your requirements and level of experience is important.
unittest
unittest has been built into the Python standard library since version 2.1. You’ll probably see it in commercial Python applications and open-source projects.
unittest contains both a testing framework and a test runner. unittest has some important requirements for writing and executing tests.
unittest requires that:
- You put your tests into classes as methods
- You use a series of special assertion methods in the
unittest.TestCase class instead of the built-in assert statement
To convert the earlier example to a unittest test case, you would have to:
- Import
unittest from the standard library
- Create a class called
TestSum that inherits from the TestCase class
- Convert the test functions into methods by adding
self as the first argument
- Change the assertions to use the
self.assertEqual() method on the TestCase class
- Change the command-line entry point to call
unittest.main()
Follow those steps by creating a new file test_sum_unittest.py with the following code:
import unittestclass TestSum(unittest.TestCase): def test_sum(self): self.assertEqual(sum([1, 2, 3]), 6, "Should be 6") def test_sum_tuple(self): self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")if __name__ == '__main__': unittest.main()
If you execute this at the command line, you’ll see one success (indicated with .) and one failure (indicated with F):
$ python test_sum_unittest.py.F======================================================================FAIL: test_sum_tuple (__main__.TestSum)----------------------------------------------------------------------Traceback (most recent call last): File "test_sum_unittest.py", line 9, in test_sum_tuple self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")AssertionError: Should be 6----------------------------------------------------------------------Ran 2 tests in 0.001sFAILED (failures=1)
You have just executed two tests using the unittest test runner.
Note: Be careful if you’re writing test cases that need to execute in both Python 2 and 3. In Python 2.7 and below, unittest is called unittest2. If you simply import from unittest, you will get different versions with different features between Python 2 and 3.