Nov 15, 2021 Python

Introduction to Testing in Django and Django Rest Framework

This article will go over the basics of testing and how to write tests for our Django APIs.

I'm gonna be honest, I only started writing tests 3 months ago. It was the start of my new project and I've decided to develop the habit of writing good maintainable code.

I didn't know how to judge my code apart from code reviews with other developers but I quickly understood from research that good code requires tests.

So I've embarked on the journey of writing tests and it was pretty bumpy the first few times. My tests were long and slow, colleagues were complaining but I eventually got the hang of it.

Right now testing is something I unconsciously do and I believe I've become a much better developer because of it.

Coming back to this article, I've wanted to share what I learned during the past three months and hopefully convince you to write some tests yourself.

I know this is a Django Rest API testing tutorial, but the principles learned here can be applied to all programming languages and domains.

Prerequisites

Basic knowledge of Python, Django and Django Rest Framework.
Python installed locally in your machine.

Why should you test?

The first hurdle I had to overcome was to convince myself and my manager on why should I spend more time on tasks to write tests. I did my research and have come up with eight different reasons to write tests.

Easy to Find Bugs

Tests help to find bugs in two distinct ways.

The first way is that if you test the common functionality then finding the edge cases where your code might break is much easier. These kinds of bugs are hard to find manually because they require certain scenarios to be present, that's why writing tests for edge cases make sure that they never happen again.

The second way that tests help with bugs, is that it immediately notifies you of regression bugs. Regression bugs happen when you alter or add new features which cause old features to break. This is one of the most important benefits of writing tests because it provides a safety net to write code. You can safely refactor code and if all tests pass, you can rest assured that no regression bugs were caused.

I wholly believe that tests allowed me to experiment with my code using different best practices. I've become a much better developer because of this in a short period.

Testing Saves Time and Money

Another myth that many developers and managers have against testing is that testing slows down development hence the project becomes more expensive. This myth has been busted by real-world statistics.

Tests do slow down in the short term but save thousands of hours and millions of dollars in the long term. Projects over time get more complicated, changes become harder to modify and features harder to add. But with tests, we can consistently refactor code to changing requirements and easily add new features without fear of breaking old ones.

On the other hand, if you didn't have any tests, there would be no guarantee that the system is fully working and most of the development time will be spent on tracking bugs hence losing the company money.

Testing is an Integral Part of Extreme Programming

Extreme programming is a programming methodology that focuses on software quality and consistent changes in business requirements. It advocates for frequent releases in short cycles. This is essentially Agile methodology but more code-oriented.

Testing is a central part of extreme programming. The methodology is that if a few tests can eliminate a few flaws, then lots of testing will eliminate lots of flaws.

You can read more about Extreme Programming here.

Testing Provides Documentation

Testing is a good way to document code because if another developer is ever confused about some class or method, he can directly read its tests to understand how it works.

This is also good for developer onboarding. New developers would be less intimidated by the codebase and will feel more at ease with the safety net caused by tests.

Testing Improves Code Quality

Messy code leads to messy tests. That's why I usually prefer to write a clean test first which forces me to think of ways of writing better code.

This encompasses all aspects of the code which includes readability, maintainability, and complexity.

A good tip is that if it's too hard to test something then it's too complex. It would be advisable to refactor your code to something better.

Testing Helps Gauge Performance

Testing doesn't only include code, it also includes the performance of the system. These are called performance tests. Performance tests help us gauge how the system performs under a particular load.

They are essential in big companies because they have billions of users but many small companies use them to gauge how much they can scale.

Types of Tests

There are many different types of tests. Initially, we talked about performance tests that gauge how much load the system can handle. There are two categories of tests.

Functional Tests – Tests that test the functionality of the program. For example, tests to check if an API returns the correct response.
Non-Functional Tests – Tests that test the non-functional aspects of the program such as performance, reliability, usability, and security.

Software engineers usually deal only with functional tests. That's why I'm gonna quickly go over the most popular types of functional tests.

Unit Tests

Unit tests involve the smallest possible unit in software design. This can include a single unit or a group of interrelated units. But what do I mean by unit?

A unit can a class or a function or even a module. It's up to you as a developer to define the boundaries of what a unit is.

Integrations Tests

Integrations tests involve testing compatibility between components. For example, if I have a YouTube player that consists of two modules:

Search – Searches through the YouTube API.
Player Module – Given a link, the player will play a video.

I would write a unit test for each module, and an integration test to check if the two modules are working together.

PS. I've built a youtube player before if you are interested in how it's done. I've made a tutorial here.

End to End Tests

End to end testing (E2E Testing) is a software testing method that tests the application from beginning to end.

Imagine you have a simple to-do-list program, an E2E test would involve launching the site, adding or deleting todos, and finally closing the site.

This tests the full functionality of the website from start to finish.

Usually, these types of tests are written by Quality Assurance Engineers but I've seen developers do them in small teams.

What should you test?

Lots of developers get stuck in this part. They have some features ready to be tested and get cold feet. They know testing every little thing is bad but they don't know exactly what to test.

To battle that I've created a mini guideline on deciding what to test.

If it's a native module such as datetime, then don't test it.
If it's some library code, such as models from Django then don't test it.
If its core business logic, such as your program only allows users to be 18+ then test it.
Test common cases, for example, if all your APIs require pagination, then test to make sure that they are included.
Test edge cases, if a feature breaks in certain scenarios then make sure to write a test for it that is consistent.
If you find any bugs, write a test so that it does not happen again.

When to write tests?

This is a pretty controversial topic. The two schools of thought are:

Test-Driven Development, where you write your tests first and then code.
Write your code first and then write tests.

In my opinion, it does not matter. I like to write tests before because it helps me think of the feature that I'm going to implement and if any edge cases might arise.

The main point is to not get stuck in this part and to simply write tests.

Basics

Before we write our tests there are some prerequisite questions you might such as how to structure our tests and a brief introduction to PyTest.

File Structure

# Structure 1
program/
├─ app1/
│  ├─ tests/
│  │  ├─ app1_tests.py
├─ app2/
│  ├─ tests/
│  │  ├─ app2_tests.py

# Structure 2
program/
├─ app1/
├─ app2/
├─ tests/
│  ├─ app1_tests.py
│  ├─ app2_tests.py

There are two ways to organize our tests:

Have a tests folder in each of our apps.
Have a root tests folder that contains all our tests.

I prefer the first way because it provides better organization.

The other thing is our naming conventions. It's best practice for our test files to all start with test_. For example, if we are testing our serializers, then our test file would be called test_serializers.py. This is good because it provides structure and PyTest will automatically find the test files.

Why PyTest?

Why use some third party library when we can natively use the Django test modules?

PyTest is much cleaner than Django tests.

Let's take a hello world example.

This is how it would be done using Django tests:

from django.test import TestCase


class TestHelloWorld(TestCase):
   def test_hello_world(self):
       self.assertEqual("hello world", "hello world")

PyTest:

import pytest

@pytest.mark.unit
def test_hello_world():
   assert "hello_world" == "hello_world"

Other benefits of PyTest include:

Running tests faster using multiple cores.
Managing test dependencies using fixtures.
Many third-party plugins support PyTest.

You can check out the plugin list here.

Plugin List — pytest documentation

Logo

Test Flow

The test flow is a four-step process in writing tests:

Arrange – You set up the test, you might be testing an API that needs an HTTP client, or you might need data in the database.
Mock – This is an optional step, you don't always have to mock but sometimes you are just forced to. If you are using some third party API, you don't want real calls to it, so you simply mock it.
Act – This is where you do the deed, and call the action. If you're testing an API, this is where you would call the request.
Assert – This is where you test your conditions and this place determines whether your test passes or fails. If it's an API you would assert that the response was successful by checking the status code returned and asserting that it's 200

Testing DRF APIs

Now that all the theory is done, let us do something practical and test our APIs.

Our Program

Our program is a simple DRF app that has one model called Article. We also have an API that lists all articles.

This is how our code looks like:

# articles/model.py
from django.db import models

class Article(models.Model):
    title = models.CharField(max_length=100)
    content = models.TextField()

    def __str__(self):
        return self.title

# articles/views.py
from articles.models import Article
from articles.serializers import ArticleSerialize

class ArticleView(generics.ListAPIView):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer

Installing and Configuring PyTest

Install PyTest using pip:

pip install pytest-django

Let's configure PyTest to work on our project.

Create a file called pytest.ini and add this code:

# pytest.ini
[pytest]
DJANGO_SETTINGS_MODULE = main_app_name.settings
python_files = tests.py test_*.py *_tests.py

We told PyTest where our settings module is and which files to treat as tests.

Testing our Articles API

We created a tests.py file in our articles app, and added the following code:

import pytest

from django.urls import reverse

from articles.models import Article
from articles.serializers import ArticleSerializer


@pytest.mark.django_db
def test_list_articles(client):
    url = reverse('list-articles')
    response = client.get(url)

    articles = Article.objects.all()
    expected_data = ArticleSerializer(articles, many=True).data

    assert response.status_code == 200
    assert response.data == expected_dat

We first mark our test with django_db to tell PyTest that this test has access to the database. Then we get our URL and call the request. Finally, we validate our data using asserts.

This is a pretty simple test because there's just isn't much to test in our list API.

Anyways, to run our tests simply run.

pytest

If all tests pass it should look something like this:

Conclusion

I know the first few tests may seem weird, slow and clunky.

But stick with it.

It's a habit that pays off in the long run.

Slowly as the project grows, you will see how your tests make it much easier to find bugs and avoid regression bugs.

Anyways, I hope you learned something today and if you have any criticisms I would love to hear them.

At the end of the day, we are software engineers.

We grow by helping and listening to each other.

Thanks for reading.