Testing

Test-Driven Development (TDD) of Django APP

Testing Django App With Pytest

Pytest is a framework for the testing application. Pytest suggests much more pythonic tests without boilerplate.

Why we should use Pytest?

Pytest provides a new approach for writing tests, namely, functional testing for applications and libraries.

• Integration tests

• Less boilerplate code

• one asserts keyword for the test (No need to remember assertEqual, assertTrue... )

• Auto-discovery

• Parametrizing

• Detailed info on failing assert statements

• External plugins

• Coverage

• Parallel testing

• Better integration with CI/CD

• Run Django tests

Pytest Fixtures

Fixtures are functions that run before and after each test, like setUp and tearDown in unitest and labelled Pytest killer feature. Fixtures are used for data configuration, connection/disconnection of databases, calling extra actions, etc.

All fixtures have scope arguments with available values.

• function run once per test

• class run once per class of tests

• module runs once per module

• session runs once per session

NOTE: Default value of scope is the function

import pytest


@pytest.fixture
def function_fixture():
   print('Fixture for each test')
   return 1


@pytest.fixture(scope='module')
def module_fixture():
   print('Fixture for module')
   return 2

Another kind of fixture is the yield fixture which provides access to test before and after the run, analogous to setUp and tearDown.

import pytest

@pytest.fixture
def simple_yield_fixture():
   print('setUp part')
   yield 3
   print('tearDown part')

Use Fixtures with tests in Pytest

To use the fixture in the test, we can put fixture name as a function argument:

def test_function_fixture(function_fixture):
  assert function_fixture == 1

def test_yield_fixture(simple_yield_fixture):
  assert simple_yield_fixture == 3

Note: Pytest automatically registers our fixtures and can have access to fixtures without extra imports.

Parametrize in Pytest

Parametrize` is a built-in mark and one of the killer features of Pytest. With this mark, you can perform multiple calls to the same test function.

import pytest

@pytest.mark.parametrize(
   'text_input, result', [('5+5', 10), ('1+4', 5)]
)
def test_sum(text_input, result):
   assert eval(text_input) == result

Test-Driven Development (TDD) of APIs

As you may have already learned, test-driven development is an approach that focuses on writing tests before you start implementing the business logic of your application. Writing tests first requires you to really consider what do you want from the code. But apart from this, TDD has numerous other benefits:

1. Fast feedback and detailed specification;

2. Reduced time spent on reworking and time spent in the debugger;

3. Maintainable, flexible, and easily extensible code;

4. Shorter development time to market;

5. Increased developer’s productivity;

6. SOLID code; and

7. A clean interface.

Setting up Pytest for Django Project

1. Installation

pip install pytest-django

2. Point our Django Settings to Pytest

We need to tell Pytest which Django settings should be used for test runs. The easiest way to achieve this is to create a Pytest configuration file with this information.

Create a file called pytest.ini in your project root directory that contains:

[pytest]
DJANGO_SETTINGS_MODULE = yourproject.settings
python_files = tests.py test_*.py *_tests.py

3. Run tests

pytest -v

Specific test files or directories or single test can be selected by specifying the test file names directly on the command line:

pytest a_directory                     # directory
pytest test_something.py               # tests file
pytest test_something.py::single_test  # single test function

Django Testing with Pytest

1. Database Helpers

To gain access to the database pytest-django get django_db mark or request one of the db, transactional_db or django_db_reset_sequences fixtures.

django_db: to get access to the Django test database, each test will run in its own transaction that will be rolled back at the end of the test

import pytest

from django.contrib.auth.models import User


@pytest.mark.django_db
def test_user_create():
  User.objects.create_user('john', '[e-mail]', 'johnpassword')
  assert User.objects.count() == 1

2. Client

The more frequently used thing in Django unit testing is django.test.client, because we use it for each request to our app, pytest-django has a build-in fixture client:

import pytest

from django.urls import reverse

@pytest.mark.django_db
def test_view(client):
   url = reverse('homepage-url')
   response = client.get(url)
   assert response.status_code == 200

3. Admin Client

To get a view with superuser access, we can use admin_client, which gives us client with login superuser:

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_unauthorized(client):
   url = reverse('superuser-url')
   response = client.get(url)
   assert response.status_code == 401


@pytest.mark.django_db
def test_superuser_view(admin_client):
   url = reverse('superuser-url')
   response = admin_client.get(url)
   assert response.status_code == 200

4. Create User Fixture

To create a user for our test we have two options:

1) Use Pytest Django Fixtures:

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_user_detail(client, django_user_model):
   user = django_user_model.objects.create(
       username='someone', password='password'
   )
   url = reverse('user-detail-view', kwargs={'pk': user.pk})
   response = client.get(url)
   assert response.status_code == 200
   assert 'someone' in response.content

django_user_model: pytest-django helper for the shortcut to the User model configured for use by the current Django project, like settings.AUTH_USER_MODEL

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_superuser_detail(client, admin_user):
   url = reverse(
       'superuser-detail-view', kwargs={'pk': admin_user.pk}
   )
   response = client.get(url)
   assert response.status_code == 200
   assert 'admin' in response.content

admin_user: pytest-django helper instance of a superuser, with username “admin” and password “password” (in case there is no “admin” user yet).

2) Create own Fixture:

import uuid

import pytest


@pytest.fixture
def test_password():
   return 'strong-test-pass'

  
@pytest.fixture
def create_user(db, django_user_model, test_password):
   def make_user(**kwargs):
       kwargs['password'] = test_password
       if 'username' not in kwargs:
           kwargs['username'] = str(uuid.uuid4())
       return django_user_model.objects.create_user(**kwargs)
   return make_user

Re-write tests above:

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_user_detail(client, create_user):
   user = create_user(username='someone')
   url = reverse('user-detail-view', kwargs={'pk': user.pk})
   response = client.get(url)
   assert response.status_code == 200
   assert 'someone' in response.content


@pytest.mark.django_db
def test_superuser_detail(client, create_user):
   admin_user = create_user(
       username='custom-admin-name',
       is_staff=True, is_superuser=True
   )
   url = reverse(
       'superuser-detail-view', kwargs={'pk': admin_user.pk}
   )
   response = client.get(url)
   assert response.status_code == 200
   assert 'custom-admin-name' in response.content

5. Auto Login Client

Let’s test some authenticated endpoints:

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_auth_view(client, create_user, test_password):
   user = create_user()
   url = reverse('auth-url')
   client.login(
       username=user.username, password=test_password
   )
   response = client.get(url)
   assert response.status_code == 200

The major disadvantage of this method is that we must copy the login block for each test.

Let’s create our own fixture for auto-login users:

import pytest


@pytest.fixture
def auto_login_user(db, client, create_user, test_password):
   def make_auto_login(user=None):
       if user is None:
           user = create_user()
       client.login(username=user.username, password=test_password)
       return client, user
   return make_auto_login

auto_login_user: own fixture, that takes a user as a parameter or creates a new one and logins it to client fixture. And at the end, it returns the client and user back for future actions.

Use our new fixture for the test above:

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_auth_view(auto_login_user):
   client, user = auto_login_user()
   url = reverse('auth-url')
   response = client.get(url)
   assert response.status_code == 200

6. Parametrizing tests with Pytest

Let’s say we must test very similar functionality, for example, different languages.

Previously, you had to do single tests, like:

...
def test_de_language():
   ...
def test_gr_language():
   ...
def test_en_language():
   ...

It’s very funny to copy paste your test code, but not for a long time.

— Andrew Svetlov

To fix it, Pytest has parametrizing fixtures feature. After the upgrade we had the next tests:

import pytest

from django.urls import reverse


@pytest.mark.django_db
@pytest.mark.parametrize([
   ('gr', 'Yasou'),
   ('de', 'Guten tag'),
   ('fr', 'Bonjour')
])
def test_languages(language_code, text, client):
   url = reverse('say-hello-url')
   response = client.get(
       url, data={'language_code': language_code}
   )
   assert response.status_code == 200
   assert text == response.content

7. Test Mail Outbox with Pytest

For testing your mail outbox pytest-django has a built-in fixture mailoutbox:

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_send_report(auto_login_user, mailoutbox):
   client, user = auto_login_user()
   url = reverse('send-report-url')
   response = client.post(url)
   assert response.status_code == 201
   assert len(mailoutbox) == 1
   mail = mailoutbox[0]
   assert mail.subject == f'Report to {user.email}'
   assert list(mail.to) == [user.email]

Testing Django REST Framework with Pytest

1. API Client

The first thing to do here is to create our own fixture for API Client of REST Framework:

import pytest


@pytest.fixture
def api_client():
   from rest_framework.test import APIClient
   return APIClient()

Now we have api_client for our tests:

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_unauthorized_request(api_client):
   url = reverse('need-token-url')
   response = api_client.get(url)
   assert response.status_code == 401

2. Get or Create Token

For getting authorized, your API users usually uses Token. Let’s create a fixture to get or create a token for a user:

import pytest

from rest_framework.authtoken.models import Token


@pytest.fixture
def get_or_create_token(db, create_user):
   user = create_user()
   token, _ = Token.objects.get_or_create(user=user)
   return token

get_or_create_token: inheritance create_user

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_unauthorized_request(api_client, get_or_create_token):
   url = reverse('need-token-url')
   token = get_or_create_token()
   api_client.credentials(HTTP_AUTHORIZATION='Token ' + token.key)
   response = api_client.get(url)
   assert response.status_code == 200

3. Auto Credentials

The test demonstrated above is a good example, but setting credentials for each test will end up in a boilerplate code. And we can use other APIClient method to bypass authentication entirely.

We can use yield feature to extend new fixture:

import pytest


@pytest.fixture
def api_client_with_credentials(
   db, create_user, api_client
):
   user = create_user()
   api_client.force_authenticate(user=user)
   yield api_client
   api_client.force_authenticate(user=None)

api_client_with_credentials: inheritance create_user and api_client fixtures and also clear our credential after every test.

import pytest

from django.urls import reverse


@pytest.mark.django_db
def test_authorized_request(api_client_with_credentials):
   url = reverse('need-auth-url')
   response = api_client_with_credentials.get(url)
   assert response.status_code == 200

4. Data Validation with Pytest Parametrizing

Most tests for your API endpoint constitute and focus on data validation. You have to create the same tests without counting the difference in several values. We can use Pytest parametrizing fixture for such solution:

import pytest


@pytest.mark.django_db
@pytest.mark.parametrize(
   'email, password, status_code', [
       (None, None, 400),
       (None, 'strong_pass', 400),
       ('[e-mail]', None, 400),
       ('[e-mail]', 'invalid_pass', 400),
       ('[e-mail]', 'strong_pass', 201),
   ]
)
def test_login_data_validation(
   email, password, status_code, api_client
):
   url = reverse('login-url')
   data = {
       'email': email,
       'password': password
   }
   response = api_client.post(url, data=data)
   assert response.status_code == status_code

Useful Tips for Pytest

1. Using Factory Boy as fixtures for testing Django model

There are several ways to create Django Model instance for test and example with fixture:

• Create object manually — traditional variant: “create test data by hand and support it by hand”

import pytest


from django.contrib.auth.models import User
@pytest.fixture
def user_fixture(db):
   return User.objects.create_user(
       'john', '[e-mail]', 'johnpassword'
   )

If you want to add other fields like relation with Group, your fixture will get more complex and every new required field will change your fixture:

import pytest

from django.contrib.auth.models import User, Group


@pytest.fixture
def default_group_fixture(db):
   default_group, _ = Group.objects.get_or_create(name='default')
   return default_group

@pytest.fixture
def user_with_default_group_fixture(db, default_group_fixture):
   user = User.objects.create_user(
       'john', '[e-mail]', 'johnpassword',
       groups=[default_group_fixture]
   )
   return user

• Django fixtures — slow and hard to maintain… avoid them!

Below I provide an example for comparison:

[
 {
 "model": "auth.group",
 "fields": {
   "name": "default",
   "permissions": [
     29,45,46,47,48
   ]
 }
},
{
 "model": "auth.user",
 "pk": 1,
 "fields": {
   "username": "simple_user",
   "first_name": "John",
   "last_name": "Lennon",
   "groups": [1],
 }
},
// create permissions here
]

Create fixture that loads fixture data to your session:

import pytest

from django.core.management import call_command


@pytest.fixture(scope='session')
def django_db_setup(django_db_setup, django_db_blocker):
   with django_db_blocker.unblock():
       call_command('loaddata', ‘fixture.json')

• Factories — a solution for creation of your test data in a simple way. I’d prefer to use pytest-factoryboy plugin and factoryboy alongside with Pytest. Alternatively, you may use model mommy.

1) Install the plugin:

pip install pytest-factoryboy

2) Create User Factory:

import factory

from django.contrib.auth.models import User, Group


class UserFactory(factory.DjangoModelFactory):
  class Meta:
       model = User

   username = factory.Sequence(lambda n: f'john{n}')
   email = factory.Sequence(lambda n: f'lennon{n}@thebeatles.com')
   password = factory.PostGenerationMethodCall(
       'set_password', 'johnpassword'
   )

   @factory.post_generation
   def has_default_group(self, create, extracted, **kwargs):
       if not create:
           return
       if extracted:
           default_group, _ = Group.objects.get_or_create(
               name='group'
           )
           self.groups.add(default_group)

3) Register Factory:

from pytest_factoryboy import register

from factories import UserFactory


register(UserFactory)  # name of fixture is user_factory

Note: Name convention is a lowercase-underscore class name

4) Test your Factory:

import pytest


@pytest.mark.django_db
def test_user_user_factory(user_factory):
   user = user_factory(has_default_group=True)
   assert user.username == 'john0'
   assert user.email == '[e-mail]'
   assert user.check_password('johnpassword')
   assert user.groups.count() == 1

You may read more about pytest-factoryboy and factoryboy.

2. Mocking your Pytest test with fixture

Using pytest-mock plugin is another way to mock your code with pytest approach of naming fixtures as parameters.

1) Install the plugin:

pip install pytest-mock

2) Re-write example above:

import pytest


@pytest.mark.django_db
def test_send_new_event_service_called(
   mocker, default_event_data, api_client
):
   mock_send_new_event = mocker.patch(
       'service.ThirdPartyService.send_new_event'
   )
   response = api_client.post(
       'create-service', data=default_event_data
   )

   assert response.status_code == 201
   assert response.data['id']
   mock_send_new_event.assert_called_with(
       event_id=response.data['id']
   )

The mocker is a fixture that has the same API as mock.patch and supports the same methods as:

• mocker.patch

• mocker.patch.object

• mocker.patch.multiple

• mocker.patch.dict

• mocker.stopall

3. Running tests simultaneously

To speed up your tests, you can run them simultaneously. This can result in significant speed improvements on multi core/multi CPU machines. It’s possible to realize with pytest-xdist plugin which expands pytest functionality

1) Install the plugin:

pip install pytest-xdist

2) Running test with multiprocessing:

pytest -n <number_of_processes>

4. Config pytest.ini file

Example of pytest.inifile for your Django project:

[pytest]
DJANGO_SETTINGS_MODULE = yourproject.settings
python_files = tests.py test_*.py *_tests.py
addopts = -p no:warnings --strict-markers --no-migrations --reuse-db
norecursedirs = venv old_tests
markers =
   custom_mark: some information of your mark
   slow: another one slow tes

DJANGO_SETTINGS_MODULE and python_files we discussed at the beginning of the article, let’s discover other useful options:

• addopts Add the specified OPTS to the set of command-line arguments as if they had been specified by the user. We’ve specified next options:

--p no:warnings — disables warning capture entirely (this might be useful if your test suites handle warnings using an external system)

--strict-markers — typos and duplication in function markers are treated as an error

--no-migrations will disable Django migrations and create the database by inspecting all models. It may be faster when there are several migrations to run in the database setup.

--reuse-db reuses the testing database between test runs. It provides much faster startup time for tests.

Exemplary workflow with --reuse-db and --create-db:

– run tests with pytest; on the first run the test database will be created. On the next test run it will be reused.

– when you alter your database schema, run pytest --create-db to force re-creation of the test database.

• norecursedirs Set the exclusion of directory basename patterns when recursing for test discovery. This will tell pytest not to look into venv and old_testsdirectory

Note: Default patterns are '.*', 'build', 'dist', 'CVS', '_darcs', '{arch}', '*.egg', 'venv'

• markers You can list additional markers in this setting to add them to the whitelist and use them in your tests.

Run all tests with mark slow:

pytest -m slow

5. Show your coverage of the test

To check coverage of your app you can use pytest-cov plugin

1) Install plugin:

pip install pytest-cov

2) Coverage of your project and example of report:

pytest --cov

-------------------- coverage: ... ---------------------
Name                 Stmts   Miss  Cover
----------------------------------------
proj/__init__          2      0    100%
proj/apps              257    13   94%
proj/proj              94     7    92%
----------------------------------------
TOTAL                  353    20   94%

References

djangostars blog for django-pytest-testing