pytest

Pytest is a test framework for python. It detects the test files and functions based on their names : test_example.py, example_test().

installation

pip3 install -U pytest
pytest --version

• -U : upgrade if not installed

To test the installation, use something small, and call you file either test_xxx or xxx_test.py

# content of test_sample.py

def func(x):
    return x + 1

def test_answer():
    assert func(3) == 5

So this is a forced wrong assertion, run pytest

pytest 

It will end in red coloured output, ending in :

=================================== FAILURES ===================================
_________________________________ test_answer __________________________________

    def test_answer():
>       assert func(3) == 5
E       assert 4 == 5
E        +  where 4 = func(3)

sample_test.py:7: AssertionError
====================== 1 failed, 2 passed in 0.08 seconds ======================

Mostly your error will be the last section to display. Some handy flags to find the problem better are these :

python -slx

• -s : means print the normal output of the program. It will be hidden by default.
• -l : show the local variables
• -x : stop after the first error

However in this simple case, it would not make a difference (no local variables, 1 error and no output)

test discovery

Pytest follows these rules when detecting tests :

• start from the current directory or the one given from the commandline
• Descend into directories, unless they match norecursedirs.
• In those directories, search for test.py or _test.py files, imported by their test package name.

• From those files, collect test items

• test_ prefixed test functions or methods outside of class.

• test_ prefixed test functions or methods inside Test prefixed test classes (without an init method).

If you need to test a specific directory, file or function, use this notation:

pytest dir/
pytest dir/test_file.py
pytest dir/test_file.py::test_function

The less you specify, the wider the test set.

exception testing

To expect an exception use raises()

# content of test_sysexit.py
import pytest
def f():
    raise SystemExit(1)

def test_mytest():
    with pytest.raises(SystemExit):
        f()

test classes

You can just use flat functions, but a test class can be used as well:

# content of test_class.py
class TestClass(object):
    def test_one(self):
        x = "this"
        assert 'h' in x

    def test_two(self):
        x = "hello"
        assert hasattr(x, 'check')

This class does not have to be instantiated, the test_ functions just get executed like flat functions are.

parameterization

This one is handy, to run a function multiple times with other parameters. This example shows three parameter sets, where the last will fail.

# content of test_expectation.py
import pytest
@pytest.mark.parametrize("test_input,expected", [
    ("3+5", 8),
    ("2+4", 6),
    ("6*9", 42),
])
def test_eval(test_input, expected):
     assert eval(test_input) == expected

The parameters are mapped to the test_eval parameters and so this is like running test_eval 3 times :

test_eval("3+5", 8);
test_eval("2+r4", 6);
test_eval("6*9", 42);

That opens up some possibilities. I could alter the login function for the portal to :

# test all logins and failures I can think of
@pytest.mark.parametrize("uname,pwd,expected", [
    ("kees","welcome08",True),
    ("kees","notwelcome08",False),
    ("sven","welcome08",True),
    ("jmaas","notwelcome08",False),
])
def test_auth_parm(cdriver,uname,pwd,expected):
    logout(cdriver)
    assert page_contains(cdriver,"Inloggen") == True
    login(cdriver,uname,pwd)
    assert page_contains(cdriver,"ingelogd") == expected
    assert page_contains(cdriver,"Inloggen") != expected

This compared to a string of blocks each with another setup. You can also use this to parallelize tests with the pytest-parallel plugin. When you install python-parallel you can specify how many workers you want running the tests.

pip install python-parallel
python --workers auto

Or any number, auto will assign workers according to your number of cpu's/cores.

This plugin just takes each test_ and puts it in a worker. So if you parametrize your function like this :

@pytest.mark.parametrize("repeat", [
    (11),
    (41),
    (21),
    (31),
])
def test_loop(cdriver,repeat):
    for i in range(0,repeat):
        initialize_user(cdriver);

You effectively made 4 test_loop functions, each can be assigned to a worker, and they will each loop through another number of tests in 4 separate browsers. The statistics message in the end will report 4 tests extra, not the number of loops !

============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-3.7.4, py-1.6.0, pluggy-0.7.1
rootdir: /home/kees/projects/portal/test, inifile:
plugins: parallel-0.0.2
collected 9 items                                                              
pytest-parallel: 4 workers (processes), 1 test per worker (thread)
.........
========================== 9 passed in 533.18 seconds ==========================

To be precise, it counts every function with test_ or _test in the title, and when they are parametrized multiple times.