Lesson 12 · Pytest deep dive

pytest Tutorial #12: Coverage with pytest-cov

On testing real Python code with pytest. Line coverage, branch coverage, fail-under, and pragma: no cover. Let's get into it. --cov=modulename enables…

⬇ Download notebook

CoursePytest deep dive
Lesson12 of 18
FormatJupyter notebook · 6 code cells

pytest

What you'll learn

A Helper - Building a Module Plus Test on Disk
term-missing - Which Lines Were Never Reached
Branch Coverage - --cov-branch
--cov-fail-under - Enforcing a Minimum
pragma: no cover - Excluding Lines on Purpose
A Real Pattern - Chasing Down the Missing Branch
Wrap-Up: What You Learned

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

pytest Deep-Dive, Video 12: Coverage With pytest-cov#

Video twelve of an eighteen-part series on testing real Python code with pytest.
Line coverage, branch coverage, fail-under, and pragma: no cover.
Let's get into it.

Part 1: A Helper - Building a Module Plus Test on Disk#

import pytest, tempfile, os, sys
def run_with_coverage(module_code, test_code, module_name, cov_args=None):
    tmpdir = tempfile.mkdtemp()
    for name in (module_name, "test_" + module_name):
        sys.modules.pop(name, None)
    for p in list(sys.path):
        if p.startswith(tempfile.gettempdir()) and p != tmpdir:
            sys.path.remove(p)
    with open(os.path.join(tmpdir, module_name + '.py'), 'w') as f:
        f.write(module_code)
    test_path = os.path.join(tmpdir, 'test_' + module_name + '.py')
    with open(test_path, 'w') as f:
        f.write(test_code)
    sys.path.insert(0, tmpdir)
    args = [test_path, '--cov=' + module_name] + (cov_args or []) + ['-v', '--no-header']
    return pytest.main(args)

Part 2: term-missing - Which Lines Were Never Reached#

module_code = """
def classify(n):
    if n < 0:
        return 'negative'
    else:
        return 'non-negative'
"""
test_code = """
from mymod import classify

def test_classify_negative():
    assert classify(-1) == 'negative'
"""
exit_code = run_with_coverage(module_code, test_code, 'mymod', ['--cov-report=term-missing'])
print(exit_code)

ERROR: usage: ipykernel_launcher.py [options] [file_or_dir] [file_or_dir] [...]
ipykernel_launcher.py: error: unrecognized arguments: --cov=mymod --cov-report=term-missing
  inifile: None
  rootdir: C:\Users\makmw\AppData\Local\Temp\tmpunvuuymk

Part 3: Branch Coverage - --cov-branch#

module_code = """
def classify(n):
    if n < 0:
        return 'negative'
    else:
        return 'non-negative'
"""
test_code = """
from mymod import classify

def test_classify_negative():
    assert classify(-1) == 'negative'
"""
exit_code = run_with_coverage(module_code, test_code, 'mymod', ['--cov-branch', '--cov-report=term-missing'])
print(exit_code)

ERROR: usage: ipykernel_launcher.py [options] [file_or_dir] [file_or_dir] [...]
ipykernel_launcher.py: error: unrecognized arguments: --cov=mymod --cov-branch --cov-report=term-missing
  inifile: None
  rootdir: C:\Users\makmw\AppData\Local\Temp\tmp_0cok1kr

Part 4: --cov-fail-under - Enforcing a Minimum#

module_code = """
def classify(n):
    if n < 0:
        return 'negative'
    else:
        return 'non-negative'
"""
test_code = """
from mymod import classify

def test_classify_negative():
    assert classify(-1) == 'negative'
"""
exit_code = run_with_coverage(module_code, test_code, 'mymod', ['--cov-report=term-missing', '--cov-fail-under=90'])
print(exit_code)

ERROR: usage: ipykernel_launcher.py [options] [file_or_dir] [file_or_dir] [...]
ipykernel_launcher.py: error: unrecognized arguments: --cov=mymod --cov-report=term-missing --cov-fail-under=90
  inifile: None
  rootdir: C:\Users\makmw\AppData\Local\Temp\tmpy595sj_l

Part 5: pragma: no cover - Excluding Lines on Purpose#

module_code = """
def classify(n):
    if n < 0:
        return 'negative'
    else:
        return 'non-negative'

def debug_dump(n):  # pragma: no cover
    print(f"debugging: {n}")
"""
test_code = """
from mymod import classify

def test_classify_negative():
    assert classify(-1) == 'negative'

def test_classify_non_negative():
    assert classify(5) == 'non-negative'
"""
exit_code = run_with_coverage(module_code, test_code, 'mymod', ['--cov-report=term-missing'])
print(exit_code)

ERROR: usage: ipykernel_launcher.py [options] [file_or_dir] [file_or_dir] [...]
ipykernel_launcher.py: error: unrecognized arguments: --cov=mymod --cov-report=term-missing
  inifile: None
  rootdir: C:\Users\makmw\AppData\Local\Temp\tmpsp7lhhn_

Part 6: A Real Pattern - Chasing Down the Missing Branch#

module_code = """
def grade(score):
    if score >= 90:
        return 'A'
    elif score >= 80:
        return 'B'
    elif score >= 70:
        return 'C'
    else:
        return 'F'
"""
incomplete_test_code = """
from mymod import grade

def test_grade_a():
    assert grade(95) == 'A'

def test_grade_b():
    assert grade(85) == 'B'
"""
print('--- before: missing branches ---')
run_with_coverage(module_code, incomplete_test_code, 'mymod', ['--cov-report=term-missing'])
complete_test_code = incomplete_test_code + """
def test_grade_c():
    assert grade(75) == 'C'

def test_grade_f():
    assert grade(50) == 'F'
"""
print('--- after: full coverage ---')
exit_code = run_with_coverage(module_code, complete_test_code, 'mymod', ['--cov-report=term-missing'])
print(exit_code)

--- before: missing branches ---
--- after: full coverage ---
4

ERROR: usage: ipykernel_launcher.py [options] [file_or_dir] [file_or_dir] [...]
ipykernel_launcher.py: error: unrecognized arguments: --cov=mymod --cov-report=term-missing
  inifile: None
  rootdir: C:\Users\makmw\AppData\Local\Temp\tmpjtsyimy8

ERROR: usage: ipykernel_launcher.py [options] [file_or_dir] [file_or_dir] [...]
ipykernel_launcher.py: error: unrecognized arguments: --cov=mymod --cov-report=term-missing
  inifile: None
  rootdir: C:\Users\makmw\AppData\Local\Temp\tmpzchehmpo

Wrap-Up: What You Learned#

--cov=module_name enables coverage measurement for that specific module during a pytest run.
--cov-report=term-missing lists the exact line numbers that never executed, not just a percentage.
--cov-branch also tracks whether both sides of an if statement were actually taken, not just the line.
--cov-fail-under=N fails the overall run with a non-zero exit code when total coverage drops below N.
A trailing # pragma: no cover comment excludes a line or function from the statement count entirely.
Reading the Missing column directly points to which specific test cases are still needed.
That wraps up coverage with pytest-cov. Next up: Temporary Files.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.

☕ Buy me a coffee Other ways to help