Lesson 17 · Pytest deep dive

pytest Tutorial #17: CI Integration & Configuration

On testing real Python code with pytest. pytest.ini, pyproject.toml, marker registration, and --strict-markers. Let's get into it. pytest.ini, under a…

⬇ Download notebook

CoursePytest deep dive
Lesson17 of 18
FormatJupyter notebook · 6 code cells

pytest

What you'll learn

A Helper - Building a Real Small Project on Disk
pytest.ini - the Classic Config File
pyproject.toml - the Modern Equivalent
Registering Markers to Silence the Unknown-Mark Warning
--strict-markers - Making Unregistered Markers a Hard Error
A Real Pattern - a CI-Style Project, Zero Extra Flags
Wrap-Up: What You Learned

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

pytest Deep-Dive, Video 17: CI Integration and Config#

Video seventeen of an eighteen-part series on testing real Python code with pytest.
pytest.ini, pyproject.toml, marker registration, and --strict-markers.
Let's get into it.

Part 1: A Helper - Building a Real Small Project on Disk#

import pytest, tempfile, os
def build_project(config_filename, config_content, test_files):
    tmpdir = tempfile.mkdtemp()
    with open(os.path.join(tmpdir, config_filename), 'w') as f:
        f.write(config_content)
    tests_dir = os.path.join(tmpdir, 'tests')
    os.makedirs(tests_dir, exist_ok=True)
    for name, content in test_files.items():
        with open(os.path.join(tests_dir, name), 'w') as f:
            f.write(content)
    return tmpdir

Part 2: pytest.ini - the Classic Config File#

ini_content = """[pytest]
addopts = -v --tb=short
"""
test_content = """
def test_one():
    assert 1 + 1 == 2

def test_two():
    assert 2 + 2 == 4
"""
project_dir = build_project('pytest.ini', ini_content, {'test_ini_demo.py': test_content})
exit_code = pytest.main([os.path.join(project_dir, 'tests')])
print(exit_code)

============================= test session starts =============================
platform win32 -- Python 3.12.1, pytest-9.0.2, pluggy-1.6.0 -- c:\Users\makmw\AppData\Local\Programs\Python\Python312\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\makmw\AppData\Local\Temp\tmpzf9am7jc
configfile: pytest.ini
plugins: anyio-4.14.2, dash-3.2.0, Faker-40.4.0, typeguard-4.4.4
collecting ... collected 2 items

..\..\..\Users\makmw\AppData\Local\Temp\tmpzf9am7jc\tests\test_ini_demo.py::test_one PASSED [ 50%]
..\..\..\Users\makmw\AppData\Local\Temp\tmpzf9am7jc\tests\test_ini_demo.py::test_two PASSED [100%]

============================== 2 passed in 0.12s ==============================
0

Part 3: pyproject.toml - the Modern Equivalent#

toml_content = """[tool.pytest.ini_options]
addopts = "-v --tb=short"
"""
test_content = """
def test_three():
    assert 3 * 3 == 9
"""
project_dir = build_project('pyproject.toml', toml_content, {'test_toml_demo.py': test_content})
exit_code = pytest.main([os.path.join(project_dir, 'tests')])
print(exit_code)

============================= test session starts =============================
platform win32 -- Python 3.12.1, pytest-9.0.2, pluggy-1.6.0 -- c:\Users\makmw\AppData\Local\Programs\Python\Python312\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\makmw\AppData\Local\Temp\tmp0udl0xhg
configfile: pyproject.toml
plugins: anyio-4.14.2, dash-3.2.0, Faker-40.4.0, typeguard-4.4.4
collecting ... collected 1 item

..\..\..\Users\makmw\AppData\Local\Temp\tmp0udl0xhg\tests\test_toml_demo.py::test_three PASSED [100%]

============================== 1 passed in 0.02s ==============================
0

Part 4: Registering Markers to Silence the Unknown-Mark Warning#

ini_content = """[pytest]
markers =
    slow: marks tests as slow
"""
test_content = """
import pytest

@pytest.mark.slow
def test_marked_slow():
    assert True
"""
project_dir = build_project('pytest.ini', ini_content, {'test_markers_demo.py': test_content})
exit_code = pytest.main([os.path.join(project_dir, 'tests'), '-v', '--no-header'])
print(exit_code)

============================= test session starts =============================
collecting ... collected 1 item

..\..\..\Users\makmw\AppData\Local\Temp\tmpsk63f05q\tests\test_markers_demo.py::test_marked_slow PASSED [100%]

============================== 1 passed in 0.02s ==============================
0

Part 5: --strict-markers - Making Unregistered Markers a Hard Error#

ini_content = """[pytest]
markers =
    slow: marks tests as slow
"""
test_content = """
import pytest

@pytest.mark.totally_unregistered
def test_x():
    assert True
"""
project_dir = build_project('pytest.ini', ini_content, {'test_typo.py': test_content})
exit_code = pytest.main([os.path.join(project_dir, 'tests'), '--strict-markers', '-v', '--no-header'])
print(exit_code)

============================= test session starts =============================
collecting ... collected 0 items / 1 error

=================================== ERRORS ====================================
_____________________ ERROR collecting tests/test_typo.py _____________________
'totally_unregistered' not found in `markers` configuration option
=========================== short test summary info ===========================
ERROR ..\..\..\Users\makmw\AppData\Local\Temp\tmpbkpxlre1\tests\test_typo.py - Failed: 'totally_unregistered' not found in `markers` configuration option
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 0.26s ===============================
2

Part 6: A Real Pattern - a CI-Style Project, Zero Extra Flags#

toml_content = """[tool.pytest.ini_options]
addopts = "-v --tb=short --strict-markers"
testpaths = ["tests"]
markers = [
    "slow: marks tests as slow",
    "integration: marks integration tests",
]
"""
test_content = """
import pytest

@pytest.mark.slow
def test_slow_case():
    assert True

@pytest.mark.integration
def test_integration_case():
    assert True

def test_plain_case():
    assert True
"""
project_dir = build_project('pyproject.toml', toml_content, {'test_full.py': test_content})
exit_code = pytest.main([project_dir])
print(exit_code)

============================= test session starts =============================
platform win32 -- Python 3.12.1, pytest-9.0.2, pluggy-1.6.0 -- c:\Users\makmw\AppData\Local\Programs\Python\Python312\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\makmw\AppData\Local\Temp\tmpctjhcl6g
configfile: pyproject.toml
plugins: anyio-4.14.2, dash-3.2.0, Faker-40.4.0, typeguard-4.4.4
collecting ... collected 3 items

..\..\..\Users\makmw\AppData\Local\Temp\tmpctjhcl6g\tests\test_full.py::test_slow_case PASSED [ 33%]
..\..\..\Users\makmw\AppData\Local\Temp\tmpctjhcl6g\tests\test_full.py::test_integration_case PASSED [ 66%]
..\..\..\Users\makmw\AppData\Local\Temp\tmpctjhcl6g\tests\test_full.py::test_plain_case PASSED [100%]

============================== 3 passed in 0.03s ==============================
0

Wrap-Up: What You Learned#

pytest.ini, under a [pytest] section, configures a project so CI never has to pass repeated flags manually.
pyproject.toml offers the same configuration under [tool.pytest.ini_options], TOML-quoted.
addopts injects default command-line flags automatically for every pytest invocation in that project.
Listing a name under markers registers it, silencing the PytestUnknownMarkWarning for that marker.
--strict-markers turns any unregistered marker into a hard collection error, ideal for catching typos in CI.
testpaths restricts default test discovery to specific directories when no path is given explicitly.
A complete pyproject.toml lets a CI runner invoke plain pytest with zero extra flags, fully config-driven.
That wraps up CI integration and config. Next up: the Capstone - Testing a Real Small Package End-to-End.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.

☕ Buy me a coffee Other ways to help