Best development practices and the SP guide

Slides

Intro to the Scientific Python guide¶

The Scientific Python Development Guide is packed with great guides for package authors! It collects modern best practices into a single place, keeps them in sync with a project template (cookie), and provides a tool (sp-repo-review) that grades your repo against them.

You’ve already built a package. This chapter is a whirlwind tour of what a healthy package looks like around that code: tests, static checks, and types. Each section links to the full guide page if you want to go deeper.

Testing¶

Tests let you refactor with confidence and immediately know when a new Python or platform breaks. The whole ecosystem has standardized on pytest; since tests are a developer-only requirement, there’s no reason not to use it.

The killer feature is that plain assert gives rich failure output, no self.assertEqual zoo:

def test_funct():
    assert 4 == 2**2

Put tests in a tests/ folder (no __init__.py), name files test_*.py, and configure pytest strictly in pyproject.toml:

[tool.pytest]
minversion = "9.0"
addopts = ["-ra", "--showlocals"]
strict = true
filterwarnings = ["error"]
testpaths = ["tests"]
log_level = "INFO"

The most important line is filterwarnings = ["error"]: it turns warnings into failures, so you fix deprecations before your users hit them.

A few patterns you’ll reach for constantly:

from pytest import approx


def test_approx():
    assert 1 / 3 == approx(0.3333333333333)

import pytest


def test_raises():
    with pytest.raises(ZeroDivisionError):
        1 / 0

import pytest


@pytest.mark.parametrize("n", [1, 2, 3])
def test_square(n):
    assert n**2 == n * n

def test_printout(capsys):
    print("hello")
    assert "hello" in capsys.readouterr().out

git clone https://github.com/pypa/packaging
cd packaging
uv run pytest

Once you have tests, measure how much of your code they exercise with coverage. Run it via coverage run -m pytest (or pytest --cov=<pkg>) and upload to Codecov in CI. Chase meaningful coverage; weak tests added just to bump the number aren’t worth it.

Static checks¶

Static checks run without executing your code. The guide drives them all through pre-commit (or the faster Rust rewrite prek, which you already installed). One config file, checks in isolated environments, one command:

prek run -a      # run every hook on every file
prek install     # optional: run automatically on git commit

A minimal .pre-commit-config.yaml with the essentials:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: "v6.0.0"
    hooks:
      - id: check-added-large-files
      - id: check-merge-conflict
      - id: end-of-file-fixer
      - id: trailing-whitespace
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: "v0.15.18"
    hooks:
      - id: ruff-check
        args: ["--fix"]
      - id: ruff-format

The star is Ruff: a single ultra-fast Rust binary that replaces Flake8, isort, pyupgrade, and dozens of other tools. It is both a linter (ruff-check) and a formatter (ruff-format, a near-perfect Black clone). Configure the linter in pyproject.toml by opting into rule families:

[tool.ruff]
show-fixes = true
lint.extend-select = [
  "B",    # flake8-bugbear: likely bugs
  "I",    # isort: sorted imports
  "UP",   # pyupgrade: modernize syntax
  "RUF",  # Ruff-specific rules
]

The guide covers many more optional hooks, like spelling (codespell/typos), prettier for YAML/Markdown/JSON, check-jsonschema for CI files, clang-format for C++, and more. Add them as your project grows.

uvx ruff check --fix --show-fixes messy.py   # removes unused `os`, sorts imports
uvx ruff format messy.py                      # normalizes formatting

Typing¶

Static type hints are the biggest shift in modern Python. They document intent, help readers, and---via a type checker---catch whole classes of bugs before you run anything:

def f(x: int) -> int:
    return x * 5

Types do nothing at runtime (with from __future__ import annotations they’re not even evaluated), but a checker like MyPy verifies you’re not lying about them across every branch; including code your tests rarely hit. Add it as a pre-commit hook and configure it strictly in pyproject.toml:

[tool.mypy]
files = "src"
python_version = "3.10"
strict = true

Typing is gradual: MyPy checks almost nothing by default, so you can adopt it one module at a time and tighten toward strict = true.

Two features worth knowing early:

def g(x: str | int) -> None:
    if isinstance(x, str):
        print(x.lower())  # checker knows x is str here
    else:
        print(x + 1)  # ...and int here

from typing import Protocol


class Duck(Protocol):
    def quack(self) -> str: ...


def pester(d: Duck) -> None:
    print(d.quack())

A good rule of thumb: accept the most general type you can, return the most specific. Take an Iterable[str], return a dict[str, int], not the other way around.

See the full typing guide for stubs, TypedDict, generics, and more.

Other pages¶

The guide keeps going well past this tour. Skim these when the need arises:

Using the cookiecutter¶

Rather than assembling all of this by hand, cookie generates a new package that already follows the guide. It supports ten build backends, including the compiled ones (pybind11, scikit-build-core, maturin, …) you’ll use later in this workshop.

Run it with no install using whichever templating tool you prefer:

uvx copier copy gh:scientific-python/cookie my-package

uvx cookiecutter gh:scientific-python/cookie

It asks a handful of questions (name, license, backend, …) and writes a ready-to-go repository.

uvx copier copy gh:scientific-python/cookie my-package   # pick "hatchling"
cd my-package
uv run pytest

Using sp-repo-review¶

Already have a project? sp-repo-review checks it against the guide and gives you a checklist of what’s followed and what’s missing. Green means good, red is a suggestion; some failures are fine, the goal is to surface issues, not force compliance.

Run it locally on any repository:

uvx 'sp-repo-review[cli]' <path to repo>

Every check has a code (like PY006 for pre-commit or PP302 for the pytest minversion) that links back to the exact spot in the guide explaining it.

uvx 'sp-repo-review[cli]' my-package        # freshly generated: mostly green
uvx 'sp-repo-review[cli]' ~/some/old/repo   # older: plenty of red to work through