Publishing and CI

Slides

You have a package and a set of healthy-development tools around it. This chapter automates both: run your tests and linters on every pull request with GitHub Actions, then publish to PyPI with one click using Trusted Publishing. This page covers pure Python packages; compiled packages add one more tool (cibuildwheel) that we’ll meet in the compiled section.

GitHub Actions in a nutshell¶

GitHub Actions (GHA) is the standard CI for scientific Python projects: it’s free for open source, tightly integrated with GitHub, and endlessly extensible. The model is simple:

• A workflow is a YAML file in .github/workflows/ that runs on events (a PR, a push, a release, a button press).

• A workflow contains jobs, which run in parallel on fresh virtual machines unless you say otherwise.

• A job is a list of steps: either shell commands (run:) or reusable actions (uses:).

Every CI workflow starts about the same way:

name: CI

on:
  pull_request:
  push:
    branches:
      - main

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:

This runs on every PR and on pushes to main. The concurrency block is a freebie worth copying everywhere: pushing a fix to a PR cancels the already-obsolete run instead of wasting runner time.

A lint job¶

You already run your static checks through prek/pre-commit, so CI just needs to run the same thing. One config, identical results locally and in CI:

lint:
  name: Lint
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v7
    - uses: j178/prek-action@v2

lint:
  name: Lint
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v7
    - uses: actions/setup-python@v6
      with:
        python-version: "3.x"
    - uses: pre-commit/action@v3.0.1

A test matrix¶

Because your package builds with a standard PEP 517 backend, uses dependency-groups, and tests with pytest, the test job is nearly copy-paste for any package; only the matrix is yours:

tests:
  runs-on: ${{ matrix.runs-on }}
  strategy:
    fail-fast: false
    matrix:
      python-version: ["3.10", "3.12", "3.14"]
      runs-on: [ubuntu-latest, macos-latest, windows-latest]
  name: Test ${{ matrix.python-version }} on ${{ matrix.runs-on }}
  steps:
    - uses: actions/checkout@v7

    - uses: astral-sh/setup-uv@v8

    - name: Test package
      run: uv run --python ${{ matrix.python-version }} pytest

A matrix expands into one job per combination (here 3 × 3 = 9), and fail-fast: false keeps one red job from cancelling the others; you want the full picture. uv run does everything: gets the Python, makes the environment, installs your package, runs pytest. If you have a lockfile committed, it uses that too.

Besides the three -latest images, there are ARM runners (ubuntu-24.04-arm, windows-11-arm), a final Intel macOS image (macos-15-intel), and a fast single-core ubuntu-slim; see the runner images list.

Keeping actions updated¶

Anything pinned with @v7 goes stale. One file, .github/dependabot.yml, gets you a monthly PR (with changelogs) whenever an action updates:

version: 2
updates:
  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "monthly"
    groups:
      actions:
        patterns:
          - "*"

The groups key combines everything into a single PR; cleaner, and required for paired actions like upload-artifact/download-artifact that must move together.

name: CI

on:
  pull_request:
  push:
    branches:
      - main

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: j178/prek-action@v2

  tests:
    runs-on: ${{ matrix.runs-on }}
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.14"]
        runs-on: [ubuntu-latest]
        include:
          - python-version: "3.14"
            runs-on: windows-latest
    steps:
      - uses: actions/checkout@v7
      - uses: astral-sh/setup-uv@v8
      - run: uv run --python ${{ matrix.python-version }} pytest

Publishing to PyPI¶

Publishing means uploading two things: an SDist (the source, always) and wheels (pre-built, ready to install). For a pure Python package the wheel is trivial to make, and you should still always make one: wheels install without running any code (faster, safer, no setup tools needed on the user’s machine), and you can unzip one and see exactly what will land where.

The trigger¶

Publishing gets its own workflow with different triggers; you don’t want every PR uploading to PyPI:

name: CD

on:
  workflow_dispatch:
  release:
    types:
      - published

jobs:

workflow_dispatch gives you a manual “Run workflow” button, perfect for testing the build. The release trigger fires when you publish a GitHub Release. Building runs on both; the actual upload step will be guarded so it only happens on a real release.

Building the distributions¶

dist:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v7
      with:
        fetch-depth: 0

    - uses: hynek/build-and-inspect-python-package@v2

That one action builds the SDist and wheel, runs a battery of checks on the metadata, prints a nice summary, and uploads the result as an artifact named Packages. (The DIY version is run: pipx run build followed by actions/upload-artifact, plus pipx run twine check dist/* for the checks.)

The publish job¶

The modern way to authenticate is Trusted Publishing: you tell PyPI “trust artifacts from this repo’s cd.yml workflow”, and PyPI verifies a short-lived OpenID Connect token that GitHub attaches to the run. No API tokens to generate, store, rotate, or leak.

publish:
  needs: [dist]
  environment: pypi
  permissions:
    id-token: write
    attestations: write
  runs-on: ubuntu-latest
  if: github.event_name == 'release' && github.event.action == 'published'
  steps:
    - uses: actions/download-artifact@v8
      with:
        name: Packages
        path: dist

    - name: Generate artifact attestations
      uses: actions/attest-build-provenance@v4
      with:
        subject-path: "dist/*"

    - uses: pypa/gh-action-pypi-publish@release/v1

publish:
  needs: [dist]
  runs-on: ubuntu-latest
  if: github.event_name == 'release' && github.event.action == 'published'
  steps:
    - uses: actions/download-artifact@v8
      with:
        name: Packages
        path: dist

    - uses: pypa/gh-action-pypi-publish@release/v1
      with:
        password: ${{ secrets.pypi_password }}

Note the three guard rails on this job: needs: [dist] (only runs if the build succeeded), the if: (only on a published release, so workflow_dispatch runs stop after building), and environment: pypi (you can add required reviewers or other protection rules to that environment in the repo settings).

To wire up Trusted Publishing the first time:

1. On PyPI, go to your account’s Publishing page and add a “pending publisher”: your GitHub user/org, repo name, workflow file name (cd.yml), and environment (pypi).

2. On GitHub, create the pypi environment under Settings → Environments.

3. Publish a GitHub Release; the workflow builds, then uploads. Done.

uv build
uvx twine check dist/*

What about compiled packages?¶

Everything above still applies, except one wheel is no longer enough: a compiled package needs a wheel per platform, per architecture (and without the stable ABI, per Python version). That’s a build matrix problem, and cibuildwheel solves it in a few lines of CI. That’s the compiled section’s publishing chapter.