Modern Python: Free-threading

Slides

For most of Python’s history, the Global Interpreter Lock (GIL) has meant that only one thread runs Python bytecode at a time. Threads are great for overlapping I/O, but they can’t use more than one core for computation --- for that you reached for multiprocessing, with its pickling and process-startup costs.

PEP 703 changed that. Starting with Python 3.13 there is an official free-threaded build (sometimes written python3.14t) where the GIL can be turned off, and threads run Python in parallel on every core. 3.13 was the experimental debut; 3.14 is where it got fast, so that’s what we require here. It’s the most interesting thing to happen to CPython in years --- and it has real consequences for how we package compiled extensions.

You can check whether the GIL is active at runtime:

>>> import sys
>>> sys._is_gil_enabled()
False

An embarrassingly parallel example¶

To see it work, we need a CPU-bound task that splits cleanly across threads. Estimating $\pi$ by throwing darts is perfect: throw random points into the square $[-1, 1]^2$ and count how many land inside the unit circle. The fraction inside approaches $\pi/4$ . Each dart is independent, so we can run a batch per thread and average the results.

Pure Python¶

The pure-Python version is a plain loop, run across a thread pool:

pi.py

import random
import statistics
from concurrent.futures import ThreadPoolExecutor


def pi(trials: int) -> float:
    ran = random.Random()
    inside = 0
    for _ in range(trials):
        x = ran.uniform(-1, 1)
        y = ran.uniform(-1, 1)
        if x * x + y * y <= 1:
            inside += 1
    return 4.0 * inside / trials


def pi_in_threads(threads: int, trials: int) -> float:
    if threads == 0:
        return pi(trials)
    chunks = [trials // threads] * threads
    with ThreadPoolExecutor(max_workers=threads) as executor:
        return statistics.mean(executor.map(pi, chunks))

On a normal (GIL-enabled) interpreter, adding threads doesn’t help --- the GIL serializes them, so you get one core’s worth of work no matter what. On a free-threaded build, the same code speeds up with each core.

Run it with a free-threaded interpreter (uv will fetch one for the t suffix):

uv run --python 3.14t sample.py

sample.py

import sys
import time

from freecomputepi.pi import pi_in_threads

TRIALS = 20_000_000

gil = sys._is_gil_enabled()
print(f"Python {sys.version.split()[0]}, GIL {'enabled' if gil else 'disabled'}")

for threads in [1, 2, 4, 8]:
    start = time.monotonic()
    result = pi_in_threads(threads, TRIALS)
    elapsed = time.monotonic() - start
    print(f"{threads:>2} threads: pi = {result:.5f}  ({elapsed:.2f} s)")

Python 3.14.6, GIL disabled
 1 threads: pi = 3.14159  (2.22 s)
 2 threads: pi = 3.14158  (1.13 s)
 4 threads: pi = 3.14150  (0.60 s)
 8 threads: pi = 3.14201  (0.43 s)

Drop the t (uv run --python 3.14) and the times stay flat no matter how many threads you add --- that’s the GIL.

Compiled: releasing the GIL for real¶

Pure Python is now parallel, but it’s still Python-slow. The real win is a compiled inner loop that runs in parallel and fast. There’s a catch: an extension has to declare that it doesn’t need the GIL. If you import any extension that hasn’t opted in, CPython silently switches the GIL back on (with a warning) to keep that extension safe --- so every extension in your process has to be free-threading-aware, or nobody gets the speedup.

The compute is identical to the pure version, just in C++. The interesting part is the line that marks the module as GIL-free --- and each tool does it differently:

pybind11

nanobind

C API

pybind11 marks the module in the PYBIND11_MODULE macro with py::mod_gil_not_used() (available since pybind11 2.13):

_core.cpp

#include <pybind11/pybind11.h>

#include <random>

namespace py = pybind11;

// Monte Carlo estimate of pi. The loop touches no Python objects, so nothing is
// shared between threads -- it scales cleanly once the GIL is out of the way.
double pi(int trials) {
    std::random_device rd;
    std::default_random_engine engine(rd());
    std::uniform_real_distribution<double> dist(-1, 1);

    int inside = 0;
    for (int i = 0; i < trials; ++i) {
        double x = dist(engine);
        double y = dist(engine);
        if (x * x + y * y <= 1.0) {
            ++inside;
        }
    }
    return 4.0 * inside / trials;
}

PYBIND11_MODULE(_core, m, py::mod_gil_not_used()) {
    m.def("pi", &pi, "Estimate pi with a Monte Carlo dart throw");
}

nanobind opts in from CMake instead, via the FREE_THREADED flag on nanobind_add_module (see the build config below) --- the module code stays unchanged:

_core.cpp

#include <nanobind/nanobind.h>

#include <random>

namespace nb = nanobind;

// Monte Carlo estimate of pi. The loop touches no Python objects, so nothing is
// shared between threads -- it scales cleanly once the GIL is out of the way.
double pi(int trials) {
    std::random_device rd;
    std::default_random_engine engine(rd());
    std::uniform_real_distribution<double> dist(-1, 1);

    int inside = 0;
    for (int i = 0; i < trials; ++i) {
        double x = dist(engine);
        double y = dist(engine);
        if (x * x + y * y <= 1.0) {
            ++inside;
        }
    }
    return 4.0 * inside / trials;
}

NB_MODULE(_core, m) {
    m.def("pi", &pi, "Estimate pi with a Monte Carlo dart throw");
}

With the raw C API nothing is generated for you: you write the argument conversion, the method table, and the module export by hand. This version targets Python 3.15, where the module export is a slot array returned from a PyModExport_<name> hook (PEP 793), and Py_mod_gil is the opt-in slot. Since 3.15’s stable ABI also gains free-threading support (PEP 803), the module builds against the limited API, producing a single _core.abi3t.so that future free-threaded Pythons can load without recompiling:

_core.cpp

#include <Python.h>

#include <random>

// Monte Carlo estimate of pi. The loop touches no Python objects, so nothing is
// shared between threads -- it scales cleanly once the GIL is out of the way.
double pi(int trials) {
    std::random_device rd;
    std::default_random_engine engine(rd());
    std::uniform_real_distribution<double> dist(-1, 1);

    int inside = 0;
    for (int i = 0; i < trials; ++i) {
        double x = dist(engine);
        double y = dist(engine);
        if (x * x + y * y <= 1.0) {
            ++inside;
        }
    }
    return 4.0 * inside / trials;
}

// Everything below is the binding boilerplate pybind11/nanobind write for us:
// argument conversion, the method table, and the module export.
static PyObject *pi_py(PyObject *, PyObject *arg) {
    long trials = PyLong_AsLong(arg);
    if (trials == -1 && PyErr_Occurred()) {
        return nullptr;
    }
    return PyFloat_FromDouble(pi(static_cast<int>(trials)));
}

static PyMethodDef methods[] = {
    {"pi", pi_py, METH_O, "Estimate pi with a Monte Carlo dart throw"},
    {},
};

// The 3.15+ stable ABI covers free-threaded builds (abi3t, PEP 803), but
// hides PyModuleDef: instead, a PyModExport_<name> hook returns a slot array
// (PEP 793). Py_mod_gil is the free-threading opt-in.
PyABIInfo_VAR(abi_info);

static PySlot slots[] = {
    PySlot_STATIC_DATA(Py_mod_name, (void *)"_core"),
    PySlot_STATIC_DATA(Py_mod_methods, methods),
    PySlot_DATA(Py_mod_abi, &abi_info),
    PySlot_DATA(Py_mod_gil, Py_MOD_GIL_NOT_USED),
    PySlot_END,
};

PyMODEXPORT_FUNC PyModExport__core(void) { return slots; }

A thin Python wrapper spreads the work over a thread pool, exactly as the pure version did --- it just imports pi from the compiled _core instead:

pi.py

import statistics
from concurrent.futures import ThreadPoolExecutor

from ._core import pi


def pi_in_threads(threads: int, trials: int) -> float:
    if threads == 0:
        return pi(trials)
    chunks = [trials // threads] * threads
    with ThreadPoolExecutor(max_workers=threads) as executor:
        return statistics.mean(executor.map(pi, chunks))

Build configuration¶

The CMake is a standard scikit-build-core extension build. nanobind is where the free-threading opt-in lives (FREE_THREADED); the others need nothing special here. The C API version uses FindPython’s python_add_library instead of a binding tool’s wrapper, with USE_SABI 3.15 (and the Development.SABIModule component) selecting the limited API for the stable-ABI build:

pybind11

nanobind

C API

CMakeLists.txt

cmake_minimum_required(VERSION 3.26...4.4)
project(freecomputepi LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17 CACHE STRING "The C++ standard to use")
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

find_package(pybind11 CONFIG REQUIRED)

pybind11_add_module(_core freecomputepi/_core.cpp)
install(TARGETS _core DESTINATION freecomputepi)

CMakeLists.txt

cmake_minimum_required(VERSION 3.26...4.4)
project(freecomputepi LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17 CACHE STRING "The C++ standard to use")
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

find_package(Python 3.14 REQUIRED COMPONENTS Interpreter Development.Module)
find_package(nanobind CONFIG REQUIRED)

nanobind_add_module(_core FREE_THREADED freecomputepi/_core.cpp)
install(TARGETS _core DESTINATION freecomputepi)

CMakeLists.txt

cmake_minimum_required(VERSION 4.4)
project(freecomputepi LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17 CACHE STRING "The C++ standard to use")
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

find_package(Python 3.15 REQUIRED COMPONENTS Interpreter Development.Module Development.SABIModule)

python_add_library(_core MODULE freecomputepi/_core.cpp WITH_SOABI USE_SABI 3.15)
install(TARGETS _core DESTINATION freecomputepi)

The pyproject.toml differs only in the binding tool it requires --- the C API version needs none, though it requires Python 3.15 for the new module export:

pybind11

nanobind

C API

pyproject.toml

[build-system]
requires = ["scikit-build-core>=1", "pybind11>=3"]
build-backend = "scikit_build_core.build"

[project]
name = "freecomputepi"
version = "0.0.1"
requires-python = ">=3.14"

[tool.scikit-build]
wheel.exclude = ["**.cpp"]

[tool.cibuildwheel]
build = "cp314*"

# Rebuild the editable install when the build config or C++ sources change
[tool.uv]
cache-keys = [
    { file = "pyproject.toml" },
    { file = "CMakeLists.txt" },
    { file = "freecomputepi/**/*.cpp" },
]

pyproject.toml

[build-system]
requires = ["scikit-build-core>=1", "nanobind>=2"]
build-backend = "scikit_build_core.build"

[project]
name = "freecomputepi"
version = "0.0.1"
requires-python = ">=3.14"

[tool.scikit-build]
wheel.exclude = ["**.cpp"]

[tool.cibuildwheel]
build = "cp314*"

# Rebuild the editable install when the build config or C++ sources change
[tool.uv]
cache-keys = [
    { file = "pyproject.toml" },
    { file = "CMakeLists.txt" },
    { file = "freecomputepi/**/*.cpp" },
]

pyproject.toml

[build-system]
requires = ["scikit-build-core>=1"]
build-backend = "scikit_build_core.build"

[project]
name = "freecomputepi"
version = "0.0.1"
requires-python = ">=3.15"

[tool.scikit-build]
# Build a single stable-ABI free-threaded wheel (cp315-abi3t) that future
# free-threaded Pythons can load without recompiling
wheel.py-api = "cp315t"
wheel.exclude = ["**.cpp"]

[tool.cibuildwheel]
build = "cp315*"

# Rebuild the editable install when the build config or C++ sources change
[tool.uv]
cache-keys = [
    { file = "pyproject.toml" },
    { file = "CMakeLists.txt" },
    { file = "freecomputepi/**/*.cpp" },
]

Build and run¶

uv run builds the extension and runs the same benchmark (use --python 3.15t for the C API version):

uv run --python 3.14t sample.py

Python 3.14.6, GIL disabled
 1 threads: pi = 3.14141  (0.26 s)
 2 threads: pi = 3.14199  (0.13 s)
 4 threads: pi = 3.14148  (0.07 s)
 8 threads: pi = 3.14149  (0.07 s)

Same near-linear scaling as pure Python, but an order of magnitude faster per thread. All three compiled versions produce identical timings --- the choice is about the binding style, not the parallelism.

Building wheels¶

Free-threaded wheels use a distinct ABI tag (cp314t), so they’re separate artifacts from the regular cp314 wheels. cibuildwheel builds them for you --- as of 3.14 free-threading is no longer experimental, so they’re on by default with no enable needed:

[tool.cibuildwheel]
build = "cp314*"

The cp314* pattern matches both the cp314 and cp314t identifiers, so each job emits both a normal and a free-threaded wheel (the 3.15-only C API example uses cp315* the same way). Users on a free-threaded interpreter automatically get the t wheel; the GIL stays off, and their threads finally use every core.