Skip to main content

Internals

CPython is the reference implementation of Python. The source tree on github.com/python/cpython at the 3.14 tag is the canonical answer to "what does Python do here?". These pages describe how the tree is laid out, what each subsystem is responsible for, and which functions to read first.

The goal is parity with the gopy internals pillar. Every page in the gopy pillar names a CPython file it mirrors; every page here goes deeper into the C than the Go port has room for. A reader can keep both windows open and treat them as paired references.

What the pillar covers

The internals pillar is about the interpreter, not the language. Syntax, semantics, and the standard library reference live in the Python documentation. What lives here is the C: the data structures, the dispatch loops, the optimisation passes, the memory model. PEPs that change the implementation surface (PEP 657, PEP 659, PEP 669, PEP 703, PEP 744) get a section on the page they touch.

A reader who has never opened the CPython source can start at the top of the pillar. A reader who has read the source but wants a pointer into a specific subsystem can skip to the page that names it. Every section is self-contained.

Layout

The CPython source tree splits roughly five ways:

DirectoryHolds
Parser/Tokenizer, PEG parser, ASDL generators, grammar.
Python/The compile pipeline, the eval loop, the specializer, the optimizer, the runtime services.
Objects/The built-in types: int, float, str, list, dict, tuple, set, type, function, code, frame.
Modules/C-implemented standard-library modules. gcmodule.c, signalmodule.c, _threadmodule.c, etc.
Include/Headers. Include/ is the stable public API; Include/cpython/ is unstable public; Include/internal/ is private.

The pillar groups these directories into four editorial pillars:

PillarSubsystems
PipelineTokenizer, parser, AST, symbol table, codegen, flowgraph, assembler.
ExecutionEval loop, frames, specializer, Tier-2 optimizer, monitoring, GIL, exceptions, generators.
RuntimeObject model, type system, garbage collector, import machinery.
FoundationsNumbers, persistent hash maps, hashing, format spec, codecs, time, lifecycle.

The pillars are an editorial grouping, not a build-time one; the CPython build is one shared library plus a thin entry binary.

Reading order

The pillar is built to be read top to bottom. Each page is self-contained, but the recommended path is:

  1. Start with pipeline. It traces a .py file from bytes to a PyCodeObject and names every C file involved.
  2. Read the Pipeline group end to end: parser, ast, symtable, compile. By the end of compile you know the shape of a code object and the layout of the exception table.
  3. Cross to Execution. Read vm first; everything else in the group hangs off the dispatch loop. frame walks the runtime stack. specializer and optimizer are the two adaptive tiers. monitor is sys.monitoring. gil, exceptions, and generators close the chapter.
  4. Read Runtime for the object model and the import machinery.
  5. Read Foundations for the low-level primitives the interpreter rests on: numeric types, hashing, the format mini-language, codecs, clocks, and the init/finalise dance.

Conventions

Every page follows a fixed shape:

  • A one-paragraph lede that names the subsystem.
  • A source map table: file path, role, key entry points.
  • The body, organised into noun-phrase sections, each describing one mechanism with at least one concrete listing or citation.
  • A PEP touchpoints section for any PEP the subsystem implements.
  • A Reference list of CPython files, PEPs, and external papers.

Citations are inline. A CPython citation reads `Python/ceval.c:1234 _PyEval_EvalFrameDefault`. Line numbers drift as upstream refactors; the file plus function name remain stable anchors. When the line number matters (when the page quotes a specific snippet), it is pinned to CPython 3.14.0; when the function is what matters, the line is a hint.

Code listings are real C copied from the tree, not paraphrased. A listing too long to inline is trimmed with /* ... */ rather than reflowed. Header listings include the surrounding #ifdef only when the conditional carries meaning.

CPython idioms to internalise

Three idioms recur on almost every page:

  • Reference counting. Every PyObject * carries an ob_refcnt. Functions document whether they "steal", "borrow", or "return a new reference". Py_INCREF bumps the count; Py_DECREF lowers it and may run finalisers. Py_NewRef(x) returns x after bumping its count, useful for transferring ownership. PEP 683 introduced immortal objects in 3.12 (refcount pinned at _Py_IMMORTAL_REFCNT); singletons and small ints are immortal.
  • The GIL. A single per-interpreter lock guards Python state. C code that runs without the GIL uses Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS. The 3.13+ free-threaded build (PEP 703) optionally compiles the GIL out and replaces it with finer-grained locks and biased reference counting; the 3.14 default build still holds the GIL.
  • Slot dispatch. Every type is a PyTypeObject filled with function pointers (tp_call, tp_iter, nb_add, ...). Operations on values reach into the type for the slot. The slots are populated either directly (for C types) or through fixup_slot_dispatchers (for Python types that define dunder methods).

Status disclosure

CPython is the reference; "status" here means which PEPs the 3.14 release shipped versus which remain in flight. The pages name the PEP-by-PEP status where it matters. Free threading is shipped as an optional build mode (PEP 703); the Tier-2 optimizer is enabled behind PYTHON_UOPS=1 and the JIT behind --enable-experimental-jit (PEP 744); sys.monitoring is fully implemented (PEP 669).

Cross-pillar references

The CPython internals pillar links forward and backward within itself. It links across to the gopy pillar where a Go port file mirrors the C function being discussed. It does not link to the specs pillar; the port specs are about gopy, not CPython.

Reference

  • The CPython developer's guide.
  • PEP 626. Precise line numbers for debugging.
  • PEP 657. Including fine-grained error locations in tracebacks.
  • PEP 659. Specializing adaptive interpreter.
  • PEP 669. Low impact monitoring for CPython.
  • PEP 683. Immortal objects, using a fixed refcount.
  • PEP 684. A per-interpreter GIL.
  • PEP 703. Making the global interpreter lock optional in CPython.
  • PEP 744. JIT compilation.
  • CPython 3.14 source tree at https://github.com/python/cpython.