Internals
CPython is the reference implementation of Python. The source tree
on github.com/python/cpython at the 3.14 tag is the canonical
answer to "what does Python do here?". These pages describe how
the tree is laid out, what each subsystem is responsible for, and
which functions to read first.
The goal is parity with the gopy internals pillar. Every page in the gopy pillar names a CPython file it mirrors; every page here goes deeper into the C than the Go port has room for. A reader can keep both windows open and treat them as paired references.
What the pillar covers
The internals pillar is about the interpreter, not the language. Syntax, semantics, and the standard library reference live in the Python documentation. What lives here is the C: the data structures, the dispatch loops, the optimisation passes, the memory model. PEPs that change the implementation surface (PEP 657, PEP 659, PEP 669, PEP 703, PEP 744) get a section on the page they touch.
A reader who has never opened the CPython source can start at the top of the pillar. A reader who has read the source but wants a pointer into a specific subsystem can skip to the page that names it. Every section is self-contained.
Layout
The CPython source tree splits roughly five ways:
| Directory | Holds |
|---|---|
Parser/ | Tokenizer, PEG parser, ASDL generators, grammar. |
Python/ | The compile pipeline, the eval loop, the specializer, the optimizer, the runtime services. |
Objects/ | The built-in types: int, float, str, list, dict, tuple, set, type, function, code, frame. |
Modules/ | C-implemented standard-library modules. gcmodule.c, signalmodule.c, _threadmodule.c, etc. |
Include/ | Headers. Include/ is the stable public API; Include/cpython/ is unstable public; Include/internal/ is private. |
The pillar groups these directories into four editorial pillars:
| Pillar | Subsystems |
|---|---|
| Pipeline | Tokenizer, parser, AST, symbol table, codegen, flowgraph, assembler. |
| Execution | Eval loop, frames, specializer, Tier-2 optimizer, monitoring, GIL, exceptions, generators. |
| Runtime | Object model, type system, garbage collector, import machinery. |
| Foundations | Numbers, persistent hash maps, hashing, format spec, codecs, time, lifecycle. |
The pillars are an editorial grouping, not a build-time one; the CPython build is one shared library plus a thin entry binary.
Reading order
The pillar is built to be read top to bottom. Each page is self-contained, but the recommended path is:
- Start with pipeline. It traces a
.pyfile from bytes to aPyCodeObjectand names every C file involved. - Read the Pipeline group end to end:
parser,ast,symtable,compile. By the end ofcompileyou know the shape of a code object and the layout of the exception table. - Cross to Execution. Read
vmfirst; everything else in the group hangs off the dispatch loop.framewalks the runtime stack.specializerandoptimizerare the two adaptive tiers.monitorissys.monitoring.gil,exceptions, andgeneratorsclose the chapter. - Read Runtime for the object model and the import machinery.
- Read Foundations for the low-level primitives the interpreter rests on: numeric types, hashing, the format mini-language, codecs, clocks, and the init/finalise dance.
Conventions
Every page follows a fixed shape:
- A one-paragraph lede that names the subsystem.
- A source map table: file path, role, key entry points.
- The body, organised into noun-phrase sections, each describing one mechanism with at least one concrete listing or citation.
- A PEP touchpoints section for any PEP the subsystem implements.
- A Reference list of CPython files, PEPs, and external papers.
Citations are inline. A CPython citation reads
`Python/ceval.c:1234 _PyEval_EvalFrameDefault`. Line numbers
drift as upstream refactors; the file plus function name remain
stable anchors. When the line number matters (when the page
quotes a specific snippet), it is pinned to CPython 3.14.0; when
the function is what matters, the line is a hint.
Code listings are real C copied from the tree, not paraphrased.
A listing too long to inline is trimmed with /* ... */ rather
than reflowed. Header listings include the surrounding
#ifdef only when the conditional carries meaning.
CPython idioms to internalise
Three idioms recur on almost every page:
- Reference counting. Every
PyObject *carries anob_refcnt. Functions document whether they "steal", "borrow", or "return a new reference".Py_INCREFbumps the count;Py_DECREFlowers it and may run finalisers.Py_NewRef(x)returnsxafter bumping its count, useful for transferring ownership. PEP 683 introduced immortal objects in 3.12 (refcount pinned at_Py_IMMORTAL_REFCNT); singletons and small ints are immortal. - The GIL. A single per-interpreter lock guards Python state.
C code that runs without the GIL uses
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS. The 3.13+ free-threaded build (PEP 703) optionally compiles the GIL out and replaces it with finer-grained locks and biased reference counting; the 3.14 default build still holds the GIL. - Slot dispatch. Every type is a
PyTypeObjectfilled with function pointers (tp_call,tp_iter,nb_add, ...). Operations on values reach into the type for the slot. The slots are populated either directly (for C types) or throughfixup_slot_dispatchers(for Python types that define dunder methods).
Status disclosure
CPython is the reference; "status" here means which PEPs the 3.14
release shipped versus which remain in flight. The pages name the
PEP-by-PEP status where it matters. Free threading is shipped as
an optional build mode (PEP 703); the Tier-2 optimizer is enabled
behind PYTHON_UOPS=1 and the JIT behind --enable-experimental-jit
(PEP 744); sys.monitoring is fully implemented (PEP 669).
Cross-pillar references
The CPython internals pillar links forward and backward within itself. It links across to the gopy pillar where a Go port file mirrors the C function being discussed. It does not link to the specs pillar; the port specs are about gopy, not CPython.
Reference
- The CPython developer's guide.
- PEP 626. Precise line numbers for debugging.
- PEP 657. Including fine-grained error locations in tracebacks.
- PEP 659. Specializing adaptive interpreter.
- PEP 669. Low impact monitoring for CPython.
- PEP 683. Immortal objects, using a fixed refcount.
- PEP 684. A per-interpreter GIL.
- PEP 703. Making the global interpreter lock optional in CPython.
- PEP 744. JIT compilation.
- CPython 3.14 source tree at
https://github.com/python/cpython.