Chapter 19. Theory and Foundations

Automatic differentiation is often described by a simple rule:

9 items

Section	Title
1	Chapter 19. Theory and Foundations
2	Algebraic Semantics
3	Categorical Semantics
4	Differential Categories
5	Lambda Calculus and AD
6	Program Equivalence
7	Formal Verification
8	Denotational Models
9	Differentiation as Functorial Transformation

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Chapter 19. Theory and Foundations

Automatic differentiation is often described by a simple rule:

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Algebraic Semantics

Automatic differentiation is often introduced operationally. A program executes elementary operations, and derivative information propagates alongside the computation. This...

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Categorical Semantics

Algebraic semantics describes differentiation through derivations, tangent maps, and linear structure. Categorical semantics goes further. It studies differentiation as a...

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Differential Categories

Cartesian differential categories model differentiation in categories with products. Differential categories generalize this idea further by shifting attention from cartesian...

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Lambda Calculus and AD

Automatic differentiation becomes substantially more difficult once programs contain higher-order functions.

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Program Equivalence

Automatic differentiation transforms programs. A fundamental semantic question therefore arises:

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Formal Verification

Automatic differentiation systems are trusted infrastructure. Scientific computing, machine learning, optimization, simulation, and control systems depend on gradients being...

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Denotational Models

Operational semantics explains how automatic differentiation executes. Denotational semantics explains what differentiable programs mean.

Writes › Book › Auto Diff › Chapter 19. Theory and Foundations ›

Differentiation as Functorial Transformation

The preceding sections described automatic differentiation through algebraic, categorical, logical, and denotational models. These viewpoints converge on one central idea: