Chapter 3. Programs as Mathematical Objects

A straight-line program is the simplest model of computation used in automatic differentiation. It is a program with a fixed sequence of assignments, no branches, no loops,...

8 items

Section	Title
1	Chapter 3. Programs as Mathematical Objects
2	Intermediate Variables
3	Dependency Graphs
4	Control Flow
5	Loops and Recurrence Relations
6	Memory and State
7	Purity and Side Effects
8	Differentiable Subprograms

Writes › Book › Auto Diff › Chapter 3. Programs as Mathematical Objects ›

Chapter 3. Programs as Mathematical Objects

A straight-line program is the simplest model of computation used in automatic differentiation. It is a program with a fixed sequence of assignments, no branches, no loops,...

Writes › Book › Auto Diff › Chapter 3. Programs as Mathematical Objects ›

Intermediate Variables

Intermediate variables are the named values created between program inputs and program outputs. They make automatic differentiation mechanical.

Writes › Book › Auto Diff › Chapter 3. Programs as Mathematical Objects ›

Dependency Graphs

A dependency graph describes how values in a computation depend on earlier values. Automatic differentiation operates on these dependencies.

Writes › Book › Auto Diff › Chapter 3. Programs as Mathematical Objects ›

Control Flow

Control flow determines which operations a program executes. Straight-line programs have a fixed sequence of operations, but ordinary programs contain branches, loops,...

Writes › Book › Auto Diff › Chapter 3. Programs as Mathematical Objects ›

Loops and Recurrence Relations

Loops express repeated computation. Recurrence relations express the same idea mathematically: each state is computed from one or more earlier states.

Writes › Book › Auto Diff › Chapter 3. Programs as Mathematical Objects ›

Memory and State

Automatic differentiation operates on computations, but computations execute inside a memory model. Variables occupy storage locations, arrays are mutated, buffers are reused,...

Writes › Book › Auto Diff › Chapter 3. Programs as Mathematical Objects ›

Purity and Side Effects

A pure computation is easier to differentiate because every output is determined only by its explicit inputs. There is no hidden state, no external mutation, and no dependence...

Writes › Book › Auto Diff › Chapter 3. Programs as Mathematical Objects ›

Differentiable Subprograms

A differentiable subprogram is a program fragment that can participate in derivative propagation as a coherent unit. Instead of differentiating an entire application...