Chapter 11. Compiler and Runtime Design

Source transformation is an implementation strategy for automatic differentiation in which a program that computes a function is rewritten into another program that computes...

10 items

Section	Title
1	Chapter 11. Compiler and Runtime Design
2	Operator Overloading
3	Intermediate Representations
4	SSA Form
5	Graph IRs
6	Tracing Systems
7	Staging and Partial Evaluation
8	Memory Planning
9	Kernel Fusion
10	Ahead-of-Time vs Just-in-Time Differentiation

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Chapter 11. Compiler and Runtime Design

Source transformation is an implementation strategy for automatic differentiation in which a program that computes a function is rewritten into another program that computes...

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Operator Overloading

Operator overloading implements automatic differentiation by changing the meaning of ordinary arithmetic operations for special numeric objects.

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Intermediate Representations

An intermediate representation, or IR, is the internal program form used by a compiler or AD system after parsing and before final code generation.

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

SSA Form

Static single assignment form, or SSA, is an intermediate representation where each variable is assigned exactly once.

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Graph IRs

A graph intermediate representation models a program as nodes and edges.

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Tracing Systems

Tracing is an implementation strategy where an AD system observes a program while it runs and records the operations that occur.

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Staging and Partial Evaluation

Staging is the separation of a program into phases.

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Memory Planning

Memory planning determines where values are stored, how long they remain alive, and when storage can be reused.

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Kernel Fusion

Kernel fusion combines several small operations into one larger executable unit.

Writes › Book › Auto Diff › Chapter 11. Compiler and Runtime Design ›

Ahead-of-Time vs Just-in-Time Differentiation

Automatic differentiation can be performed before a program runs, while it runs, or in a staged phase between the two.