Chapter 1. Introduction

A derivative measures how an output changes when an input changes. That sentence is simple, but it is one of the main ideas behind numerical computing, optimization, machine...

7 items

Section	Title
1	Chapter 1. Introduction
2	Numerical Differentiation
3	Symbolic Differentiation
4	Automatic Differentiation
5	Accuracy, Complexity, and Stability
6	Historical Development
7	Applications Across Science and Engineering

Writes › Book › Auto Diff › Chapter 1. Introduction ›

Chapter 1. Introduction

A derivative measures how an output changes when an input changes. That sentence is simple, but it is one of the main ideas behind numerical computing, optimization, machine...

Writes › Book › Auto Diff › Chapter 1. Introduction ›

Numerical Differentiation

Numerical differentiation estimates derivatives by evaluating a function at nearby input values. It treats the function as a black box. The method does not need access to the...

Writes › Book › Auto Diff › Chapter 1. Introduction ›

Symbolic Differentiation

Symbolic differentiation computes derivatives by manipulating expressions. The input is a formula. The output is another formula.

Writes › Book › Auto Diff › Chapter 1. Introduction ›

Automatic Differentiation

Automatic differentiation computes derivatives by applying the chain rule to the operations of a program. The input is ordinary code that computes a value. The output is code,...

Writes › Book › Auto Diff › Chapter 1. Introduction ›

Accuracy, Complexity, and Stability

Derivative computation is not only a mathematical problem. It is also a numerical and systems problem. A derivative method must answer three questions simultaneously:

Writes › Book › Auto Diff › Chapter 1. Introduction ›

Historical Development

Automatic differentiation developed from a simple observation: a numerical computation already contains the structure needed to compute its derivative. The program evaluates...

Writes › Book › Auto Diff › Chapter 1. Introduction ›

Applications Across Science and Engineering

Automatic differentiation became important because derivatives are required everywhere numerical models are optimized, controlled, calibrated, or analyzed. Once a system can...