Chapter 18. Advanced Topics

Many programs do not compute their output by applying a fixed sequence of explicit operations. Instead, they define the output as the solution of another problem.

10 items

Section	Title
1	Chapter 18. Advanced Topics
2	Differentiating Through Solvers
3	Differentiable Optimization Layers
4	Continuous-Time Adjoint Methods
5	Neural ODEs
6	Probabilistic Automatic Differentiation
7	Quantum Differentiation
8	Differentiable Programming Languages
9	Verified Differentiation
10	Unified Differentiable Infrastructure

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Chapter 18. Advanced Topics

Many programs do not compute their output by applying a fixed sequence of explicit operations. Instead, they define the output as the solution of another problem.

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Differentiating Through Solvers

A solver is a program that computes a value by search, iteration, or factorization. Instead of evaluating a closed-form expression, it finds a value that satisfies a condition.

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Differentiable Optimization Layers

An optimization layer is a program component whose output is the solution of an optimization problem. Instead of computing

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Continuous-Time Adjoint Methods

Many systems evolve continuously over time rather than through discrete layers. A state variable changes according to a differential equation:

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Neural ODEs

Classical neural networks apply a finite sequence of transformations:

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Probabilistic Automatic Differentiation

Classical automatic differentiation computes derivatives of deterministic programs.

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Quantum Differentiation

Quantum computation introduces a computational model fundamentally different from classical programs.

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Differentiable Programming Languages

Automatic differentiation began as a transformation applied to numerical programs. A differentiable programming language instead treats differentiation as a native semantic...

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Verified Differentiation

Automatic differentiation systems are usually trusted because they implement mathematically established rules such as the chain rule, product rule, and linearization of...

Writes › Book › Auto Diff › Chapter 18. Advanced Topics ›

Unified Differentiable Infrastructure

Automatic differentiation began as a numerical technique for computing gradients of scalar functions.