Chapter 12. AD in Modern Programming Languages

Lisp is one of the natural homes of automatic differentiation. It treats programs as data, has a simple expression syntax, and supports macro systems that can transform code...

10 items

Section	Title
1	Chapter 12. AD in Modern Programming Languages
2	Functional Languages
3	Differentiable Programming
4	AD in C and C++
5	AD in Python
6	AD in Rust
7	AD in Julia
8	AD in Swift
9	Type Systems for Differentiation
10	Effect Systems and Mutation

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

Chapter 12. AD in Modern Programming Languages

Lisp is one of the natural homes of automatic differentiation. It treats programs as data, has a simple expression syntax, and supports macro systems that can transform code...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

Functional Languages

Functional programming languages provide a natural semantic foundation for automatic differentiation. Programs are expressed as compositions of functions, immutable values,...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

Differentiable Programming

Differentiable programming treats differentiation as a general programming-language feature. A program can contain numerical kernels, control flow, data structures, solvers,...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

AD in C and C++

C and C++ are important targets for automatic differentiation because much scientific, engineering, graphics, finance, and machine learning infrastructure is written in these...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

AD in Python

Python became the dominant language for modern machine learning and differentiable computing because it combines a simple programming model with access to high-performance...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

AD in Rust

Rust is an attractive language for automatic differentiation because it combines low-level performance with strong static guarantees. It gives the programmer control over...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

AD in Julia

Julia was designed for high-performance technical computing. It combines interactive syntax with a compiler capable of specializing code aggressively based on types. This...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

AD in Swift

Swift became an important experiment in language-integrated automatic differentiation because it attempted to make differentiation a core compiler feature rather than a...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

Type Systems for Differentiation

Automatic differentiation interacts deeply with type systems because differentiation changes the structure of computation. A derivative operator maps one function into another...

Writes › Book › Auto Diff › Chapter 12. AD in Modern Programming Languages ›

Effect Systems and Mutation

Automatic differentiation is easiest to define for pure functions. A pure function behaves like a mathematical mapping: it consumes inputs, produces outputs, and has no...