Chapter 10. Matrix and Tensor Differentiation

Matrix calculus is the notation and rule system used to differentiate functions whose inputs, outputs, or intermediate values are vectors, matrices, or tensors. Automatic...

9 items

Section	Title
1	Chapter 10. Matrix and Tensor Differentiation
2	Tensor Operations
3	Broadcasting Semantics
4	Linear Algebra Primitives
5	Differentiating Factorizations
6	Eigenvalue Problems
7	Singular Value Decomposition
8	Sparse Tensor Derivatives
9	GPU Tensor Kernels

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

Chapter 10. Matrix and Tensor Differentiation

Matrix calculus is the notation and rule system used to differentiate functions whose inputs, outputs, or intermediate values are vectors, matrices, or tensors. Automatic...

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

Tensor Operations

Tensor operations generalize scalar, vector, and matrix operations to arrays with arbitrary rank. In automatic differentiation, a tensor is usually treated as a typed array...

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

Broadcasting Semantics

Broadcasting is the rule system that allows tensor operations between arrays of different shapes without explicitly materializing expanded copies. It is one of the most...

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

Linear Algebra Primitives

Linear algebra primitives are tensor operations with algebraic structure: matrix multiplication, triangular solves, factorizations, inverses, determinants, norms, and spectral...

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

Differentiating Factorizations

Matrix factorizations rewrite a matrix into structured factors. They are used because the factors make later computations cheaper, more stable, or easier to interpret. In...

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

Eigenvalue Problems

Eigenvalue problems are fundamental in numerical analysis, optimization, physics, graph methods, control theory, and machine learning. They are also among the most subtle...

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

Singular Value Decomposition

The singular value decomposition SVD is one of the most important matrix factorizations in numerical linear algebra. It appears in dimensionality reduction, least squares,...

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

Sparse Tensor Derivatives

Most real computational problems are sparse. Large matrices and tensors often contain mostly zeros, structured blocks, or local interactions. Sparse representations reduce...

Writes › Book › Auto Diff › Chapter 10. Matrix and Tensor Differentiation ›

GPU Tensor Kernels

Modern automatic differentiation systems are fundamentally tensor compiler systems. Their performance depends less on mathematical differentiation rules than on how...