Differentiable Physics Engines

A differentiable physics engine computes gradients of physical simulation outputs with respect to inputs, parameters, or control signals. Instead of treating simulation as a...

autodiffbook

Differentiable Physics Engines

A differentiable physics engine computes gradients of physical simulation outputs with respect to inputs, parameters, or control signals. Instead of treating simulation as a terminal forward process, the simulator becomes part of the computational graph.

The system computes:

$$ x_{t+1} = f(x_t, u_t, \theta) $$

where:

Symbol Meaning
$x_t$ System state
$u_t$ Control input
$\theta$ Physical parameters
$f$ Physics transition operator

A loss over trajectories:

$$ L(x_0, x_1, \ldots, x_T) $$

can then be differentiated with respect to:

  • initial conditions
  • controller parameters
  • forces
  • geometry
  • masses
  • friction coefficients
  • material constants

Automatic differentiation propagates gradients backward through the simulation itself.

Physics Simulation as Computation

A classical simulator performs:

initial state
  -> timestep integration
  -> collision handling
  -> constraint solving
  -> next state

repeated over time.

A differentiable simulator augments this with backward propagation:

loss
  -> adjoint propagation
  -> gradients of states and parameters

The simulation becomes an optimization-compatible dynamical system.

Why Differentiable Physics Matters

Many real problems require optimizing physical behavior rather than merely simulating it.

Examples include:

Problem Optimization Target
Robotics Control policy
Animation Motion trajectories
Material design Elastic parameters
Inverse mechanics Hidden forces
System identification Physical constants
Reinforcement learning Long-term reward
Scientific inference Unknown governing parameters
Trajectory optimization Minimal energy or time

Without differentiability, optimization requires black-box search or reinforcement methods with weak gradient signals.

Differentiable simulation provides direct sensitivity information.

Dynamical Systems

A continuous dynamical system is:

$$ \frac{dx}{dt} = g(x, u, \theta) $$

Numerical integration produces discrete updates:

$$ x_{t+1} = x_t + \Delta t \cdot g(x_t, u_t, \theta) $$

for explicit Euler integration.

Automatic differentiation computes:

$$ \frac{\partial L}{\partial \theta} $$

through all timesteps.

This is structurally similar to recurrent neural networks and backpropagation through time.

Simulation Rollout

A rollout is:

$$ x_0 \rightarrow x_1 \rightarrow x_2 \rightarrow \cdots \rightarrow x_T $$

The total loss may be:

$$ L = \sum_{t=0}^{T} \ell(x_t, u_t) $$

Reverse mode computes adjoints backward:

$$ \lambda_t = \frac{\partial L}{\partial x_t} $$

using:

$$ \lambda_t = \lambda_{t+1} \frac{\partial x_{t+1}}{\partial x_t} + \frac{\partial \ell_t}{\partial x_t} $$

This propagates future consequences backward through physical evolution.

Example: Projectile Optimization

Suppose a projectile state is:

$$ x_t = (p_t, v_t) $$

with updates:

$$ p_{t+1} = p_t + \Delta t , v_t $$

$$ v_{t+1} = v_t + \Delta t , g $$

A target loss:

$$ L = |p_T - p^\star|^2 $$

can be differentiated with respect to initial velocity:

$$ \frac{\partial L}{\partial v_0} $$

allowing gradient-based optimization of launch conditions.

System Identification

A differentiable simulator can infer unknown physical parameters.

Suppose:

$$ x_{t+1} = f(x_t; \theta) $$

and observations are:

$$ y_t $$

The objective:

$$ L = \sum_t |x_t(\theta) - y_t|^2 $$

optimizes:

  • mass
  • damping
  • stiffness
  • drag
  • friction
  • elasticity

This is inverse physics.

Rigid Body Simulation

Rigid body systems evolve according to:

$$ M(q)\ddot{q} + C(q,\dot{q}) + G(q) = \tau $$

where:

Symbol Meaning
$q$ Generalized coordinates
$M$ Mass matrix
$C$ Coriolis and centrifugal forces
$G$ Gravity
$\tau$ Applied forces

Differentiable rigid body simulation computes gradients through:

  • forward dynamics
  • integration
  • constraint resolution
  • contact forces

This enables optimization of robot motion and control.

Contact and Collision

Collision handling is one of the hardest parts of differentiable physics.

Classical contact is discontinuous:

if penetration:
    apply impulse

Tiny perturbations may abruptly create or remove contacts.

This causes unstable gradients.

Approaches include:

Method Strategy
Soft contact Smooth penetration forces
Penalty methods Continuous force approximation
Implicit differentiation Differentiate solver equilibrium
Relaxed collision models Approximate contact manifolds
Event-aware gradients Handle discontinuities explicitly

Contact dynamics remain a major research problem.

Soft Body Simulation

Soft materials deform continuously.

A deformation field:

$$ u(x) $$

describes displacement.

Energy-based models often define:

$$ E(u; \theta) $$

The dynamics minimize or evolve according to this energy.

Differentiable soft body simulation computes gradients through:

  • elastic deformation
  • stress computation
  • finite element assembly
  • constitutive models

Applications include:

  • cloth simulation
  • biological tissue
  • deformable robotics
  • material optimization

Fluid Simulation

Fluid systems satisfy equations such as Navier-Stokes:

$$ \frac{\partial v}{\partial t} + (v \cdot \nabla)v = -\nabla p + \nu \nabla^2 v + f $$

Differentiable fluid simulation computes sensitivities of flow behavior.

This supports:

Application Goal
Aerodynamic optimization Minimize drag
Flow control Stabilize turbulence
Weather inference Estimate hidden parameters
Shape optimization Improve flow geometry
Scientific calibration Match observations

Large-scale fluid differentiation is computationally expensive because state dimensionality is enormous.

Differentiable Constraints

Many simulators solve constrained systems:

$$ g(x) = 0 $$

Examples:

  • joints
  • incompressibility
  • nonpenetration
  • conservation laws

Constraint solvers may involve iterative optimization.

Differentiation through solvers can occur via:

Method Idea
Unrolled differentiation Differentiate every iteration
Implicit differentiation Differentiate equilibrium conditions
Custom adjoints Manually derived backward rules

Implicit differentiation is often more memory efficient.

Implicit Differentiation

Suppose the simulator solves:

$$ F(x^\star, \theta) = 0 $$

Differentiating gives:

$$ \frac{\partial F}{\partial x} \frac{dx^\star}{d\theta} + \frac{\partial F}{\partial \theta} = 0 $$

Therefore:

$$ \frac{dx^\star}{d\theta} = - \left( \frac{\partial F}{\partial x} \right)^{-1} \frac{\partial F}{\partial \theta} $$

This avoids storing all iterative solver states.

Differentiable Controllers

A controller may generate forces:

$$ u_t = \pi(x_t; \phi) $$

where $\phi$ are policy parameters.

The simulator defines dynamics:

$$ x_{t+1} = f(x_t, u_t) $$

The full system becomes:

state
  -> controller
  -> forces
  -> physics
  -> next state

Gradients flow through both controller and physics.

This enables trajectory optimization and model-based reinforcement learning.

Model Predictive Control

Differentiable simulation is central in model predictive control.

The controller solves:

$$ \min_{u_{0:T}} \sum_t \ell(x_t, u_t) $$

subject to dynamics:

$$ x_{t+1} = f(x_t, u_t) $$

Differentiability allows efficient optimization of control sequences using gradient methods.

Differentiable Robotics

Modern robotics pipelines increasingly combine:

policy network
  -> physics simulator
  -> trajectory
  -> reward loss

Gradients may optimize:

  • motor torques
  • gait parameters
  • grasp trajectories
  • balance policies
  • actuator calibration

Differentiable simulation often improves sample efficiency relative to black-box reinforcement learning.

Learned Physics

Some systems replace analytic physics with learned approximations:

$$ x_{t+1} = f_\theta(x_t, u_t) $$

The learned model may represent:

  • fluid dynamics
  • rigid body interactions
  • cloth deformation
  • contact dynamics

These systems trade exact physical guarantees for differentiability and efficiency.

Hybrid systems are common:

analytic physics
  + learned residual model

Adjoint Methods

Large scientific simulators often use adjoint methods instead of general-purpose AD.

An adjoint system propagates sensitivities backward through time:

$$ \lambda_t = \left( \frac{\partial f}{\partial x_t} \right)^T \lambda_{t+1} + \frac{\partial \ell_t}{\partial x_t} $$

Adjoint methods are mathematically equivalent to reverse-mode differentiation but optimized for PDE-scale computation.

Numerical Stability

Differentiating long simulations introduces instability.

Common problems include:

Problem Cause
Exploding gradients Chaotic dynamics
Vanishing gradients Dissipative systems
Contact instability Discontinuous forces
Integrator sensitivity Numerical discretization
Solver divergence Ill-conditioned constraints
Floating point drift Long trajectory accumulation

Chaotic systems may have gradients that become meaningless over long horizons.

Memory Complexity

Reverse-mode differentiation stores intermediate states:

$$ x_0, x_1, \ldots, x_T $$

Large simulations may require terabytes of memory.

Techniques include:

Method Purpose
Checkpointing Recompute states during backward pass
Reversible integration Recover earlier states
Implicit adjoints Avoid full storage
Truncated gradients Limit backward horizon
Mixed precision Reduce memory cost

Memory management is often the dominant systems problem.

Parallel and GPU Simulation

Differentiable simulators increasingly target GPUs.

Challenges include:

  • parallel contact handling
  • differentiable sparse solvers
  • dynamic topology
  • irregular memory access
  • synchronization costs

Modern simulators integrate tightly with tensor runtimes such as CUDA-based computation graphs.

Hybrid Symbolic-Continuous Systems

Practical physics engines often combine:

Component Type
Collision detection Symbolic
Constraint graph Symbolic
Dynamics integration Numerical
Material models Continuous
Controllers Learned
Contact relaxation Approximate differentiable

Fully smooth physics engines are rare. Most systems selectively introduce differentiability where optimization benefits justify the complexity.

Failure Modes

Differentiable simulators fail in recognizable ways.

Failure Cause
Contact explosions Unstable collision gradients
Energy drift Integration error
Unrealistic optimization Exploiting simulator artifacts
Gradient chaos Sensitive dynamics
Solver nonconvergence Ill-conditioned systems
Soft-contact artifacts Unrealistic penetration
Overfitting to simulator Reality gap

Optimization can exploit weaknesses in the simulator rather than discovering physically meaningful behavior.

Systems Architecture

A differentiable physics engine typically contains:

Component Purpose
State representation Physical configuration
Integrator Time evolution
Constraint solver Physical validity
Contact system Collision response
Tensor backend GPU acceleration
Differentiation runtime Backward propagation
Adjoint solver Efficient sensitivity computation
Optimization loop Parameter updates

At scale, simulator design becomes inseparable from differentiation strategy.

Relation to Automatic Differentiation

Differentiable physics extends automatic differentiation into dynamical systems.

The simulator becomes a recurrent differentiable program:

$$ x_{t+1} = f(x_t, u_t, \theta) $$

Automatic differentiation propagates gradients through:

  • integration
  • constraints
  • forces
  • contacts
  • material response
  • control systems

The central challenge is not differentiating elementary arithmetic. The challenge is preserving meaningful gradients through long, nonlinear, often discontinuous physical evolution.

Core Idea

A differentiable physics engine transforms simulation into an optimization-compatible process. Physical trajectories become differentiable computations whose parameters can be learned, inferred, or controlled through gradient-based methods.

This allows observations, rewards, or objectives defined at the end of a simulation to shape the physical parameters and control signals that generated the trajectory.