Forward and Backward Pass

A tiny network: each neuron is a weighted sum Σ followed by an activation σ. Step forward to compute the loss, then step backward to see the chain rule at work.

Hidden layers Neurons per layer Activation x₁ x₂ target y

Blue: values (forward). Orange: derivatives of the loss (backward). Edge colour: sign of the weight; thickness: its size. Click a weight or a bias in the tables to turn that knob.

Forward and Backward Pass

What happens at this step

Turn one knob

Parameters (value, and ∂L/∂parameter once the backward pass reaches it)