How Neural Networks Actually Learn

The forward pass

A neural network makes a prediction by passing input data through layers of weighted connections, each applying a simple transformation, until an output emerges.

Measuring and assigning blame

A loss function scores how wrong that output was. Backpropagation then works backward through the network, computing how much each individual weight contributed to the error.

Nudging toward better

Gradient descent uses that information to nudge every weight slightly in the direction that would have reduced the error, repeated over millions of examples until the network's predictions converge on something useful.