Multi-Layer Networks and Forward Propagation
One neuron can only draw a single straight line through your data. The magic of deep learning begins when we stack many neurons into layers and let the signal flow forward from the inputs, through hidden layers, to a final answer. In this lesson you will build a tiny network by hand and push real numbers through it, step by step, until it outputs a probability.
A single neuron takes some numbers, multiplies each by a weight, adds them up, adds a bias, and squashes the result. That is powerful but limited. Real problems — recognising a handwritten digit, telling a cat from a dog, predicting a house price — are too tangled for one straight decision line.
The fix is almost silly in its simplicity: use lots of neurons, arranged in rows called layers, and feed the output of one row into the next row. Each layer transforms the data a little, and by the time the signal reaches the end, the network has reshaped a messy problem into an easy one. Moving numbers through the network from input to output is called forward propagation (or a "forward pass"), and it is the single most important process you will learn today.
- Name the three kinds of layer (input, hidden, output) and say what each one holds.
- Explain in plain words what hidden layers actually learn, and why depth often beats width.
- Count every single weight and bias in a small network and connect that to "models with billions of parameters".
- Run a complete forward pass by hand on a 2-3-2 network: weighted sums, ReLU, softmax, final probabilities.
- See how the whole thing is really one big matrix multiply — the reason GPUs are so good at it.
- Choose the right output layer for regression, binary classification, or multiclass classification.