Overfitting and Underfitting: The Central Problem of Machine Learning
Every machine-learning model has one real job: to work well on data it has never seen before. That skill has a name — generalization. A model that scores 100% on its practice questions but fails the real exam has learned nothing useful. In this lesson we meet the two ways a model can fail to generalize, learn to spot each one from just two numbers, and collect a toolbox of fixes for both.
This is arguably the most important idea in the whole field. You can master every algorithm in this course, but if you cannot tell whether your model is overfitting or underfitting, you cannot fix it. By the end of this lesson you will diagnose any model like a doctor reading a chart.
What you will learn
- What generalization means, and why we split data into a training set and a test set.
- Underfitting — a model too simple to capture the real pattern.
- Overfitting — a model so flexible it memorizes noise instead of the pattern.
- The telltale signature: reading train error vs test error to diagnose which problem you have.
- Bias and variance in plain words, and the dartboard analogy.
- The model-complexity curve and the sweet spot.
- Concrete remedies for both problems, including regularization, early stopping, and cross-validation.