Decision Trees and Random Forests
A decision tree is a machine that learns a flowchart of yes/no questions straight from your data. A random forest is a whole crowd of those flowcharts voting together. In this lesson you will build one by hand, watch it decide, understand exactly how it chooses its questions, and see why a crowd of trees beats a single one.
Imagine a shopkeeper in Osh deciding whether a person walking in will buy an expensive phone. She does not solve equations in her head. She asks small questions: "Is this person young or old? Do they look like they have money to spend?" Each answer sends her down a different mental path, and at the end of the path she makes a guess: buy or not buy.
A decision tree is exactly this, written down as a diagram and learned automatically from past data. It is one of the most human-readable models in all of machine learning: you can literally print it out and follow the arrows. A random forest takes many such trees, each slightly different, and lets them vote. The vote is usually more accurate and far more stable than any single tree.
- Read a decision tree as a flowchart and trace one example from the root down to a leaf.
- Explain how a tree chooses its questions by making groups purer, and compute a simple Gini impurity by hand.
- Understand depth, leaves, and why an unlimited tree memorises (overfits) the training data.
- List the real strengths (no scaling, mixed features, readable) and weaknesses (unstable, overfits) of a single tree.
- Explain how a random forest uses randomness and voting to fix those weaknesses, and read its feature importance.
- Decide when to reach for trees/forests versus a linear model, and write the sklearn code for both.