Classification: Teaching a Computer to Sort Things into Groups
So far in this course you have mostly seen regression — predicting a number, like the price of a flat in Bishkek. Classification is the other giant half of supervised learning: instead of a number, you predict a category — a label from a fixed list of choices.
Is this email spam or not spam? Is this photo a cat, a dog, or a horse? Will this student pass or fail the exam? Every one of these questions has an answer that is a name (a class), not a measurement. In this lesson you meet two of the most important beginner classifiers ever invented: k-Nearest Neighbours (k-NN) and Logistic Regression. They are simple enough to compute by hand, yet they teach the ideas behind almost every classifier you will ever use.
What you will be able to do after this lesson
- Explain what classification is and tell binary from multiclass problems.
- Classify a brand-new point by hand with k-NN, using a distance table you compute yourself.
- Say what the number k changes, and why k-NN needs its features scaled.
- Turn a linear score into a probability with the sigmoid function, and use the 0.5 threshold (and know when to move it).
- Picture both methods as a decision boundary drawn across a scatter plot.
- Tell lazy learners from eager ones, and run both classifiers in
scikit-learn.
What Exactly Is Classification?
A classifier is a function that takes some features and returns a class label. Features are just the numbers or facts you measured about the thing: the weight of a fruit, the number of links in an email, the hours a student studied. The label is the answer you want: apple or orange, spam or ham, pass or fail.