Preparing Data: Cleaning, Scaling and Splitting
A machine learning model is only as good as the data you feed it. Real data arrives messy: with holes, typos, repeated rows, wildly different number ranges, and text where the model expects numbers. Before any algorithm can learn, you have to turn that raw mess into clean, numeric, fairly-measured tables and then hide part of it away for an honest test.
People sometimes call this the boring part. It is not. In real projects, data preparation is usually 60–80% of the whole job, and it is the single biggest lever on how well your model performs. A brilliant algorithm on bad data loses to a simple algorithm on well-prepared data almost every time. This lesson is your complete guide to that preparation.
The big idea: Raw data is never model-ready. Your job is to clean it, encode the text into numbers, scale the numbers so no single column shouts louder than the others, and split it into a part to learn from and a part to be tested on — in that order, without ever letting the test part leak into the training part.
What you will be able to do
- Explain why raw data is never ready for a model.
- Handle missing values by dropping or filling with the mean, median, or mode — and know the trade-offs of each.
- Remove duplicate rows, fix wrong data types, and tame outliers.
- Turn categories (text) into numbers with label encoding and one-hot encoding.
- Put every number on a fair scale using min–max normalisation and standardisation, computing the numbers by hand.
- Split data into train and test sets and say clearly why you must never grade a model on data it studied.
- Spot and avoid data leakage — the silent mistake that fools you into trusting a bad model.