Embeddings: Representing Meaning as Vectors
A neural network can only add and multiply numbers. So how does it "understand" the word cat, or notice that Bishkek is a city like Astana? The trick is to turn every word into a list of numbers — a vector — arranged so that words with similar meaning land near each other. This one idea, called an embedding, quietly powers search engines, recommendations, translation and every large language model you have ever used. This is the bridge lesson: it connects the raw arithmetic of networks to the messy world of human meaning.
What you will learn
- Why a network needs meaning to become numbers before it can compute with it.
- Why the two obvious tricks — token IDs and one-hot vectors — both fail, shown with real distances worked by hand.
- The embedding idea: a dense vector per word so that similar meaning = nearby vector.
- Where those numbers come from — they are learned from the company a word keeps, not typed in by hand.
- The famous vector arithmetic: king − man + woman ≈ queen, computed step by step.
- Cosine similarity: why direction matters more than length, with a tiny calculation.
- Contextual embeddings from transformers, plus sentence and document embeddings.
- Real applications that tie the whole course together, and the limitations you must respect.