The Big Idea: A Machine That Predicts the Next Word
When you chat with ChatGPT, Claude, or Gemini, it can feel like you are talking to a very well-read person. But under the hood there is no little person, no database of ready-made answers, and no true understanding. There is a Large Language Model (LLM): a giant neural network trained on enormous amounts of text to do one deceptively simple job — predict the next token (a word or piece of a word) in a sequence. Repeat that prediction thousands of times, and you get fluent paragraphs, poems, translations, and explanations.
By the end of this lesson you will be able to:
- Explain what a Large Language Model is and what "large" refers to.
- Describe tokens and show how a sentence is split into them.
- Explain how repeated next-token prediction produces whole answers, and what temperature does.
- Describe what LLMs are trained on and why their knowledge has a cutoff date.
- Explain the context window and why a chatbot "forgets" old conversations.
- Explain why LLMs sound human without truly understanding or having goals.
1. What Exactly Is a Large Language Model?
A Large Language Model is a neural network — the same kind of layered, adjustable-connection system you met earlier in this course — but scaled up to an extraordinary size and trained on an extraordinary amount of text. Let's unpack the three words in the name:
- Large — the network has billions of internal adjustable numbers, called parameters. These are the "knobs" that training slowly tunes. More parameters generally means the model can capture more patterns of language.