1.1.1: How language models generate text by predicting the next word
Demystifying AI: A Practical Introduction for Instructors · LibreTexts
0%
Contents
Text
Size
From LibreTexts, under CC BY 4.0, as the source states it; its equations are written in TeX, which the PDF shows typeset. This page at LibreTexts.
1.1.1: How language models generate text by predicting the next word
Opening the book…
Not Understanding—Just Guessing Well
At the heart of language models like ChatGPT is a surprisingly simple mechanism: prediction. When you give the AI a prompt, it doesn’t “think” about what to say next—it uses probability to guess which word is most likely to come next based on everything it has seen before.
This process happens one word at a time. And with each new word, the AI recalculates what the next most likely word should be. It’s like high-speed autocomplete, operating on a much larger scale.
🧠 Example: Prediction in Action
Prompt:
“The mitochondria is the…”
The AI predicts:
“…powerhouse of the cell.”
Why? Because those words often appear together in its training data. The model has seen that phrase repeated so many times in textbooks and online content that it becomes a highly likely sequence.
🔁 It’s All About Probability
Here’s what’s happening under the hood:
The AI breaks down your input into small units (called tokens).
It looks at the sequence and assigns a probability to every possible next token.
It picks the one with the highest probability.
Then it repeats that process again. And again. And again.
The result? A response that often sounds fluent and accurate—even though the AI has no idea what it's saying.
🎓 Why This Matters for Instructors
AI doesn’t “know” facts—it knows what looks like a fact.
It can sound confident but be wrong, especially with unfamiliar or complex topics.
The fluency of the text can trick students (and instructors) into over-trusting the content.
Understanding this prediction-based process helps instructors evaluate AI output more critically—and teach students to do the same.
✅ Quick Reflection
Were you surprised to learn that AI is just predicting the next word, not understanding meaning? Think about how this might affect the way you evaluate AI-generated content—or how your students might interpret something that sounds fluent but isn't necessarily accurate.