GoKawiilWhy calling LLMs 'next-token predictors' misses the reinforcement learning step
A technical essay argues that describing large language models as mere next-token predictors is outdated once reinforcement learning with verifiable rewards (RLVR) enters the picture. Unlike pre-training, which only reinforces sequences already present in training data, RLVR lets models generate novel token sequences and learn from evaluating their outcomes. The piece walks through pseudocode contrasting the two training loops to show how post-training changes what the model is actually optimizing for.
GoKawiil's interpretation of the reporting above, not reported fact.
The framing debate matters because it shapes how researchers, policymakers and users reason about what LLMs can do and why they sometimes behave unpredictably. Treating models as simple statistical predictors underestimates the exploratory, outcome-driven learning introduced by RLVR, which is central to how modern chatbots acquire reasoning and task-completion skills beyond pattern-matching text.
- Pre-training only reinforces token sequences that already exist in the training data.
- RLVR post-training lets models explore new sequences and learn from evaluated outcomes, not just imitation.
- The 'next-token predictor' label is an oversimplification that obscures how modern LLMs are actually shaped by reinforcement learning.
Build a Large Language Model (From Scratch) by Sebastian Raschka — If this article made you want to go past the "next-token predictor" slogan, Raschka's book walks you through coding a transformer LLM yourself, from tokenization and attention to pretraining and fine-tuning. Seeing the training loop implemented line by line is the fastest cure for hand-wavy mental models.
See Build a Large Language Model (From Scratch) by Sebastian Raschka on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: gmcgoldr.github.io — Garrin Mcgoldrick, 2026-09-04
Published there as: “Stop Thinking of LLMs as Next-Token Predictors”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.