GoKawiil
Tech News
A visual explainer tool called Transformer Explainer illustrates how Transformer-based neural networks work, using the 124-million-parameter GPT-2 (small) model as its example. It walks through core components like tokenization, embeddings, attention mechanisms, and Transformer blocks to show how these models predict the next word in a sequence.
A technical essay argues that describing large language models as mere next-token predictors is outdated once reinforcement learning with verifiable rewards (RLVR) enters the picture. Unlike pre-training, which only reinforces sequences already present in training data, RLVR lets models generate novel token sequences and learn from evaluating their outcomes. The piece walks through pseudocode contrasting the two training loops to show how post-training changes what the model is actually optimizing for.