Running GPT-2 in WebGL: Rediscovering the Lost Art of GPU Shader Programming

May, 24 2025 • 16 min read • 2461 words Preface: A few weeks back, I implemented GPT-2 using WebGL and shaders (Github Repo) which made the front page of Hacker News (discussion). By popular demand, here is a short write-up over the main ideas behind GPU shader programming (for general-purpose computing). The Origins of General Purpose GPU Programming In the early 2000s, NVIDIA introduced programmable shaders with the GeForce 3 (2001) and GeForce FX (2003). Instead of being limited to predetermined transformations and effects of earlier GPUs, developers were now given unprecedented control over the rendering pipeline, enabling much more sophisticated visual effects. These programmable shaders laid the foundation for modern GPU computing. Researchers soon discovered that certain computations (like linear algebra involving matrices and vectors) could be accelerated by casting them as graphics operations on the GPU. However, using shader languages like OpenGL’s GLSL for no-graphics ta ... Read full article.