GoKawiilDeveloper reimplements DeepSeek V3 architecture in 190 lines of pure JAX
A Show HN project recreates the DeepSeek-V3 model architecture—including multi-head latent attention, decoupled RoPE, DeepSeekMoE routing and multi-token prediction—in roughly 190 lines of pure JAX with no Flax, Equinox or PyTorch dependencies. The author pretrained a 36.2M-parameter version from scratch on a MacBook Air M5 using public-domain detective fiction, reaching a validation loss of 3.51 after 12,000 training steps over about 2.6 hours.