GoKawiilIndependent developer trains 3.8B-parameter LLM for under $1,000 on rented B200 GPUs
Hugo Vergnes built a config-driven training framework called little-lm and used it to train a 3.8-billion-parameter language model from scratch on 65 billion tokens, taking 43 hours on eight rented B200 GPUs at a total cost of $998. The model scored 0.384 on the CORE benchmark, outperforming Andrej Karpathy's nanochat d32 model, which cost about the same to train but scored 0.310.
GoKawiil's interpretation of the reporting above, not reported fact.
The project shows that meaningful, non-trivial language models can now be trained by a single person on a modest budget, without access to a research lab's compute resources. It also suggests that the price-to-performance ratio for training frontier-adjacent models keeps improving as hardware and techniques mature, lowering the barrier for independent researchers and small teams.
- A 3.8B-parameter model was trained for $998 in 43 hours using 8 rented B200 GPUs.
- It scored 0.384 on the CORE benchmark, beating a similarly priced nanochat d32 model that scored 0.310.
- The project used a custom config-driven framework called little-lm, emphasizing modular, swappable components for efficient experimentation.
NVIDIA GeForce RTX 5090 Founders Edition — The author debugged this whole from-scratch LLM run on a 5090 before renting cloud B200s — it's the local workstation card that makes iterating on training code at home realistic. With its huge memory pool and Blackwell tensor cores, it's the natural centerpiece for anyone wanting to watch language emerge from random weights on their own desk.
See NVIDIA GeForce RTX 5090 Founders Edition on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: hugovergnes.github.io — Hugo Vergnes, 2026-09-10
Published there as: “Training a 3.8B LLM to 0.384 CORE for $998 – Hugo Vergnes”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.