GoKawiilStartup claims new pretraining recipe cuts compute 10x vs open-weight models
A research team reports developing a pretraining method that matches DeepSeek V4 Pro Base's performance using roughly 50 times less compute, at an estimated cost of about $0.5 million on GB200 chips. Scaling the same recipe up 10x further, to roughly $4 million, reportedly surpassed all publicly available open base models on perplexity benchmarks.
GoKawiil's interpretation of the reporting above, not reported fact.
Frontier-scale pretraining has largely been the domain of labs with massive chip clusters, but this result suggests algorithmic efficiency gains could let smaller teams compete without hundred-million-dollar budgets. If the scaling trend holds, it could shift who is able to build state-of-the-art base models and accelerate progress toward agentic coding systems and automated AI research.
- The new recipe matches DeepSeek V4 Pro Base using about 50x fewer FLOPs.
- Scaling 10x further outperformed all public open-weight base models on perplexity evals.
- The team frames pretraining, agentic RL, and long-context work as the path to superhuman coding agents.
NVIDIA GeForce RTX 4090 Founders Edition GPU — If reading about trillion-parameter pretraining makes you want to train something yourself, the RTX 4090 is the go-to desktop card for local model experimentation and fine-tuning. Its large VRAM pool and CUDA support make it a workhorse for PyTorch workloads at home.
See NVIDIA GeForce RTX 4090 Founders Edition GPU on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: magic.dev, 2026-09-08
Published there as: “>10x More Efficient Pretraining”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.