GoKawiilAsk about this article… Search
Update from July 15, 2026:
After launch, critics argued that Soofi S was heavily "overtrained" by the standards of the classic Chinchilla scaling laws. Google DeepMind published those laws in 2022, describing how to balance model size and training data for a fixed compute budget. The sweet spot they identified was roughly 20 tokens per parameter. Soofi S blows past that ratio. With about 27 trillion tokens and 30 billion parameters, it lands at several hundred to one. Factor in only the 3.2 billion parameters active per token, and the ratio jumps to several thousand to one.
Michael Fromm, part of the project's technical leadership, pushes back on that criticism. He argues those rules don't simply carry over to Mixture-of-Experts (MoE) architectures. "There's new research showing that the old scaling laws from dense models no longer apply to MoE architectures," Fromm said. The reason comes down to how MoE models are built. Individual experts benefit from seeing the same documents, so repeated data in a large, high-quality dataset is less of a problem than it would be with dense models. As a point of comparison, Fromm points to Nvidia, which trained its own models on up to 25 trillion tokens.
Ad
Original article from July 13, 2026:
Ad DEC_D_Incontent-1
Soofi S is one of the first large language models trained entirely on Deutsche Telekom's Industrial AI Cloud in Munich. The open 30B model uses a lean hybrid architecture and a training mix deliberately weighted toward German.
A German research consortium coordinated by the KI Bundesverband (German AI Association) has released Soofi S 30B-A3B, an open language model that, according to its pretraining report, achieves the highest scores on English and German benchmarks among fully open models, surpassing previous leaders like OLMo 3 32B and Apertus 70B.
Ad
... continue reading