Earlier this week, Bloomberg reported that Google and Apple are close to reaching a yearly $1 billion agreement for a version of the Gemini model that will power the revamped Siri next year.
But perhaps more interesting than the price tag is one factor that will actually affect everyone’s experience: its architecture. Here’s a look at how it will likely work.
Is 1.2 trillion parameters a lot?
According to Bloomberg’s report, Google will provide Apple with a 1.2 trillion parameter model, which will live on Apple’s Private Cloud Compute servers, effectively preempting Google from accessing any of it. Privacy-wise, that’s great.
Size-wise, a 1.2 trillion parameter model is nothing to sneeze at. However, a direct comparison with the latest and greatest competing models is quite challenging.
That’s because in recent years, closed frontier AI labs like OpenAI, Anthropic, and Google have stopped disclosing the parameter counts of their latest flagship models. This has led to wildly varying speculation as to the true parameter count of offerings such as GPT-5, Gemini 2.5 Pro, and Claude Sonnet 4.5. Some put them below a trillion parameters, while others suggest they reach a few trillion. In reality, nobody really knows.
On the other hand, one thing most of these huge latest models have in common is an underlying architecture known as mixture of experts (MoE). In fact, Apple already employs a flavor of MoE on its current cloud-based model, which is rumored to have 150 billion parameters.
Siri’s Gemini-powered model will likely use a mixture of experts
In a nutshell, MoE is a technique that structures a model with multiple specialized sub-networks called ‘experts.’ For each input, only a few relevant experts are activated, which results in a faster and more computationally efficient model.
In other words, this allows MoE models to have very high parameter counts, while keeping inference costs much lower than if 100% of their parameters had to be activated for every input.
... continue reading