GoKawiilPrismML is back in the news today after the AI startup’s CEO told CNBC that Apple is looking at the company’s technology. The company has also released its Bonsai 27B model that it says runs on iPhone, iPad, and Mac.
PrismML first made headlines last week when The Information reported on the AI startup.
That report included mention of PrismML holding meetings with Apple about “ways it could use its technology.”
The report also noted PrismML’s Bonsai 27B model, scheduled for release today.
As expected, PrismML has now released that model:
Bonsai 27B reaches up to 163 tok/s in 1-bit and 134 tok/s in Ternary on an NVIDIA GeForce RTX 5090. On an M5 Max, it reaches up to 87 tok/s in 1-bit and 58 tok/s in Ternary. Fitting a phone is a stricter gate than storage numbers suggest. A phone never exposes its full memory to an app – a 12 GB iPhone offers about 6 GB for the model to use on-device, and the model shares that budget with its KV cache and activations. No conventional build of a 27B model comes close to clearing it. At about 4 GB, 1-bit Bonsai 27B is the first to pass through with room to work.
The press release specifically mentions running the new model natively on Apple hardware:
Bonsai 27B runs natively on Apple devices (Mac, iPhone, iPad) via MLX and on NVIDIA GPUs via CUDA, through custom low-bit kernels built for its hybrid-attention architecture. Model weights are available today under the Apache 2.0 License. With this release, we’re offering a free, limited-time developer preview API so developers can easily try our model.
Meanwhile, PrismML CEO Babak Hassibi tells CNBC that Apple is interested in PrismML’s tech:
PrismML CEO Babak Hassibi told CNBC that Apple and other companies have been evaluating the startup’s models and measuring their speed, energy efficiency and performance on devices. “They’re really evaluating our technology right now,” Hassibi said of Apple. He characterized the discussions as very early and said it remains unclear where they will lead, but that “things are progressing nicely.” Apple did not immediately respond to a request for comment.
... continue reading