Android is about to take another AI leap over Apple

Rushil Agrawal / Android Authority TL;DR Future Android processors will support SME2 for faster machine learning running directly on the CPU. The improvements look set to appear with Arm’s upcoming mobile CPU core. Developers using Arm’s KleidiAI won’t have to change anything to benefit. Despite its naysayers, AI features remain the cornerstone of modern smartphone innovation. But powering the latest and greatest AI tools quickly and efficiently requires processors that are up to the task, whether a dedicated AI accelerator or a CPU equipped with special instructions to accelerate machine learning workloads. In a blog post, Arm detailed more about what to expect from its Scalable Matrix Extension 2 (SME2) CPU extension and teased that the trick will be coming to future Android smartphones whenever it drops its next-gen mobile CPUs. Based on previous announcements, this will likely happen in the next few months and will be rebranded under Lumex rather than the old Cortex brand. For a quick overview, SME, originally part of the Armv9 architecture, is a set of optional, advanced CPU architecture extensions explicitly designed to accelerate matrix math operations — the type of complex multiplication that machine learning algorithms rely on. SME2, which actually debuted around the end of 2022, builds on these features, which, according to Arm, enable real-time mobile inference tasks, from image and natural language processing to voice generation. SME2 speeds up AI workloads on your CPU, and it's coming soon to future Android phones. Arm also shares some numbers, which certainly make SME2 look like the real deal. According to Arm, Google’s Gemma 3 model delivers 6x faster AI responses with SME2-enhanced hardware than without. It can run a text summary of 800 words in under one second on just a single CPU core, though it’s unclear which CPU Arm refers to here. That all sounds super promising for future text summary and smart reply features, which could feel vastly more responsive than today’s implementations. In any case, the key part of the announcement for consumers is confirmation that SME2 hardware capabilities are “coming soon” to Android smartphones. Server-grade features for mobile Tushar Mehta / Android Authority So far, SME has mostly been reserved for server and workstation-class processors due to its demanding nature. However, SME2 is designed to be more scalable, enabling deployment in lower-power tiers like laptops, tablets, and soon, high-end smartphones. While Android chipsets have mostly skipped the original SME generation, Apple’s M4 chip — currently used in iPads — is the closest mobile device with SME2 support. That said, Apple has yet to bring this feature to its iPhone line. This opens the door for next-gen Android devices to gain a significant AI performance advantage over their competitors. Importantly, Android is already set up for SME2 support. SME2 is enabled in Google’s XNNPACK library for Android and is supported across multiple frameworks like llama.cpp, Alibaba’s MNN, and Microsoft’s ONNX. Likewise, developers already using Arm’s KleidiAI software library (which integrates with these frameworks) will automatically take advantage of SME2 hardware once it becomes available in Android smartphones while retaining backward compatibility with SME and NEON extensions. That’s a big thumbs up for easy adoption. Apple has SME for iPads, but not for iPhones. Android stands to leap ahead. In fairness, we already knew that SME was coming to future mobile Arm CPUs courtesy of Arm’s Chris Bergey at Computex. A presentation slide revealed that Arm’s next-gen TRAVIS CPU will feature SME (presumably SME2, based on this latest announcement). This core has been rumored to power the MediaTek Dimensity 9500 and potentially other next-gen mobile SoCs such as Samsung’s Exynos line. Google’s Tensor often lags several generations behind, but it might adopt Arm’s SME capabilities by 2026’s Tensor G6. However, with Qualcomm now going down the custom CPU core route, the upcoming Snapdragon 8 Elite 2 might not share the same credentials, meaning that a large portion of next-gen Android flagships won’t jump to SME2 right away. According to early rumors, the 8 Elite 2 will support SME1 and SVE2 extensions, which will still be a notable upgrade on he current setup but won’t feature the same SME2 capabilities as Arm’s in-house Lumex cores. Whether Apple silicon will catch up with its own SME implementation in the next generation remains to be seen. It’s possible, as A18 chips are already on Armv9.2, but it’ll need to bring the SME2 hardware adopted from the M4 into its custom phone CPU cores. Ryan Haines / Android Authority In any case, faster machine learning running on your smartphone’s CPU is a big deal. Better performance for a range of tasks, such as text summarization, on-device translation, and image recognition, is the obvious benefit for end users, but it’s also a positive move for the AI development ecosystem as a whole. Despite the availability of Android NN, coding specifically for a SoC’s bespoke machine learning hardware remains a chore for smaller developers, who are more likely to resort to running on the CPU to reach as many devices as possible. Handsets sporting more powerful instructions available directly on the CPU will be a boon for these use cases. If that turns out to be the case, it’s another big win for next-gen Android smartphones, while its biggest rival, Apple, already remains stuck in the AI starting blocks.