00:00 – Mishaal Rahman: So just to start, can you briefly introduce yourself and the overall mission of Arm’s Edge AI business unit? For our readers who only see Arm on their phone’s spec sheet, but they don’t really fully grasp the business model, where exactly does your team sit in the AI stack?
00:18 – Chris Bergey: Mishaal, thanks for having me. It’s great to be here. Wow, you gave me a very broad first question. So, I do run the Edge AI business unit. We recently consolidated. We had about four business units before and now we’ve reconsolidated around Edge AI, Cloud AI, and physical AI. Obviously, that’s a lot of what I think we’re going to talk about today is the impact of AI and how that’s changing the future of computing. Arm is present in most smartphones and obviously in not just single processors, but processors that are not only running the Android system or the iOS system, but also the Wi-Fi controllers or many other different aspects of computing. That’s just part of Arm’s broad, broad reach. I think to date there’s over 400 billion Arm processors that have been shipped to date and through our various partners and that goes across many different marketplaces from IoT to client computing, data centers, automotive, all those kinds of things.
But what we focus on is, first off, the Arm architecture. Arm has existed for almost 35 years or I guess we just hit our 35th anniversary. Much of that actually came from a low power background. Some of the earliest designs that Apple was, for example, an early investor in Arm back 35 years ago and the Apple Newton, which was a product that I got a chance to play with. Not the most successful product, but clearly a future of what was to come. Products like the Apple Newton, products like the Nintendo DS, were based on Arm. So we have this history of power, low power, and of course that’s come into play in many different markets from IoT to smartphone and computing. Of course, now we see that with AI, the power of computing becomes super important. So we’ve made significant strides in data center as well as in other markets. So I’m focused on, one, we have the architecture, we have a whole set of companies that license the architecture from us that take implementations or RTL (Register-transfer level), which is used in designing chips, and basically takes that RTL, combines it with their system IP and their special sauce and creates chips that get built into so many consumer electronics devices today.
03:08 – Mishaal Rahman: Awesome. You guys, Arm, you’ve been doing your thing for quite a while now. With this recent boom in AI, people kind of think of it as a recent, a new innovation. But actually, AI has been a thing that’s been around for quite a while as well. It didn’t start with 2023 and ChatGPT. It’s been around for a long time. And I think you’ve talked about this in previous interviews. There has been AI before the LLM, before the ChatGPT, before Gemini, before all that. I want to ask you, what impact has Arm had on the pre-LLM era of AI? How have Arm chips been enabling AI before the AI boom as we came to know it today?
03:56 – Chris Bergey: That’s a great question. You’re right. AI and a lot of complex math that we were doing before that wasn’t branded AI clearly was doing a lot of things in multimedia and the like. But I think there’s two key areas. One is the architecture itself. As I mentioned, we have the Arm architecture. We believe it’s the most advanced computing architecture out there. We’ve been very aggressive due to our partnership and our partners in driving future innovation around security as well as around complex math and doing a lot of the AI acceleration. That goes from everything to the way we’re doing vector acceleration to some of our latest products that we’ve announced that actually now have matrix engines in them that actually the CPU cluster that’s made up of several different CPUs is able to leverage this same matrix engine in the cluster to help improve the performance of AI workloads.
What’s unique about that is CPUs are often thought of as general purpose and they are. But they’re also quite easy to program. And so that’s a programming model that a lot of people understand and a lot of people can leverage and is very consistent because of the footprint of Arm computing. So the fact we were able to do this matrix acceleration in the Arm architecture and a programming model that is the standard as you would program any kind of other application, that has been quite unique and quite powerful for us.
Of course, there still are other accelerators that are often in these systems, whether that be a GPU, whether that be a network processing unit. Arm has actually come up with several different NPUs that we actually provide to in the super low power area. One of the ones I like to point to is for your listeners that are familiar with the new Meta glasses that have a display and actually have the neural wristband that actually is starting to take neural sensing of your wristband. That is actually powered by, for example, one of Arm’s latest neural processors as well. So we’ve been in this for a while, as you said, way before 2023 and I guess we really started introducing some of these features actually back in 2017. So it’s been almost a decade.
06:30 – Mishaal Rahman: Wow. And with that early head start on preparing and optimizing for early AI machine learning workloads, what are some of the biggest benefits that you’ve seen from those early optimizations in terms of how you’re preparing Arm chips to run some of the new massively increased demanding workloads we’re seeing from the new LLM era of AI?
06:56 – Chris Bergey: Well, I think that one thing that Arm has done is really set a standard around the way that many chips are architected. And that goes beyond the CPU. Many of Arm’s protocols around how do you build extensions to various multimedia items or to memory systems and all those kinds of things is something that we’ve been working with our partners on in refining and creating a very rich ecosystem of those things. That’s one of the key building blocks right now as you get to AI because obviously the matrix computing and some of that can be super important. We’ve also, I think many of your listeners are aware of the importance of memory in AI and just the size of the model as well as the bandwidth associated with the model. And so the ability for our partners to be able to create these different architectures that can scale, because as I mentioned earlier on, we’ve got solutions that are scaling all the way down to well less than a dollar to super high computing price points. So I think giving the ecosystem those building blocks, that’s one.
The second was just learning from the workloads. We get users, we get workloads that people come to us and say, “Hey, we need tools to do this and can you accelerate that?” And that’s really been a lot of the evolution of the architecture. If you look at areas like our vector extensions, if you look at our matrix engines, something like that, that has been inspired by many of our partner feedback and what software developers have told us is important. We’re quite proud that right now we have a software developer ecosystem of more than 20 million developers, which is probably the largest out there. And so we get a lot of really good feedback on workloads that people are playing with for the future.
... continue reading