At a certain point between building Apple’s developer tools, leading a core part of Google’s AI infrastructure team, and clashing with Elon Musk during a stint as Tesla’s Autopilot chief, Chris Lattner’s vision for his life’s work started to come into focus. AI was taking over the world, and demand was growing for the chips that powered it. But the software stack for those chips was dominated by just a few big companies. Would developers be able to easily run their code across all the different chips dotting the AI landscape?
Lattner’s answer to that question is Modular, a software startup he founded in 2022 with his former Google colleague Tim Davis. Modular makes a unifying software layer that helps cloud businesses squeeze as much juice as possible out of GPUs and CPUs—the high-powered chips that underpin generative AI. The startup has also built a new coding language, based on Python, that lets developers use a single language to build AI apps that run across multiple GPUs and CPUs. Modular’s basic premise is that if a developer builds an app for one chip, they shouldn’t have to jump through hoops in order to run it on another vendor’s chip.
But Modular’s long-term goal is even more ambitious: to loosen the software choke hold that companies like Nvidia and AMD have on the industry, and become the de facto software for AI chips.
“Our thesis is that the need for compute power is just exploding, but there is no unified compute platform,” Lattner says. “Sovereign AI will be everywhere. There will be many Stargates. But there will be different types of chips optimized for different use cases, and there needs to be a unified layer for that.”
There are early signs that Modular’s thesis bears out. AI giants like Nvidia, AMD, and Amazon have partnered with the startup to test the waters. The GPU cluster company SF Compute also worked with Modular to build what they claim is the world’s cheapest API for large AI models. As of this week, Modular’s developer platform now supports Apple Silicon GPUs, in addition to Nvidia and AMD chips.
Building on this momentum, Modular just raised $250 million in venture capital funding, its third round of financing in three years, bringing its total valuation to $1.6 billion. The round was led by the Pittsburgh-based US Innovative Technology Fund. DFJ Growth also invested, along with existing investors General Catalyst, Greylock, and GV (formerly known as Google Ventures).
“We’ve spent a bunch of time and energy trying to figure out what makes a startup in this space interesting, and with every company that has tried to build their own chip—and even the big players, like AMD and Nvidia—it all comes back to the software,” says Dave Munichiello, managing partner at GV. “Chris convinced me that the software was the most interesting and valuable problem to address.”
It might be valuable—but it’s also extremely complicated. Part of that complication stems from Nvidia’s closed ecosystem. Nvidia’s chips make up the vast majority of the GPU market, but the company’s 20-year-old proprietary software platform, CUDA, keeps developers locked in. AMD’s software platform for high-performance computing, called ROCm, differs in that it’s open source. This allows developers to more easily move code to different chips.