GoKawiilOn Tuesday, AI infrastructure company Runware announced the launch of its own modular data center called Sonic Inference Pod. Designed as a single transportable unit, the Pod represents a more flexible kind of compute that can sit alongside hyperscalers’ massive data center projects.
Runware says the Pod can offer inference at a higher quality but lower cost than other serverless inference platforms and GPU clouds. The modular design means it’s easy add capacity quickly by creating new pods rather than having to expand a fixed data center. In some ways, this is the future, Flaviu Radulescu, co-founder and CEO of Runware, told TechCrunch.
“We believe distributed compute, positioned closer to end users for faster inference, is what will win in the long term,” he said, noting his company as an example. Aside from a lower price, Radulescu noted that the runware system can scale and add capacity fast, deploy anywhere there is power, and adapt quickly to new hardware releases. The Runware pods also do not use water, but rather a closed-loop cooling system that can be built in days, compared to the months or even years it takes to build traditional data centers.
“Demand for inference is growing faster than facilities can be built,” Radulescu said. “What we want is to power the world’s intelligence, to be the backbone every AI model runs on with capacity that keeps up with demand instead of throttling it.”
Runware currently has 10 pods in deployment across the U.S., Europe, and Asia-Pacific, Radulescu said. The company already provides inference to a few companies, including Higgsfield AI and Wix, and has 160 sites available to power its pods right now. Runware announced a $50 million Series A in December to provide the infrastructure needed for companies to generate images. They see the expansion into pods as part of the company’s core mission: providing inference to companies, rather than a single product.
Image Credits:Runware
AI labs like OpenAI and SpaceX are still racing to build data centers throughout the U.S. OpenAI, for example, is close to striking a $500 billion deal that would see it build a data center in Ohio, according to reports. But Radulescu doesn’t see those projects as a threat to the Sonic Inference Pods, describing the flexibility of the pods as a key differentiator.
“Every pod runs as part of a single network, so requests go wherever there’s capacity, closer to the users, and if one pod goes offline, traffic moves to another,” he said, adding that a system failure means one pod is down rather than a whole fixed facility. “Customers who want dedicated hardware get whole pods to themselves.”
He’s also not too worried about other companies building this for themselves, saying simply that hardware is slow and finding the talent pool to build and fix this technology is small.
“A mistake in a circuit board design costs months between redesign, simulation, fabrication, testing and delivery,” he said. “Every one of those calls needs someone who understands exactly what each component does and what breaks if it’s gone.”
... continue reading