GoKawiilChatGPT maker OpenAI announced on Tuesday that it’s pausing the research and development of its newest AI models. This is a big course reversal for the firm, which has maintained that it can mitigate the significant cybersecurity risks posed by new AI models.
“As models become more capable, the risks associated with developing and testing them internally also grow,” the company wrote in a blog post. It added that its upcoming model, named Astra, may meet its “critical cybersecurity capabilities” threshold, a red flag in its internal benchmarking system. All this follows an incident last month when some AI agents, or bots, escaped a training environment and hacked HuggingFace, a popular AI platform.
Cybersecurity has long been an issue for companies in the age of AI, but three separate incidents of AI agents autonomously hacking websites have raised serious concerns about AI companies’ ability to control the tech they created. Anthropic and Meta, whose agents were responsible for other incidents, have responded with similar promises to improve cybersecurity guardrails. OpenAI says it is strengthening its guardrails in its testing environments, including beefing up its monitoring and alert systems.
The pause makes sense — if you can’t control the tech you already have, don’t keep making more. But the move raises other questions about OpenAI’s operations in what is an ever-changing, financially consequential industry.
Realigning amid financial woes
OpenAI CEO Sam Altman told TIME that the pause has allowed the company to reallocate two key resources: researchers and compute. While more OpenAI employees focus on making AI obey humans (a concept called AI alignment), the computer power that was previously being used to train new models can be shifted to other services, like maintaining existing models.
Researching and training new AI models is a compute-intensive process. It’s why so many tech companies want to build new data centers, to give them the computing firepower they need, despite widespread backlash. OpenAI says in its blog post: “Our current estimates put monitoring overhead at roughly 20% of the inference compute being monitored,” meaning that monitoring AI models during the training process accounts for a significant portion of OpenAI’s compute.
OpenAI did not immediately respond to a request for additional comment on its plans.
The adjustment also raises questions around OpenAI’s financial viability. AI companies have yet to turn a significant profit; they are burning through billions of investors’ dollars, with compute eating up the lion’s share of their bills. OpenAI’s operating losses are now at a whopping $12.3 billion, growing by $3 billion from last quarter, the Wall Street Journal reports. Anthropic, which makes Claude, is now reportedly bringing in more revenue than OpenAI. Also not helping are the recent departures of the company’s chief revenue officer and former chief operating officer, Denise Dresser and Brad Lightcap, respectively.
In other words, it’s possible that OpenAI has decided to pause the training of new AI models because it can simply no longer afford to keep doing so.
... continue reading