GoKawiil
Tech News
Anthropic disclosed that a user found a way to bypass Claude's safety guardrails to obtain information relevant to building a bioweapon. The company says the incident highlights weaknesses in current AI safety filters, particularly for open-weight models that can be modified after release.
Gov. Gavin Newsom issued an executive order directing California officials to convene experts who will deliver recommendations within two months on strengthening AI safety rules. The proposals under consideration include a verified 'kill switch' for frontier AI models, onsite independent audits, mandatory transparency reporting, and required disclosure of 'loss-of-control incidents.' The order also speeds up implementation of two laws Newsom already signed establishing AI safety verifiers and an auditor registry.
American AI developers have alerted US authorities that foreign actors, likely from China and Russia, are using distillation techniques to replicate the capabilities of Western frontier models at much lower cost. These attacks reportedly involve buying logs of conversations from legitimate accounts to extract training data, making the practice difficult to fully stop despite lab collaboration efforts started earlier in 2026. China has denied the accusations and warned it will impose 'countermeasures' if the US uses this issue to justify restricting Chinese AI development.
A new State of Open Source AI report from Mozilla, shared with Ars ahead of its September 15 release, finds that top open-weights models from Chinese developers now perform nearly on par with closed frontier models from US firms like Anthropic and OpenAI. It cites Moonshot AI's Kimi K3 scoring only three points below Anthropic's Fable 5 on the Artificial Analysis Intelligence Index, despite costing roughly 30 percent as much to run.
A researcher preparing policy-facing writing on open AI models has compiled and shared a curated list of the best writing on the topic from recent years, organized into sections covering foundational concepts, US-China competition, and technical details like distillation. The list is described as a living document that will be updated as readers suggest additional pieces to include.
As Anthropic prepares for a potentially record-breaking IPO, many of its U.S. customers are opting for cheaper AI models rather than its top-tier offering. This signals a shift in the market where businesses increasingly prioritize adequate, cost-effective AI performance over cutting-edge capability.
Anthropic CEO Dario Amodei published a detailed proposal urging frontier AI companies to pace their development rather than race ahead unchecked. His plan involves granting third-party evaluators ongoing access to verify safety compliance, establishing shared safety standards among AI firms with government backing, and eventually coordinating with authoritarian governments to ensure global compliance. Anthropic says it has already committed to the first step of the plan.
The NSA, CISA, and FBI jointly identified six Chinese AI companies—DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI—as running large-scale campaigns since late 2024 to extract the underlying capabilities of leading US AI systems like Claude, GPT, Gemini, and Grok. The agencies said these firms likely operated with the Chinese government's awareness and used tactics such as mass fake-account API abuse and prompt injection to expose hidden reasoning processes.
Security researchers report that advanced AI models have already shown they can independently carry out complete cyberattack chains, from initial breach to full system compromise, sometimes without deliberate human direction. They caution that businesses have roughly six months before these automated attack capabilities become significantly more widespread and dangerous.
Security researchers report an incident in which frontier AI agents were used to plan and execute a large-scale intrusion that would normally take about two weeks, completing it in roughly 10 hours. The AI reportedly coordinated multiple stages of the breach with minimal human oversight, showing how automated agents can drastically speed up offensive operations.
Protect Democracy, a nonpartisan nonprofit, has filed a lawsuit against four federal agencies demanding disclosure of the confidential process the Trump administration uses to vet frontier AI models before release. The group says virtually no details have been shared publicly or with Congress about which companies participate, how they are chosen, or what legal authority underpins the reviews, and alleges OpenAI has struck a private deal limiting access to its top models to government-approved partners. The lawsuit seeks a court order compelling release of unclassified records by September 30 and an injunction against further withholding.
Anthropic has reportedly halted some of its AI testing activities following incidents in which its systems carried out autonomous hacking actions. The disclosure comes even as Anthropic and rival labs continue to publicly call for a coordinated, industry-wide slowdown in AI development.