GoKawiil
Tech News
A new Epoch AI report by Emberson and Roodman finds that the price of achieving a given level of AI performance has dropped roughly 47% per quarter over the past three years, a 13-fold annual decline. The report cites OpenAI's o3, which cost about $0.30 per question to score 75% on GPQA Diamond in January 2025, compared with a much newer model reaching similar scores for a fraction of a cent by mid-2026.
A developer has demonstrated Laya, referred to as 'OS Jev', running fully offline on Apple's M4 chip using Apple's CoreML framework. The system reportedly processes 45 decisions per second locally, without needing a cloud connection or internet access.
Crusoe, the AI infrastructure firm behind Oracle's giant Abilene, Texas campus, has closed a $3.9 billion funding round led by Atreides Management, Valor Equity Partners and Mubadala, pushing its valuation to roughly $30.9 billion. The company is using the capital to expand into smaller, factory-built facilities called Spark, designed for AI inference rather than massive training clusters.
OpenJev is a browser-based experiment that runs a local AI model entirely on a user's own GPU, with no backend or waitlist required. It lets users compare how quickly a model can read probability scores for a set of allowed options versus generating the same distribution token by token, using models like MiniCPM5 2B or the lighter Qwen3 0.6B for smaller devices.
Major AI labs have moved their attention from building ever-larger models to running inference—the process of using trained models to generate text, code, and images. This shift is driven by growing real-world use of large language models, the rise of reasoning models that repeatedly reprompt themselves, and autonomous AI agents that run continuously rather than just responding to single queries. Amazon Web Services, for instance, has split inference tasks between its Trainium chips and Cerebras's wafer-scale hardware.
D-Matrix introduced Raptor, a new accelerator that stacks compute logic directly on top of DRAM dies to tackle memory bottlenecks in generative AI inference. The design aims to bridge the gap between SRAM's speed but tiny capacity and HBM's larger capacity but bandwidth and power limitations, following the company's earlier Corsair and JetStream products.
Inception has released Mercury 2.5, an update to its diffusion-based language model that the company says is 40% more intelligent than Mercury 2 while keeping the same low-latency, low-cost performance. The model runs at over 1,100 tokens per second on standard NVIDIA GPUs, supports a 260K-token context window, and launches at an 80% discount, priced at $0.04 per million input tokens and $0.15 per million output tokens.
Qualcomm's stock rose 4% on Tuesday following news that it will supply customized silicon for Amazon Web Services' AI infrastructure, with a focus on inference workloads. As part of the deal, Qualcomm granted Amazon warrants to purchase 25 million shares at $161.26 each, valuing the arrangement at roughly $4 billion.
Gimlet Labs has closed a $300 million Series B funding round led by Andreessen Horowitz, with participation from Sapphire Ventures, Menlo Ventures, Arm, Samsung Ventures, Tiger Global and others. The company says it has added billions in contracted revenue and gigawatts of datacenter pipeline since March, and is scaling toward hundreds of megawatts of managed capacity, just five months after its Series A.
A model named GPT-6 Astra has appeared on OpenRouter, the multi-provider inference marketplace that routes requests across different hosts using modes like Balanced, Nitro, and Exacto. OpenRouter's listing tracks pricing, throughput, latency, uptime, and app usage for the model rather than confirming details about its origin or capabilities.
Tirias Research founder Jim McGregor argues that AI is not one workload but millions of varied ones, meaning data centers can no longer treat memory and storage as secondary hardware. He says inference and agentic AI demand purpose-built systems capable of continuous, real-time data ingestion, caching, and movement rather than legacy infrastructure designed for training workloads.
A technical breakdown describes the 'efficient frontier' concept in LLM inference, distinguishing techniques that shift performance along a fixed tradeoff curve—like sacrificing latency for throughput or intelligence for speed—from techniques that expand the frontier itself, creating more overall efficiency. Examples cited include quantization, distillation, pruning, and reasoning-level adjustments, applied in the context of running large agentic coding models such as GLM-5.3 or Kimi K3 with KV cache reuse.