LemonSeed Studio is an iPad app that embeds the unmodified upstream Linux amdgpu and amdkfd driver as a PCIDriverKit extension, allowing it to control an AMD GPU connected through a Thunderbolt enclosure. The app also bundles the LemonSeed Engine for on-device LLM inference running on that GPU via its HSA runtime, plus amdgpu_mtopg for live GPU telemetry. The project's source is available via git with recursive submodules, and is dual-licensed under MIT or GPL-2.0-only.
github.com
· 2026-10-11
Prime Labs has released a Rust rewrite of its Prime Agent coding tool, which it says has been downloaded over 300,000 times and processed more than 8 trillion tokens since its August launch. The company says the rewrite was carried out over two weeks by a swarm of more than 2,000 AI subagents running across over 10,000 Prime Sandboxes, consuming over 200 billion tokens via its Prime Inference GLM-5.3 endpoint.
primeintellect.ai
· 2026-10-09
Google launched EmbeddingGemma 2, a follow-up to its original EmbeddingGemma model that reportedly surpassed 20 million downloads. The new 740-million-parameter model is built on the Gemma 4 architecture, released under an Apache 2.0 license, and extends beyond text to embed code, images, video, and audio in a single shared space for on-device inference.
blog.google
· 2026-10-06
Cerebras Systems, a key player in inference-focused AI chips, successfully launched its IPO on Nasdaq, raising $5.55 billion and achieving a valuation of approximately $23 billion. The company's early backing by the author, who invested when it was valued at around $2 billion, underscores its prominence in the hardware infrastructure for running AI models efficiently at scale.
entrepreneur.com
· 2026-10-05
Salvatore Sanfilippo, the creator of Redis, has released DwarfStar 4 (ds4), a lightweight C-based inference engine for running large open-weight language models locally on high-memory Mac, CUDA and ROCm hardware. The tool supports models including DeepSeek V4, V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next, covering text and vision, with a local API, CLI and native agent, and is released under the MIT license.
dwarfstar.sh
· 2026-10-02
A development team set out to run an entire month of coding work on the GLM 5.3 Flash model, keeping within a $68 budget for the first two weeks. The second half of the month saw usage spike to 1 billion tokens across other models, driven by an expensive vibe-coded Wagtail MCP server prototype that alone cost $150 and 5kWh, plus infrastructure capacity issues that forced switches to DeepSeek V4.1 Flash and Qwen 3.8 Flash.
wagtail.org
· 2026-10-02
A Show HN project called Jevstiller introduces a local CPU model trained on outputs from the Jev classification API, aiming to answer most requests in about 15 milliseconds instead of the roughly 300 milliseconds a network call to Jev takes. Users set a target agreement rate (e.g. 98%), and the system routes only enough traffic to the local model to keep its combined coverage and error rate within that budget, sending the rest to Jev.
jevstiller.pages.dev
· 2026-09-29
AI inference infrastructure provider Modal Labs is close to finalizing a $750 million funding round led by Accel, valuing the company at $15.75 billion including the new capital, according to a source familiar with the deal. That figure more than triples Modal's $4.65 billion valuation from a $355 million raise just four months earlier. Modal declined to comment on the report.
techcrunch.com
· 2026-09-28
An open-source project splits a 0.5-billion-parameter 1.58-bit BitNet language model across seven ESP32S3 microcontrollers connected via SPI daisy-chain. One master unit handles tokenization, embeddings and final sampling, while six compute nodes each process four transformer blocks using 1.58-bit attention and MLP layers with KV caching in PSRAM.
github.com
· 2026-09-28
MicroLLM Lab is a browser-based tool that lets users load and run up to seven small language models, including ones as small as 135M parameters, directly in the browser. It runs objective test suites using regex and exact-token checks rather than judging writing quality, and measures metrics like peak speed, sustained decode speed across 256 tokens, and total benchmark wall time.
stateofutopia.com
· 2026-09-28
OpenAI's VP of Hardware, Richard Ho, spoke with Tom's Hardware following the August 2026 Hot Chips reveal of the company's Jalapeño inference ASIC. Ho said the chip's development leaned heavily on AI tools to compress the design timeline, and that efficiency, not raw performance, was the primary driver behind building custom silicon.
tomshardware.com
· 2026-09-28
A new Epoch AI report by Emberson and Roodman finds that the price of achieving a given level of AI performance has dropped roughly 47% per quarter over the past three years, a 13-fold annual decline. The report cites OpenAI's o3, which cost about $0.30 per question to score 75% on GPQA Diamond in January 2025, compared with a much newer model reaching similar scores for a fraction of a cent by mid-2026.
marginalrevolution.com
· 2026-09-23