GoKawiilWebLLM brings OpenAI-compatible LLM inference directly into web browsers via WebGPU
WebLLM is an open-source engine that runs large language models entirely inside a web browser, using WebGPU for hardware acceleration instead of relying on a server. It offers full compatibility with the OpenAI API, including streaming, JSON-mode structured output, and support for models like Llama 3, Phi 3, Gemma, Mistral, and Qwen. Developers can install it as an npm package and build applications on top of it, or integrate custom models in MLC format.