GoKawiilOllaya launches as local runtime for open-source decision models, mimics TypeSafe API
Ollaya is a new local server for running open-source 'decision models' like Laya, gliclass and NLI, designed to answer structured questions in a single forward pass rather than generating text token by token. It implements the same /v1/systemone and /v1/models endpoints as TypeSafe's hosted API, so the official TypeSafe Python SDK can point at a local Ollaya server without code changes.
GoKawiil's interpretation of the reporting above, not reported fact.
By reproducing TypeSafe's request and response shapes, Ollaya could let developers switch from a hosted decision-model API to local, self-hosted inference with minimal code changes, keeping sensitive data like tickets and user messages on their own infrastructure. The published benchmarks, comparing an RTX 4090 setup against third-party measurements of TypeSafe's hosted API, suggest large latency gains, though the project itself cautions that the two setups differ enough that the comparison is only order-of-magnitude.
- Ollaya runs open-source decision models locally with single-pass inference instead of token-by-token generation.
- It is drop-in compatible with TypeSafe's hosted API, working with the existing Python SDK via a local base URL.
- Benchmarks show local models like Laya responding in single-digit to low-double-digit milliseconds versus 236–276ms for TypeSafe's hosted API, though the comparison is not apples-to-apples.
NVIDIA GeForce RTX 4090 GPU — This benchmark's impressive 8-10ms inference times were measured on an RTX 4090, making it a great reference point for anyone wanting to run local decision models like Laya at similar speeds. Its large VRAM and compute power make it well suited for hosting fp16/fp32 inference workloads locally instead of relying on hosted APIs.
See NVIDIA GeForce RTX 4090 GPU on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: ollaya.dev, 2026-09-25
Published there as: “Ollaya – Ollama for open-source, Jev-style decision models”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.