GoKawiilOpen-source spinf-serve lets developers run bulk AI scoring on a single GPU
A developer has released spinf-serve, an Apache 2.0-licensed inference server that reads a piece of content once and answers many fixed-choice questions about it in a single pass, rather than generating text. It runs on a single 24GB NVIDIA GPU, offers an OpenAI-compatible API matching the existing hosted spinf service, and supports both open instruction models and dedicated decision models.
GoKawiil's interpretation of the reporting above, not reported fact.
The project grew out of the creator's own need to run a daily market index derived from classifying millions of news articles against dozens of calibrated questions, a task the maker says standard chat-oriented inference engines handle slowly and expensively because they regenerate tokens and re-read content per query. By open-sourcing the engine, the team is enabling others to run similar bulk classification or decision-scoring workloads locally rather than relying solely on the hosted API, which could lower costs for throughput-heavy use cases like content moderation or large-scale labeling.
- spinf-serve returns probabilities over fixed answer sets instead of generated text, aimed at bulk classification tasks.
- It runs on a single 24GB GPU (Ada, Hopper, or Blackwell architectures, with Ampere support via int8) and is released under Apache 2.0.
- It shares its API with the hosted spinf service and was built originally for scoring millions of news articles against many questions per article.
NVIDIA GeForce RTX 4090 24GB GPU — This inference server is built around a single 24GB GPU, and the RTX 4090 is the go-to consumer card hitting that VRAM sweet spot for running bulk classification workloads locally. It gives hobbyists and small teams enough memory headroom to experiment with spinf-serve's batched inference without needing datacenter hardware.
See NVIDIA GeForce RTX 4090 24GB GPU on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: github.com, 2026-10-30
Published there as: “Show HN: Spinf-serve – inference server for bulk scoring and decisions”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.