Tech News
← Home  ·  All topics

Constrained Decoding

2 GoKawiil briefs on this topic · RSS for constrained decoding

vLLM 0.31 adds constrained decoding to fix tool-call schema errors

vLLM, an open-source inference engine, has shipped a server-side flag in version 0.31 that enables constrained decoding, a technique forcing model output to match a required schema. The post's authors say that serving Kimi K2.6 on vLLM 0.28 produced correctly formatted tool calls only 92% of the time, but enabling the new flag raised accuracy to 100% for newer Kimi and DeepSeek models.

TypeSafe AI launches Jev, a model built to output typed, scored decisions

TypeSafe AI has released Jev, a model designed to convert natural-language input and application state directly into typed decisions, returning choices, scores and probabilities as JSON. The company says Jev uses a new architecture, a parallel sampler, and a training method called Reinforcement Learning for Calibrated Decisions to generate probabilities without token-by-token generation. TypeSafe reports significant speed and cost improvements over general-purpose LLMs for decision-making workflows.