GoKawiilFor the past couple of years, almost every major AI launch has followed the same script: a bigger model, better reasoning, a longer context window, smoother conversation. However, this month a startup called TypeSafe AI went the other way. It also matters that the founder of that startup, Diogo Almeida, worked at OpenAI on the instruction-following research (known as RLHF) that turned raw language models into ChatGPT.
Almeida's startup spent two years working in stealth mode and just recently received $40 million in funding.
What he's showcasing now is the company's first model, called "Jev." Unlike GPT and other chatbots, Jev can't write an email. It won't try to explain itself, and it has no chat window. What it does is make fast decisions, in about a tenth of a second, for a fraction of a cent. Almeida argues that chat was never the right tool for automation.
The Problem With Talking Computers
Large language models do one thing at their core: they generate text, one token at a time, each word conditioned on the last. That's great when a human is on the other end. But with AI "agents" taking over, this seems to be a poor fit when the reader is another piece of software. That's the conundrum at this stage of AI: whether we like it or not, it's happening, and a growing share of it involves software talking to other software rather than to us.
But software doesn't want a paragraph. It wants a yes or no, a number, or a category. Today, developers who put AI inside an app ask an LLM a question and wait while it writes out an answer. Then they parse that text, validate it, and retry when the model goes off script. Every one of those words also costs money, since output tokens usually cost several times more than input.
Here is a 45-second illustration from developer Matija Sosic that captures the problem well...
An app asks whether an invoice is fraudulent. The LLM spends 8+ seconds typing out a sentence concluding the invoice looks legitimate. Jev instead answers in 0.1 seconds with three numbers: 7% fraud, 88% clean, 5% needs review. It gave up writing to get the speed. That trade is the whole product.
I'm not an AI expert and I'm looking at this as an interested outsider, trying to understand the basics and explain why it matters. Whether Jev turns out to be the future (or any other AI model, for that matter) or not, novel technology like this is what drives our interest as tech enthusiasts anyway.
What Jev Actually Does
... continue reading