Why This Matters
This is a playful, experimental project that repurposes a language model (Jev) into a character-by-character chatbot by having it predict one symbol at a time rather than generating full responses. It matters as a lighthearted illustration of how LLM sampling mechanics work under the hood, and as a showcase of AI-assisted coding (using Claude to implement the sampling logic) rather than a practical or efficient chat solution.
Key Takeaways
- The project reimplements chat generation as symbol-by-symbol sampling, asking the model to predict one character at a time from a defined alphabet until a stop symbol is drawn.
- It was built as an 'AI-accelerated' side project, with Claude implementing the sampling strategies and alphabets described by the developer, highlighting a growing workflow pattern of AI-assisted coding.
- The tool is explicitly impractical and costly to run in this fashion, positioned as an educational and entertaining novelty rather than a production-ready chatbot.
jevchat
We know Jev.
jevchat turns that into a chat model. At every step it asks Jev one question:
Given the user's question and the reply written so far, which symbol comes next?
The options are an alphabet plus an option to stop emitting. Jev returns a probability for each one, and the sampler draws the next symbol from that normalised distribution. Append, repeat, and stop when STOP is drawn.
There are several alphabets (including truncated token lists) and sampling strategies available.
The idea is for fun, the cost is somewhat impractical, and the results are hilarious.
This was a Claude accelerated experiment. I described the sampling algorithms, strategies, and so on, and it implemented them.
Setup
poetry install
... continue reading
GoKawiil