GoKawiilpicchio One Python file that measures local LLMs: effective bits per weight, the three tok/s lanes, and silent CPU fallback. Install · Commands · Quant · Lanes · Measured · Examples
Most GPU speed claims are one tok/s number. That number can be correct and still tell you the wrong story. Three failure modes, each one command:
Four quantizations of the same Qwen3.5-9B, all labeled Q4_K_M, measure 5.02, 5.02, 5.07 and 5.27 bits per weight (the quant label).
Losing the GPU cost prefill 22x and decode under 2x on the same model and file (three lanes).
The 36 tok/s I remembered from bare llama.cpp reproduced in no cell of a 32 cell matrix (silent CPU fallback).
picchio splits prefill, decode and wallclock, reads the engine's log against the OS's GPU meter, and prints a verdict that says whether the GPU did the work, and why.
Install
curl -fsSLO https://raw.githubusercontent.com/logxio/picchio/main/picchio.py python3 picchio.py
With no arguments it finds your models (ollama tags, the current folder, the HF and LM Studio caches) and runs the one you pick. A .gguf path gets the full llama.cpp diagnosis; an ollama tag gets measurement mode.
Needs python3 and either llama.cpp or ollama. Three passes with a fixed prompt, the first one cold. About a minute here with the GPU engaged, a few minutes on CPU. It writes one cache file under ~/.cache/picchio and nothing else.
... continue reading