GoKawiilRun huge mixture-of-experts models from a swarm of peers, with the colibri engine. Pure C, no dependencies.
One machine shares a model. Any other machine chats with it: nothing is downloaded up front, the bytes an inference actually touches arrive from the peer on first use and stay in a local mirror. The second question is served from local disk at full speed. The engine binary is unmodified.
The founding principle: any machine may join, GPU or not. The engine was built for CPU and SSD first; a GPU only makes it faster, never different, and the output is byte-identical either way. A swarm with no GPU at all is a working swarm. Networks that pool GPUs recruit from the few; lumabri recruits from everyone.
Quick start
Build:
make
On the machine that has a model (any colibri model directory):
./lumabri serve --model /path/to/model
On the machine that wants to chat (needs a colibri build for the engine):
./lumabri chat --tracker < server-ip > :7300 --engines-dir /path/to/colibri/c
... continue reading