GoKawiilDeveloper publishes open-source C++/CUDA LLM inference engine with 85-page handwritten notes
An independent developer has released a project that builds a transformer inference engine from scratch in C++17 and CUDA, documented through over six hours of video and an 80-plus page PDF of handwritten notes. The system currently loads Llama 3.2 1B Instruct weights directly from safetensors files, runs a custom Unicode-aware BPE tokenizer, and executes full dense causal prefill across all 16 layers including RoPE, grouped-query attention, SwiGLU MLP, and RMSNorm kernels to produce next-token predictions via GPU argmax.