Tech News
← Home  ·  All topics

Llm Compression

3 GoKawiil briefs on this topic · RSS for llm compression

New quantization method claims 3x LLM compression with under 50MB RAM use

A newly described compression technique reconstructs neural network weight matrices while maintaining cosine similarity above 0.996 and signal-to-noise ratios above 21.5 dB compared to the original uncompressed parameters. The method reportedly preserves performance on reasoning benchmarks like ARC-Challenge within a tight margin of the original model, while requiring less than 50 MB of RAM to run and using a calibration-free approach to identify which weights matter most for normalization layers.

Researchers release LittleBit code for compressing LLMs below 1 bit per weight

A research team has published the implementation of LittleBit, a method accepted at NeurIPS 2025, which compresses large language model weights into a sub-1-bit range, down to 0.1 bits per weight, by factorizing weight matrices into binarized low-rank latent factors with learned scaling. A follow-up method, LittleBit-2 (accepted at ICML 2026), adds an optional initialization technique called Joint Iterative Quantization that aligns latent factors geometrically before training, without changing the model's inference-time structure. The codebase supports models including OPT, Llama, Phi-4, Qwen2.5, QwQ, Gemma 2/3, and Qwen3.

PrismML shrinks Qwen3.8 27B model by 90% while keeping 98% of performance

PrismML, a startup founded by Caltech researchers led by CEO Babak Hassibi, has released Bonsai 2 27B, a compressed version of Alibaba's open-source Qwen3.8 27B model that shrinks it from full size down to 5.9 GB. The compressed model retains about 98% of the original's benchmark performance, up from 95% in the prior Bonsai release, while being small enough to run on PCs and possibly high-end smartphones. The company has raised a $22.25 million seed round and counts Databricks co-founder Ion Stoica as an advisor.