Icml - GoKawiil

Researchers release LittleBit code for compressing LLMs below 1 bit per weight

A research team has published the implementation of LittleBit, a method accepted at NeurIPS 2025, which compresses large language model weights into a sub-1-bit range, down to 0.1 bits per weight, by factorizing weight matrices into binarized low-rank latent factors with learned scaling. A follow-up method, LittleBit-2 (accepted at ICML 2026), adds an optional initialization technique called Joint Iterative Quantization that aligns latent factors geometrically before training, without changing the model's inference-time structure. The codebase supports models including OPT, Llama, Phi-4, Qwen2.5, QwQ, Gemma 2/3, and Qwen3.

github.com · 2026-10-08

Amazon researchers unveil Ising-model method to fix correlated LLM judge panels

Amazon Science researchers, together with Shiva Kasiviswanathan, presented a new paper at ICML that tackles a flaw in using multiple LLM judges to evaluate outputs like retrieval passages: when judges share prompts, training data, or model lineage, their agreement can look stronger than it actually is. Their method models correlations between judges using Ising models and reweights votes to account for shared blind spots rather than treating every vote as independent evidence.

amazon.science · 2026-09-14