GoKawiilA complete guide to what quantization is, how it works, and how it's used to compress large language models
Annie Sexton is a Developer Educator at ngrok with a passion for nerd-sniping developers. She also has over a decade of experience working at PaaS companies such as Heroku, Render, and Fly.io.
I was reading about compression recently when I stumbled upon something crazy: that compressors and LLMs are, at their core, trying to solve the exact same problem.
In this post, I’m going to walk us through the basics of compression to understand its deep relationship with language modeling. It’s probably going to blow your mind.
Bookmark this section How compression works
There are many ways of shrinking data. Take minification, for example: it works by stripping code down to the bare minimum that machines need to parse. Human-readable variables are reduced to single letters; whitespace and comments are removed.
Click “Minify” to see it in action:
sum-numbers.js // Sum every number in the list function sumNumbers ( numbers ) { let total = 0 ; for ( const number of numbers ) { total += number ; } return total ; } Original source, 156 characters: // Sum every number in the list function sumNumbers(numbers) { let total = 0; for (const number of numbers) { total += number; } return total; } Minified to 62 characters — 60 percent smaller — by removing the comment, shortening the variable names to single letters, and stripping the whitespace, braces, and semicolons. Minify Start over
The resulting file is considerably smaller, and yet you’d almost never hear minification mentioned in the field of data compression. Why is that?
Minification is fairly straightforward: it just tosses out any syntax that’s not required by machines. But “true” compression relies on redundancy to condense data.
... continue reading