GoKawiil
Tech News
Cloudflare re-tuned the hash-mapping algorithm its Pingora framework uses to route URL requests to cache servers, drastically reducing the number of hash entries needed per server. The optimization freed up roughly 100 terabytes of RAM across its infrastructure, marking a second major memory-saving effort by the company.
Cloudflare engineers found that its internal load-balancing service, Pingora Backend Router, was consuming far more memory than necessary in its pingora-ketama consistent-hashing library. By reworking the underlying algorithm with better math and Rust code, the team eliminated the excess memory usage across its global fleet.
Cloudflare built a prototype called Cache Transcoding, developed during an intern project, that compresses eligible cached assets with Zstandard before writing them to disk inside its Pingora proxy. Assets stay compressed while stored and while moving between data centers via Tiered Cache, then get decoded only when served to the client. Early tests show this shrinks eligible assets to about a third of their original size with only a small CPU overhead.