The Case for Hybrid Cache for Object Stores
How RisingWave built Foyer, a hybrid caching library in Rust to beat S3 latency Yingjun Wu 9 min read · 3 days ago 3 days ago -- 1 Listen Share
Press enter or click to view image in full size
Foyer GitHub repo: https://github.com/foyer-rs/foyer RisingWave GitHub repo: https://github.com/risingwavelabs/risingwave
S3 Is Slow; You Need a Cache
In recent years, many modern data systems have been designed around Amazon S3 or other object storage services. Projects such as Neon, TiKV, and Warpstream all treat S3 as their primary storage layer. This trend is not accidental. Object storage offers practically unlimited capacity, strong durability, and very low cost compared to block storage or SSDs. For database developers, it also removes the burden of managing disks directly and allows them to focus on building features while relying on the cloud provider for durability and scalability. With these advantages, S3 has become the de facto backbone for new database and data infrastructure projects.
However, S3 is not built for low-latency access. Every read requires a network round trip, and every request adds cost. Latency is already orders of magnitude higher than memory or local SSD, and when applications repeatedly fetch the same data these delays accumulate, making systems sluggish while inflating storage bills. Many systems that rely solely on S3 face this constant tradeoff between low cost and acceptable performance.
Press enter or click to view image in full size Object storage offers low cost, durability, and scalability, but its high latency and per-access cost become major bottlenecks, especially when the working set is large and requires frequent data exchange with the cache. Image Source: the author.
For RisingWave, the challenge is even greater. As a streaming system, it executes continuous queries where results depend on processing data in real time. A single S3 access may already add hundreds of milliseconds. If multiple S3 reads are needed within one second, the delays compound rapidly until they become unacceptable. This extreme sensitivity to latency is why we felt the pain of S3 more acutely than others.
A cache provides a way to close this gap. By keeping hot data in memory and warm data on local disk, a cache reduces the number of requests sent to S3. This lowers latency, stabilizes performance, and reduces recurring storage costs. Applications no longer need to fetch the same data repeatedly from remote storage but can instead serve most queries locally while still relying on S3 for durability and scale. To make this practical for real-time systems like RisingWave, we built Foyer, a hybrid caching library in Rust that unifies memory and disk caching into one coherent layer.
... continue reading