Cachey
High-performance read-through cache for object storage.
Simple HTTP API
Hybrid memory + disk cache powered by foyer
Designed for caching immutable blobs
Works with any S3-compatible backend, but has its own /fetch API requiring a precise Range
API requiring a precise Fixed page size (16 MiB) – maps requested byte range to page-aligned lookups
Coalesces concurrent requests for the same page
Makes hedged requests to manage tail latency of object storage
Can attempt redundant buckets for a given object
Motivating context
API
Fetching data
Request
HEAD|GET /fetch/{kind}/{object}
kind + object form the cache key
+ form the cache key kind identifies the bucket set (up to 64 chars)
identifies the bucket set (up to 64 chars) object is the S3 object key
Header Required Description Range yes Byte range in format bytes={first}-{last} C0-Bucket no Bucket(s) containing the object C0-Config no Override S3 request config
C0-Bucket behavior:
Multiple headers indicate bucket preference order
If omitted, kind is used as the singular bucket name
is used as the singular bucket name Client preference may be overridden based on internal latency/error stats
At most 2 buckets attempted per page miss
C0-Config overrides: Space-separated key-value pairs to override S3 request configuration per page miss.
ct= Connect timeout (in case an existing connection could not be reused)
Connect timeout (in case an existing connection could not be reused) rt= Read timeout (time-to-first-byte)
Read timeout (time-to-first-byte) ot= Operation timeout (across retries)
Operation timeout (across retries) oat= Operation attempt timeout
Operation attempt timeout ma= Maximum attempts
Maximum attempts ib= Initial backoff duration
Initial backoff duration mb= Maximum backoff duration
Example Request
GET /fetch/prod-videos/movie-2024.mp4 HTTP/1.1 Range: bytes=1048576-18874367 C0-Bucket: us-west-videos C0-Bucket: us-east-videos-backup C0-Config: ct=1000 oat=1500 ma=5 ib=10 mb=100
Response
The service maps requests to 16 MiB page-aligned ranges and the response has standard HTTP semantics ( 206 Partial Content , 404 Not Found etc.)
Header Description Content-Range Actual byte range served Content-Length Number of bytes in response Last-Modified Timestamp from first page Content-Type Always application/octet-stream C0-Status Status for first page
C0-Status format: {first}-{last}; {bucket}; {cached_at}
Byte range and which bucket was used
cached_at is Unix timestamp with 0 implying a cache miss
is Unix timestamp with 0 implying a cache miss Only first page status is sent as a header; status for subsequent pages follows the body as trailers
Example Response
HTTP/1.1 206 Partial Content Content-Range: bytes 1048576-18874367/52428800 Content-Length: 17825792 Content-Type: application/octet-stream C0-Status: 1048576-16777215; us-west-videos; 1704067200 C0-Status: 16777216-18874367; us-west-videos; 0
Monitoring
GET /stats returns throughput stats as JSON for load balancing and health checking.
GET /metrics returns a more comprehensive set of metrics in Prometheus text format.
Command line
Docker images are available.
Usage: server [OPTIONS] Options: --memory Maximum memory to use for cache (e.g., "512MiB", "2GB", "1.5GiB") [default: 4GiB] --disk-path Path to disk cache storage, which may be a directory or block device --disk-kind Kind of disk cache, which may be a file system or block device [default: fs] [possible values: block, fs] --disk-capacity Maximum disk cache capacity (e.g., "100GiB") If not specified, up to 80% of the available space will be used --hedge-quantile Latency quantile for making hedged requests (0.0-1.0, use 0 to disable hedging) [default: 0.99] --tls-self Use a self-signed certificate for TLS --tls-cert Path to the TLS certificate file (e.g., cert.pem) Must be used together with --tls-key --tls-key Path to the private key file (e.g., key.pem) Must be used together with --tls-cert --port Port to listen on [default: 443 if HTTPS configured, otherwise 80 for HTTP] -h, --help Print help -V, --version Print version
Development