GoKawiilturbolite
turbolite is a SQLite VFS in Rust that serves point lookups and joins directly from S3 with sub-250ms cold latency.
It also offers page-level compression (zstd) and encryption (AES-256) for efficiency and security at rests, which can be used separately from S3.
turbolite is experimental. It is new and contains bugs. It may corrupt your data. Please be careful.
Object storage is getting fast. S3 Express One Zone delivers single-digit millisecond GETs and Tigris is also extremely fast. The gap between local disk and cloud storage is shrinking, and turbolite exploits that.
The design and name are inspired by turbopuffer's approach of ruthlessly architecting around cloud storage constraints. The project's initial goal was to beat Neon's 500ms+ cold starts. Goal achieved.
If you have one database per server, use a volume. turbolite explores how to have hundreds or thousands of databases (one per tenant, one per workspace, one per device), don't want a volume for each one, and you're okay with a single write source.
turbolite ships as a Rust library, a SQLite loadable extension ( .so / .dylib ), and language packages for Python and Node.js, plus Github deps for Go. Any S3-compatible storage works (AWS S3, Tigris, R2, MinIO, etc.). It's a standard SQLite VFS operating at the page level, so most SQLite features should work: FTS, R-tree, JSON, WAL mode, etc.
If you want to contribute to turbolite or find bugs, please create a pull request or open an issue.
Performance
... continue reading