Colossus for Rapid Storage

The above image shows the steps that are taken to append data to a file. The application opens the file in append mode. The Colossus Curator constructs a handle and sends it to the Colossus Client running in-process, which caches the handle. The application issues a write call for an arbitrary-sized log entry to the Colossus Client. The Colossus Client, using the disk addresses in the handle, writes the log entry in parallel to all the disks. Rapid Storage builds on Colossus’s stateful protocol, leveraging gRPC-based streaming for the underlying transport. When performing low-latency reads and writes to Rapid Storage objects, the Cloud Storage client establishes a stream, providing the same request parameters used in Cloud Storage’s REST protocols, such as the bucket and object name. Further, all the time-consuming Cloud Storage operations such as user authorization and metadata accesses are front-loaded and performed at stream creation time, so subsequent read and write operations g ... Read full article.