GoKawiilYouTube System Design for Robotics Data Infrastructure Kingston Kuan 2026-07-20
When we first started building a data platform for robotics, we weren’t thinking about YouTube.
Robotics teams wanted to be able to upload their LeRobot datasets, search across demonstrations, inspect synchronized camera feeds and robot states, and export curated episodes for training. That sounded like a fairly specialized problem in robotics. So we built Pareto.
Once we got into the infrastructure work, the requirements started to feel familiar. We needed to reliably upload large media files, preserve the originals, process them in the background, generate thumbnails and preview videos, index their metadata, and stream them to a browser without downloading the entire dataset.
“Design YouTube” is a classic system-design exercise. It can feel a little pointless: YouTube dominates, and almost none of us will ever need to build a global consumer video platform at that scale. If we reframe thousands of users uploading single videos to your platform as a single robotics dataset uploading thousands of videos, the similarities become much clearer.
ByteByteGo scopes the exercise to upload and playback: original files go to blob storage, background workers transcode them, a completion flow updates metadata, and the CDN delivers the finished versions. That is the simplified shape above.
An episode of robot data
The classic video platform starts with one dominant object: a video, perhaps with audio and maybe subtitles. A robot-learning dataset has a more complicated unit of data.
An episode of robot data is a multimodal record: it can contain several camera feeds (wrists, top, chest, etc.) alongside joint positions, actions, force readings, timestamps, task descriptions, and other sensor streams. The cameras may be the largest part of the dataset, but they are not useful in isolation. The data only makes sense when it remains connected to what the robot was sensing and doing at that moment.
Pareto searches across episodes, so someone searching for an “orange block” can scrub the surrounding trajectory, compare cameras, and decide whether a demonstration belongs in a training set.
... continue reading