RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning (2023)

@ Conference on Robot Learning (CoRL) 2023 @inproceedings{robopianist2023, author = {Zakka, Kevin and Wu, Philipp and Smith, Laura and Gileadi, Nimrod and Howell, Taylor and Peng, Xue Bin and Singh, Sumeet and Tassa, Yuval and Florence, Pete and Zeng, Andy and Abbeel, Pieter}, title = {RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning}, booktitle = {Conference on Robot Learning (CoRL)}, year = {2023}, } TLDR We train anthropomorphic robot hands to play the piano using deep RL and release a simulated benchmark and dataset to advance high-dimensional control. This is a demo of our simulated piano playing agent trained with reinforcement learning. It runs MuJoCo natively in your browser thanks to WebAssembly . You can use your mouse to interact with it, for example by dragging down the piano keys to generate sound or pushing the hands to perturb them. The controls section in the top right corner can be used to change songs and the simulation section to pause or res ... Read full article.