GoKawiilAmong the many things we humans like to lord over the rest of the animal kingdom is our complex language. Sure, other creatures talk to one another, but we’ve got all these wildly complicated written languages with syntax and fun words like defenestrate. This we can also lord over robots, who, in addition to lacking emotion and the ability to not fall on their faces, can’t write novels.
Atsunobu Kotani
At least not yet. Researchers at Brown University just got a robot to do something as linguistically improbable as it is beautiful: After training to hand-write Japanese characters, the robot then turned around and started to copy words in a slew of other languages it’d never written before, including Hindi, Greek, and English, just by looking at examples of that handwriting. Not only that, it could do English in print and cursive. Oh, and then it copied a drawing of the Mona Lisa on its own for good measure.
Like walking on two legs, handwriting is one of those seemingly simple human charms that is in fact elaborate. When you write a word, you have to know where to put down your pen, how long to draw a line and in which direction, then pick up your pen, sometimes mid-letter (like with a capital A), and know where to put it down again.
Matt Simon covers cannabis, robots, and climate science for WIRED.
So to get a kid to write, you can’t just show them a sample and set them loose—you have to give them instructions on how to form each letter. “They give you these little algorithms for what strokes to do and what order to put them in to make the character,” says Brown University roboticist Stefanie Tellex, who developed the system with Atsunobu Kotani, also at Brown. “And that's what our algorithm is learning to do.”
Their learning system is split into two distinct models. A “local” model is in charge of what’s going on with the current stroke of the pen—so aiming in the right direction and determining how to end the stroke. And a “global” model is in charge of moving the robot’s writing utensil to the next stroke of the character.
To train the robot, the researchers fed it a corpus of Japanese characters, and provided information about how the component strokes of a character are supposed to work. “From that, it basically learns a model that looks at pixels of the image and predicts where it needs to go to start the next stroke, and then where it needs to move while it’s drawing the stroke to reproduce the image,” Tellex says.
LEARN MORE The WIRED Guide to Robots
Then they decided to try to confuse the hell out of the robot by writing hello on a whiteboard in Hindi and Tamil and Yiddish—all of which use unique scripts. Incredibly, the robot could eyeball each with machine vision and write its own copies of the words, even though it only ever trained to write Japanese. Also, they showed it English cursive in addition to print, and it handled both fine.
... continue reading