Expression-matching robot will haunt your dreams but someday it might be your only friend

Most of the best robots, ones that can walk, run, climb steps, and do parkour, do not have faces, and there may be a good reason for that. If any of them did have mugs like the one on this new research robot, we’d likely stop in our tracks in front of them, staring wordlessly as they ran right over us.

Building robots with faces and the ability to mimic human expressions is an ongoing fascination in the robotics research world but, even though it might take less battery power and fewer load-bearing motors to make it work, the bar is much much higher for a robot smile than it is for a robot jump.

Even so, Columbia Engineering’s development of its newest robot, Emo and “Human-robot Facial Co-Expression” is impressive and important work. In a recently published scientific paper and YouTube video, researchers describe their work and demonstrate Emo’s ability to make eye contact and instantly imitate and replicate human expression.

To say that the robot’s series of human-like expressions are eerie would be an understatement. Like so many robot faces of its generation, its head shape, eyes, and silicon skin all resemble a human face but not enough to avoid the dreaded uncanny valley.

That’s okay, because the point of Emo is not to put a talking robot head in your home today. This is about programming, testing, and learning … and maybe getting an expressive robot in your home in the future.

Emo’s eyes are equipped with two high-resolution cameras that let it make “eye contact” and, using one of its algorithms, watch you and predict your facial expressions.

Because human interaction often involves modeling, meaning that we often unconsciously imitate the movements and expressions of those we interact with (cross your arms in a group and gradually watch everyone else cross their arms), Emo uses its second model to mimic the facial expression it predicted.

“By observing subtle changes in a human face, the robot could predict an approaching smile 839 milliseconds before the human smiled and adjust its face to smile simultaneously.” write the researchers in their paper.

In the video, Emo’s expressions change as rapidly as the researcher’s. No one would claim that its smile looks like a normal, human smile, that its look of sadness isn’t cringeworthy, or its look of surprise isn’t haunting, but its 26 under-the-skin actuators get pretty close to delivering recognizable human expression.

“I think that predicting human facial expressions represents a big step forward in the field of human-robot interaction. Traditionally, robots have not been designed to consider humans,” said Columbia PhD Candidate, Yuhang Hu, in the video.

How Emo learned about human expressions is even more fascinating. To understand how its own face and motors work, the researchers put Emo in front of a camera and let it make any facial expression it wanted. This taught Emo the connection between its motor movements and the resulting expressions.

They also trained the AI on real human expressions. The combination of these training methods gets Emo about as close to instantaneous human expression as we’ve seen on a robot.

The goal, note researchers in the video, is for Emo to possibly become a front end for an AI or Artificial General Intelligence (basically a thinking AI).

Emo arrives just weeks after Figure AI unveiled its OpenAI-imbued Figure 01 robot and its ability to understand and act on human conversation. That robot, notably, did not have a face.

I can’t help but imagine what an Emo head on a Figure 01 robot would be like. Now that’s a future worth losing sleep over

You might also like