Researchers at Washington State University are using ideas from animal training to help non-expert users teach robots how to do desired tasks. The researchers recently presented their work at the international Autonomous Agents and Multiagent Systems conference, a leading scientific gathering for agents and robotics research.

As robots become more pervasive in society, humans will want them to do chores like cleaning house or cooking. But to get a robot started on a task, people who aren't computer programmers will have to give it instructions. "We want everyone to be able to program, but that's probably not going to happen," said Matthew Taylor, Allred Distinguished Professor in the WSU School of Electrical Engineering and Computer Science. "So we needed to provide a way for everyone to train robots - without programming."

User feedback improves robot performance
With Bei Peng, a doctoral student in computer science, and collaborators at Brown University and North Carolina State University, Taylor designed a computer program that lets humans teach a virtual robot that looks like a computerized pooch. Non-computer programmers worked with and trained the robot in WSU's Intelligent Robot Learning Laboratory.

For the study, the researchers varied the speed at which their virtual dog reacted. As when somebody is teaching a new skill to a real animal, the slower movements let the user know that the virtual dog was unsure of how to behave. The user could then provide clearer guidance to help the robot learn better.

"At the beginning, the virtual dog moves slowly. But as it receives more feedback and becomes more confident in what to do, it speeds up," Peng said.

The user taught tasks by either reinforcing good behavior or punishing incorrect behavior. The more feedback the virtual dog received from the human, the more adept the robot became at predicting the correct course of action.

Applications for animal training
The researchers' algorithm allowed the virtual dog to understand the tricky meanings behind a lack of feedback - called implicit feedback. "When you're training a dog, you may withhold a treat when it does something wrong," Taylor explained. "So no feedback means it did something wrong. On the other hand, when professors are grading tests, they may only mark wrong answers, so no feedback means you did something right."

The researchers have begun working with physical robots as well as virtual ones. They also hope to eventually use the program to help people learn to be more effective animal trainers.