Engineering news
A team of researchers at Carnegie Mellon University used this unlikely combination to teach a robot to recognise water and pour it into a glass. Water is difficult for robots because it's transparent – previous attempts to get robots to pour it have relied on heating the water and using a thermal imaging camera, or putting the glass in front of a chequered background.
That might work in the lab, but wouldn't translate well to your local pub. Cracking this problem could pave the way for robot servers in restaurants, or robot pharmacists that can measure and mix medicines.
The researchers used AI and image translation to solve this problem. Imagine translation algorithms use collections of images to train AIs to convert images from one style to another – a photo could be transformed into a Monet style painting, or an image of a horse could be made to look like a zebra. The AI learns what kind of transformation it needs to apply to get the desired effect (softening the edges, for instance, or adding vertical stripes).
The team used a method called contrastive learning for unpaired image-to-image translation (CUT, for short).
“You need some way of telling the algorithm what the right and wrong answers are during the training phase of learning,” said David Held, an assistant professor in the Robotics Institute who advised lead researcher Gautham Narasimhan. “However, labelling data can be a time-consuming process, especially for teaching a robot to pour water, for which the human might need to label individual water droplets in an image.”
“Just like we can train a model to translate an image of a horse to look like a zebra, we can similarly train a model to translate an image of coloured liquid into an image of transparent liquid,” Held said. “We used this model to enable the robot to understand transparent liquids.”
Transparent liquids are particularly difficult because of how they reflect, refract and absorb light, and how that changes depending on the background. To teach the computer to see different backgrounds, the team played YouTube videos behind a glass of water. Training the system this way will allow the robot to pour water against varied backgrounds in the real world, regardless of where the robot is located.
“Even for humans, sometimes it's hard to precisely identify the boundary between water and air,” Narasimhan said. After being trained, the robot was able to pour the water until it reached a certain height in the glass. In future, it could be challenged with different lighting conditions, or asked to pour water between containers.
“People in robotics really appreciate it when research works in the real world and not just in simulation,” said Narasimhan, who now works as a computer vision engineer with Path Robotics in Columbus, Ohio. “We wanted to do something that’s quite simple yet effective.”
Get to grips with the future factory at Advanced Manufacturing (18-22 July), part of the Engineering Futures webinar series. Register for FREE today.
Content published by Professional Engineering does not necessarily represent the views of the Institution of Mechanical Engineers.