Engineering news
The bracelet, called FingerTrak, can sense many positions of the human hand, including 20 finger joint positions. It uses four miniature low-resolution thermal cameras that read contours on the wrist, translating them into 3D shapes. The device could be used in sign language translation, virtual reality, mobile health, human-robot interaction and other areas, said its developers at Cornell University in New York state and the University of Wisconsin.
“This was a major discovery by our team – that by looking at your wrist contours, the technology could reconstruct in 3D, with keen accuracy, where your fingers are,” said Cheng Zhang, director of Cornell's new SciFi Lab, where FingerTrak was developed. “It's the first system to reconstruct your full hand posture based on the contours of the wrist.”
Past wrist-mounted cameras have been considered too bulky and obtrusive for everyday use, the researchers said, and most could reconstruct only a few discrete hand gestures.
The new device is a lightweight bracelet, allowing for free movement. Instead of using cameras to directly capture the position of the fingers, FingerTrak uses a combination of thermal imaging and machine learning to virtually reconstruct the hand. The bracelet's four miniature cameras – each about the size of a pea – snap multiple silhouette images to form an outline of the hand.
A deep neural network then stitches these silhouette images together and reconstructs the virtual hand in 3D. Using this method, Zhang and his fellow researchers were able to capture the entire hand pose, even when the hand is holding an object.
While the technology has a wide range of possible uses, Zhang said the most promising is its potential application in sign language translation.
“Current sign language translation technology requires the user to either wear a glove or have a camera in the environment, both of which are cumbersome,” he said. “This could really push the current technology into new areas.”
FingerTrak could also help monitor disorders that affect fine-motor skills, said Yin Li, assistant professor of biostatistics at the University of Wisconsin, who contributed to the software behind FingerTrak.
“How we move our hands and fingers often tells about our health condition,” Li said. “A device like this might be used to better understand how the elderly use their hands in daily life, helping to detect early signs of diseases like Parkinson's and Alzheimer's.”
The research was published in Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies. It will also be presented at the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing, taking place virtually 12 - 16 September.
Want the best engineering stories delivered straight to your inbox? The Professional Engineering newsletter gives you vital updates on the most cutting-edge engineering and exciting new job opportunities. To sign up, click here.
Content published by Professional Engineering does not necessarily represent the views of the Institution of Mechanical Engineers.