Background
Monitoring the hands and their immediate environment can provide insight into what we are planning and the activities we are performing. When we want to grasp an object, our hands and body adjust to the shape, texture, and weight of the object. If at the same time, we can capture images of the object to be grasped, then we can use neural networks and image processing algorithms to identify the object being grasped, the type of task being performed, and where the object is later placed.
Invention Description
Researchers at Arizona State University have developed a lightweight wrist-worn monitoring system that measures the movement of the fingers and hands to simultaneously generate a video stream of the performed activities and immediate environment. The result is a time-stamped data set that captures the motion of the hand (by inertial sensors), the activity of the fingers, images of objects and the environment (by cameras). With voice or playback on a display device, an interactive Q&A system can then respond helpfully to a variety of user questions such as “Where did I leave my keys?” and “Did I take my medication this morning?”. When used as a stand-alone device for individuals, the system can be easily trained and customized using show and tell techniques—by showing and naming the objects to be tracked or the activities to be monitored. The technology can also be integrated with other activity-sensing and monitoring applications, such as those found in hospitals or assisted living facilities.
Camera located on the ventral side of the wrist.
Potential Applications
• Health and Fitness tracking
• Elderly care
• Aid for the vision-impaired
• Tracking everyday objects
Benefits and Advantages
• Lightweight tracker is worn on the wrist and places little burden on activities, unlike glove-based devices.
• Reduces the time and nuisance of locating misplaced objects.
• Does not require internet connection to protect privacy concerns.
• Simple “show and tell” device training process.
Research Homepage of Professor Troy McDaniel