Motion Taxonomy and Motion Code Embedding

Competitive Advantages

This taxonomy solves the issue of language where words can take on multiple meanings. With this one can represent the mechanics of motions from the robot’s point of view
Compared to other methods, this taxonomy-based approach produces more accurate results in motion classification
The motion codes are not limited but can also include attributes that could be extracted directly from data and are more representative depending on the context

Summary

The level of insights we can get from the data around us is becoming more accurate and precise, be it from the images, videos, or text. Our researchers have developed a method for accurately detecting motion from the video feed. In robotics and AI-based applications, motion recognition is a crucial component to get a more significant level of insights such as the intent of an action performed may be learning manipulations directly from the demonstration. It is necessary to have such a classifier that could properly define actions and motions. On the level of robots, it is difficult to describe motions as we describe them in human languages in words. Approaches have been made to make machines learn directly from natural languages, like Word2Vec. However, such methods have drawback when used for motion recognition and analysis, it does not give relevant output in terms of mechanics or functionality.e., a discerning discrepancy between verbs and nouns that seems to be related. Our method in these terms can understand motion from a mechanical point of view. The coined term for this representation is the motion taxonomy, in this approach we described and represented the motion in the form of binary strings.e., the motion codes, rather than the keyword-based approach, which makes it easier for a machine learning algorithm to act upon. In our approach, we have represented various basic motions and movements based on the properties like contact/force, object, and end effector. These properties are further sub-divided based upon the flow on properties like interaction type, engagement type, contact duration, etc., where each of these properties and sub-properties has a binary representation to be able to process by the machine. We tested the algorithm on the epic-kitchen dataset, which includes various test sets for kitchen-based activities like cutting, chopping, slicing, etc. the algorithm maintained the distances that closely match the reality of manipulation and represented them accurately. With the help of this algorithm and the related taxonomy, we successfully represented motions using attributes and properties defined in the taxonomy as binary bits, and vectors which can represent the mechanics of motions from the robot’s point of view.