Robotic grasping in industries like manufacturing, warehousing, and logistics is still a bottleneck when dealing with dense or crowded environments. Robotic systems are typically developed to handle single objects and fail to perform in terms of reliability, precision, and versatility when dealing with multiple objects overlapping or entangled, especially when precise vision and spatially aware grasp planning is necessitated. This bottleneck degrades productivity in applications like bin picking, kitting, or part assembly when dexterity with a human hand is still required to deal with complicated object retrieval.
The Robot Perception and Action Lab at the University of South Florida developed the technology with a vision-guided Multi-Object Grasping (MOG) pipeline to grasp two cuboid objects against the surface of a disordered pile in real time. The system combines 6D pose estimation, collision-free pair selection, adaptive thumb positioning, and confidence-ranked grasping to emulate decision-making of humans when manipulating objects. In contrast to traditional systems, it applies grasp confidence modeling (drawing upon a fine-tuned DexNet-2.0) to pick and perform high-probability grasps and avoid failed attempts. Testing resulted in up to 86% success rate in simulated scenarios and 76% in real-world experiments and up to 100% availability when prioritizing controllable grasps. The solution bridges a gap in robotic dexterity, providing both precision and efficiency to achieve real-time high-volume tasks involving multiple objects in industrial and research settings.
Pose estimation of segmented objects