Dynamic robotic grasping: A combination of real-time trajectory planning and ML-Based novel object pose detection

Abstract

This thesis investigates the integration of real-time grasping and 6 Degrees of Freedom (6-DoF) pose estimation and tracking with online trajectory optimization, focusing on enhancing grasping performance under challenging conditions such as poor lighting and with various novel objects. This is crucial for complex tasks in industrial manufacturing. A detailed selection process identified the D435i camera as optimal for its consistently lower error rates and stable performance in indoor environments, outperforming other cameras in the Realsense Depth D400 series. The experimental setup, utilizing Optitrack as the ground truth, compared the Aruco pose estimation method against the superior Foundation pose method under both static and dynamic conditions. The latter method demonstrated significant improvements in accuracy and reliability, which are essential for effective application in real-world industrial tasks. Furthermore, the integration of online trajectory optimization with real-time pose estimation facilitated precise object grasping and placement, addressing challenges such as camera calibration and frame transformation mismatches. The thesis proposes future enhancements, including real-time object detection to reduce execution times and sys tem complexity. Additionally, the pioneering integration of language-guided grasping commands aims to extend the system’s utility and applicability across diverse fields. Overall, this research demonstrates the transformative potential of advanced pose estimation and trajectory planning technologies in significantly impacting industrial automation by enabling more precise and adaptive robotic interactions in complex environments.

9