DescriptionRobotic manipulation requires accurate geometric and mechanical models of objects as a precondition as well as a reliable object perception and robust planning to be successful. If robots cannot obtain any prior knowledge about target objects, such as full 3D CAD models or mechanical properties like mass and friction in workspace, manipulation tasks become much challenging.
In such circumstance, I propose physics-driven approaches for robust robotic manipulation with unknown objects. Firstly, I present a novel perception approaches for unknown objects: efficient 2D tracking for unknown objects, 3D pose tracking and part localization using online trackers, and distributed learning across a connected network of sensing agents. Furthermore, I present new probabilistic methods to infer geometric and mechanical models of unknown objects by leveraging non-prehensile robot actions and physic simulations. Replaying the physical interactions between hypothesized objects and robots in simulation provides clues to the unknown geometries and mechanical properties. Finally, I propose new probabilistic manipulation planning methods using the identified probabilistic geometric and mechanical models. Moreover, to achieve much efficient manipulation, I present a novel non-prehensile manipulation of multiple objects that allows a robot to rearrange multiple objects simultaneously.
The proposed methods are evaluated over real-world robot manipulation experiments. The experimental results demonstrate that the proposed methods successfully identify geometric and mechanical models of unknown objects and successfully plan challenging manipulation tasks. The proposed methods demonstrate its utility in challenging robot tasks such as pre-grasp sliding manipulation and tight bin packing.