Cloud-Based Robot Grasping with the Google Object Recognition Engine

Motivation:

Due to constraints on power and cost, robots operating in unstructured environments such as homes or offices have limits on onboard computation and limited data on the objects they encounter. Cloud Robotics proposes a thin-client model where robots are connected to modern cloud-computing infrastructure for access to distributed computing resources and datasets, and the ability to share training and labeling data for robot learning.

Main Points:

l  The system has two phases: offline and online.

l  Offline phase includes training of the object recognition server, the creation of object reference data, and the creation and analysis of the candidate grasp set. For each object, we capture a representative set of digital images from different viewpoints and use these to train the Google Goggles object recognition system. We also pre-compute a set of robust grasp configurations for the PR2 robot and each object using the Columbia University GraspIt! toolkit. We use sampling of spatial perturbations of each object in stable resting configurations to estimate robustness of each grasp to spatial uncertainty.

l  Online phase starts when an object is detected by the robot system which uses an onboard digital camera and 3D scanner to capture a 2D image of the object and a 3D point cloud and sends the 2D image to servers in the cloud and retrieves an object identifier, confidence measure, semantic information about the object including a 3D CAD model and reference point set, or a report that the object is not recognized.

l  If the object is recognized, the robot system locally estimates the position and orientation (pose) of the object using the 3D CAD model, the reference 3D point set, and the measured 3D point cloud. The pose is then used to index the highest probability grasp from the cloud. The robot then transforms the grasp to the local environment and attempts to execute the grasp. Once the grasp is executed, the outcome data: the image, object label, detected point cloud, estimated pose, selected grasp, and success or failure of the grasp is uploaded to the key-value store server for future reference. Failure can occur due to false or incorrect identification, or after an unsuccessful grasp.

Trade-offs:

l  The paper just said that the outcome data is uploaded for future reference. It need more details to show how the data improve the grasp selection.

l  The weight of an item is an important parameter for grasping the item. However, for some reasons, the weight of an item can change obviously even the appearance remains the same. For example, a full bottle of water compares to the bottle without water. Therefore, we cannot hardcode the weight of an item and need some methods to deal with this problem.

l  Different size of a kind of item may have the similar image. So, we need a method to determine the size of an item. For example, we can use the robot’s hand as a reference to obtain the size of an item.