Robotic Picking and the Beauty of Human Hand-Eye Coordination

The human hand is a wondrous device. The complexity of its design is such that even the most advanced grippers pale in comparison. It does have its shortcomings, mainly in strength, stamina, and durability. But the range of tasks it can accomplish is staggering.

What makes the hand even greater is its connection to the eyes. Your arm and hand are not programmed to take one static path to grab something. Hand eye coordination enables an infinite number of moves.

Consider the difference between these five tasks:

1. Pick that up from that place.
2. Pick any of these objects from those places.
3. Pick any of these objects mixed in a pile from multiple places.
4. Pick any objects up from any place.
5. Safely do 1 to 4 in the presence of humans.

Each is relatively simple for humans, but progressively harder for robots.

Robots have been doing the first task for a long time. The second is more challenging because it requires a gripper versatile enough to handle a range of items and grab from different angles and reach.

In the third, the robot must do everything from task two plus be able to discern one item from another. This requires intense vision software that can pick out a specific shape, determine the angle at which it lays, determine how best to grab it from this angle or abort and find one that is easier to grab. It also needs to know if it failed (no grab, dropped, etc.) so it can try again. To accomplish this, we have to teach the robot about every item we choose to make it grab.

In the fourth task, the number of angles, positions, and orientations of items is nearly infinite. There is no way to discretely program a robot for each scenario. Infinite permutation equals infinite software hours.

Manually “teaching in” robots has become particularly problematic with the emergence of e-commerce. No longer do we have to contend with hundreds or even 50,000 SKUs. We are talking about handling hundreds of thousands or even millions of SKUs.

Let’s say that we have to teach in 500,000 SKUs to a robot using a non-engineer. If it takes 5 minutes per item and can be done by a $35/hour technician, it would take 41,670 man-hours at a cost of $1.45 million to teach the robot to pick all the SKUs.

If we can reduce the time to 1 minute and make it simple enough for a $15/hour employee, man-hours are reduced to 8,300 and the cost comes down to $125,000. That is a huge improvement.

Even better, what if we don’t need to teach the robots at all? What if through machine learning and gripper-vision coordination our robots can learn to grab anything and put it anywhere?

This is what we are working on at Swisslog and KUKA. We already have the first and second tasks down pat. Task three is now commercially viable and our Swisslog Robogistics team is working tirelessly to conquer task four. They are moving closer to commercial viability as I write this.
And, we have already skipped to task five: this robot can safely operate with humans without a fence.

This year at the ProMat show, you can see our Automated Item Picking Robot and learn how we have solved the hand-eye coordination problem. In line with our vision, we believe this robot will shape the future of e-commerce.