Biological and agricultural engineering associate professor Stavros Vougioukas is leading two new research projects to mechanize the orchard and pick fruit more effectively.

The first project, which received $1.2 million from the National Science Foundation’s Robotics Initiative and National Institute of Food and Agriculture (NIFA), aims to build a machine that can find and pick fruit quickly and effectively, especially when there is thick tree foliage in the way. The second project, which received $400,000 from NIFA, is focused on developing a shake-and-catch harvesting system for soft fruits to prevent fall damage. 

Both projects aim to bring machines into the orchard when the necessary number of human workers are unavailable due to labor supply, costs or public health measures.

“Finding labor to harvest orchards at a large scale, and only for a limited amount of time is challenging, so our food supply depends on having technologies that will either help people harvest faster or provide harvesting alternatives for some crops when labor is unavailable,” said Vougioukas.

Harvesting Tree Fruit with Vision-Guided Robotic Arms

Automating fruit picking is a huge challenge. Fruits comes in different maturities, shapes and sizes and can be hidden behind leaves and branches. While it’s easy for a human to quickly find and gently pick fruit under these conditions, this is much more difficult for a robot and a significant technical challenge.

To solve this, Vougioukas’ proposal is a design that combines many robotic arms, multiple cameras connected to an advanced fruit detection system powered by machine learning, and an air blowing device. The device will blow leaves and branches out of the way to make it easier to see the fruit. The cameras then send data to the neural networks, which detect the fruit, assess its location and assign and schedule one of the arms to pick it. 

“This is an operations research problem that’s very common in manufacturing and logistics,” he explained. “It’s like when Uber has multiple requests for service. The software has to decide which Uber goes to which customer or, in our case, which arm is going to pick which fruit, when, and in what sequence.”

Though each robotic arm is still slower than humans, the idea is that with enough of them, the robot will match or exceed what humans are capable of. As Vougioukas and his collaborators at Carnegie Mellon University build the prototype, they will test different methods for blowing air, different locations for the cameras and different numbers and patterns of arms to learn which is most effective. The team is also collaborating with researchers from Montana State University to address the socioeconomic aspects of deploying these machines. 

A Shake-and-Catch System for Soft Fruit Harvesting

Shake-and-catch systems are self-explanatory—the system mechanically shakes a tree while catching the fruit or nuts to collect them in bins. Though these systems have been around for years and are used for crops like nuts and processing apples, they still can’t harvest fresh-market soft fruits like peaches, apples, and pears, which are damaged by the fall and therefore have to be picked by hand. Vougioukas’ team looks to finally solve this decades-old problem by designing a system that catches these fruits sooner and gives them a softer landing. 

“If you are shaking a tree and you have fruit at 12 feet, the fruit has maybe traveled 10 feet to a catching surface near the ground and the probability that it will have hit something is very large,” he explained. “But, if you could insert soft catching surfaces every two feet, then fruits would only fall for two feet and we could significantly reduce damage from collisions.”

Since fruit can grow at several different heights in a tree, they are designing a machine that can quickly deploy landing surfaces throughout a canopy that can gently catch the fruit and help transport it into a bin. They also plan to make it collapsible for easy mobility between trees and so it can fit in commercial orchards.

To do this, he is working with UC Davis plant sciences professor Brian Bailey to build a computer model that simulates both the mechanics of a tree and how the machine interacts with it. These simulations will help them optimize the design and materials and minimize trial and error before field tests. 

He is also working with plant sciences professor Elizabeth Mitcham, a postharvest expert who will examine how the quality of machine-harvested fruits compares with those harvested by hand. He also works closely with his lab’s development engineer, Dennis Sadowski, who is very experienced in machine design and fabrication.

The goal is to develop a machine that’s comparable to the quality of hand-harvesting while being fast and efficient enough to meet the needs for timely harvesting. 

Learn more about Vougioukas’ research.