Meet the Superhero Farm Robots in Training

Story by Robin Ann Smith

In the Marvel Comics universe, Thor is known for wielding a magical hammer with superhuman stamina. 

But at NC State University, Thor is a hammer-wielding robot in training for an earthly but backbreaking task: tending vegetables.

Grocery shoppers might not think about it, but the bulk of the fresh produce we buy in the supermarket is still tended and picked by hand. NC State researchers have been trying to do things a little differently, using AI and robots.

From her home office, N.C. Plant Sciences Initiative Makerspace Director Andrea Monteza proudly showed off a rendering of Thor and a second device, dubbed SpiderBot — for weaving twine — mounted on a self-driving frame about the size of a golf cart.

On-board stereo cameras act as the system’s eyes, giving it a 360-degree view of what is around. Its LiDAR laser sensors help it scan and map a field of crops and maneuver through variable terrain.

“It’s like how a Roomba uses cameras and sensors to create a floor plan of your house,” said horticultural sciences professor Emmanuel Torres, who is working with Monteza to bring the robots to life.

While machines now grow and harvest crops such as corn, wheat, soybeans and cotton, many fruit and vegetable farmers still rely on large numbers of human laborers, Torres said.

Take tomatoes and peppers. These vegetables get top-heavy as they grow. So farmers use supports like stakes and twine to keep them from toppling over onto the ground, where they become susceptible to blight.

To build this support system, workers drive thousands of tall stakes into the ground using heavy tools. They then run twine between the stakes, adding additional levels as the plants grow taller.

It’s a labor-intensive job representing a sizable expense in farm budgets. Yet it’s becoming harder to find workers willing to take it on.

Devising Super-Powered Tools

Torres, an N.C. PSI faculty affiliate, and Monteza, a mechatronics engineer, want to see if robots could automate some of the more arduous and repetitive tasks.

They have been developing prototypes for a suite of tools named after superheroes, each with unique abilities. Enter Thor. But for the tomato-staking robot to do its job properly, it must be able to hammer stakes in a precisely spaced way without damaging the plants.

To start building the intelligent tool they wanted, the researchers needed mountains of images of plants and other things a machine might encounter in farmers’ fields to train it to recognize, say, a tomato plant from a weed.

So they built another tool to streamline the process: the Hawkeye.

“We’re trying to make tools that are flexible enough for multiple potential uses.”

In the Marvel Universe, Hawkeye is an archer known for his keen eyesight and master marksmanship. At NC State, Hawkeye’s superpower is computer vision.

With support from the N.C General Assembly-funded Ag Analytics Platform and the U.S. Department of Agriculture, they came up with a tractor-mounted device equipped with low-cost computers and high-resolution, wide-lens cameras configured to capture top-down and side views of tomato plants.

The Hawkeye got its first test run last summer. The 50,000 images they collected will become the training data to make their AI algorithms smarter.

Eventually, Monteza said, the system could be used to automate the monitoring of fields for potential problems like pests, diseases and nutrient deficiencies. Traditionally, people must trek through fields and manually inspect a crop to monitor its health and see if there are any issues. A robot equipped with machine vision could automate the process.

“We’re trying to make tools that are flexible enough for multiple potential uses,” Monteza said.

Devising a Delicate Tomato Harvester

In late November, students in an electrical and computer engineering senior design course met Monteza at NC State’s horticultural field lab to test out their ideas for a robotic arm to do another type of work: picking tomatoes.

It sounds simple: their task was to design a system that can find and identify which tomatoes are ready for picking, pinpoint their locations and pluck them without squishing them.

“The beauty about machines is that they don’t need to stop, right? They can pick all day and into the night.”

That’s not easy for a machine. Tomatoes are not located in exactly the same places on every plant. One tomato might be in plain sight, another might be partially hidden or nestled among the leaves.

Electrical and computer engineering major Luke Holt showed off one of their designs. The device uses a mechanical arm guided by AI and a depth-sensing camera to scan plants and locate tomatoes, plus a silicone gripper to avoid damaging the fruit.

Currently, humans can do the task much faster and more efficiently, but the students are trying to narrow the gap.

“The beauty about machines is that they don’t need to stop, right? They can pick all day and into the night,” Torres said.

Not all the robots being developed have a superhero namesake. 

Another tool, in development with another N.C. PSI faculty affiliate, Lorena Lopez, is a device that could hoover up insects to help identify what kinds of pests are present in different crops. Lopez, an NC State Extension integrated pest management specialist with the Department of Entomology and Plant Pathology, happens to be Torres’ wife.

They hope to have the first prototype built by March.

“For now, we’re just calling it the bug vacuum,” Monteza said.

This post was originally published in College of Agriculture and Life Sciences News.