Robotic vision for crop monitoring
Harvesting and weed control methods for many horticultural crops are currently labour-intensive and costly, relying for the most part on manual labour. This project aims to overcome these problems with automation and robotics by developing an intelligent crop monitoring system, also enabling targeted decision-making for farmers.
The first step towards this technology is to develop a computer vision system that can accurately detect fruits within a crop as well as determine when fruits are ready for harvest. The team, including PhD and undergraduate students, have begun their tests with strawberries. They collected image data from strawberries in outdoor settings and developed a robot vision algorithm using deep-learning techniques which could detect fruits with 84% average precision.
Advanced sensing technologies can detect subtle changes in fruit composition invisible to the human eye. To build ripeness detection into the system, the team used a hyperspectral camera to measure reflectance data from the fruits. The hyperspectral data was correlated with water content and integrated into the model, resulting in two algorithms – one which can discriminate between underripe and ripe fruits with 84% accuracy, and another can further classify underripe, ripe and overripe with 75% accuracy.
The next steps for the project are to test the technology in a commercial horticultural setting. Long term goals are to include robotic manipulation capability alongside advanced crop identification to allow for fully automated harvesting
Professor Dana Kulic
Dr Akansel Cosgun