Skip to main content

Validation of computer vision models applied to harvests

This use case provides a service for validating fruit detection algorithms from agricultural images. It uses datasets with real annotations to compare the predictions generated by the models with the ground truth. The objective is to offer companies in the sector a reliable tool to evaluate the accuracy of their algorithms without conflicts of interest.

Fig. 1. Commercial logo of the use case. (source: AgrospAI)

Introduction​

Objectives​

  • Provide a validation service that allows companies to evaluate the accuracy and reliability of their fruit detection algorithms using agricultural images.
  • Guarantee neutral validation without conflicts of interest, being driven by the University of Lleida (UdL), a public entity that acts as a trusted third party.
  • Contribute to the improvement of the quality of predictive models used in the agri-food sector, promoting the responsible and transparent use of artificial intelligence in agricultural environments.
  • Guarantee data sovereignty, ensuring that producers maintain complete ownership and control over the information generated on their farms.

Problem description​

Currently, many companies in the agri-food sector face the difficulty of objectively and reliably validating the detection and prediction algorithms they use for agricultural image analysis. The absence of standardized and accessible methods to check the accuracy of these tools generates uncertainty about the quality of the predictions made.

This deficiency hinders confidence in the adoption of artificial intelligence and computer vision technologies, limiting the potential for innovation and improvement in production processes. Furthermore, the lack of independent validation services leads to a possible conflict of interest when evaluations are carried out by the developing companies themselves.

🔎 Therefore, it is necessary to have a neutral and rigorous system that allows the effectiveness of algorithms to be evaluated with guarantees, offering companies transparent and objective validation that promotes the reliable use of these technologies in the agricultural sector.

Proposed solution​

Fig. 2. Visual scheme of the proposed solution in this use case. (source: AgrospAI)

The previous diagram visually shows the operation of the proposed system for the validation of fruit detection models using agricultural images.

In this system, different participants contribute their artificial intelligence models, which are executed on a set of real data containing agricultural images annotated with the ground truth.

These annotations serve as a reliable reference to evaluate model performance. The models generate automatic fruit predictions or detections that are compared with the real annotations through an evaluation service. This service automatically activates a performance estimation module that calculates objective metrics.

With the results obtained, a complete evaluation report is generated and delivered to the consumer interested in validating their model. The entire process is carried out in an infrastructure managed by a neutral access and computing provider, guaranteeing an independent, transparent evaluation without conflicts of interest, and also ensuring that the data used remains under the control of its original owners.

Harvest validation algorithm​

What does the algorithm do?​

The validation algorithm executes a process that compares the predictions made by a computer vision model with the ground truth contained in a set of annotated agricultural images. To do this, it generates a confusion matrix from the true positives, false positives, and false negatives detected, and calculates standard evaluation metrics.

Results and benefits​

The final result of the validation is a comprehensive report that can be downloaded by the consumer, implemented in HTML, JavaScript, and Python. This report is designed to offer a clear and structured view of the evaluated model's performance. It includes an executive summary with the main results, a detailed description of the applied methodology, a visual and quantitative analysis of the confusion matrix, a section dedicated to average metrics, and a performance comparison section between different models.

Fig. 3. Result obtained from the application of the validation algorithm. (source: AgrospAI)

Thanks to this report, companies can objectively identify the strengths and limitations of their algorithms, justify their technological decisions with quantitative evidence, and move towards a more responsible and efficient adoption of artificial intelligence-based tools in agricultural environments.