Extended Kalman Filter Based Tracking Method for Accurate Fruit Yield Estimation Preserving SE(3) Equivariance

Automatic yield estimation is crucial for fruit cultivation, impacting everything from harvesting to marketing. This article introduces an efficient tracking mechanism for accurate yield estimation in mango farming, addressing challenges such as fruit detection inconsistency and over-counting. We utilized this tracking-based solution on a video dataset collected in a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$360^\circ$</tex-math></inline-formula> viewpoint of each mango tree in one-acre Banginapalle orchard during daylight. The videos underwent preprocessing, including gamma correction, Gaussian smoothing, and stabilization to minimize the quivering of video frames. We also implemented a cosine similarity technique to remove redundant frames with 90% similarity and segmented the canopy to identify the regions of interest. The mango detection system employs YOLOv8s and an extended Kalman filter that preserves special Euclidean group [SE(3)] equivariance, ensuring accurate mango tracking across frames, which is robust to camera movements through angular estimation. Our method surpasses existing tracking-bas algorithms such as Sort, DeepSort, and Bot-sort in tests with ten video sequences. In addition, the results are also comparable to the harvest count obtained from the farmer and the labeling count performed manually in the video frames, achieving results close to a mean absolute error of 0.341 and 0.089, respectively.