Mandarin count estimation with 360-degree tree video and transformer-based deep learning

In mandarin orange production, knowing the number of mandarins on a tree is crucial for performing appropriate thinning operations. This is necessary to prevent overbearing and to maintain the size and quality of individual mandarins. However, the dense foliage of mandarin trees often obscures many fruits. Before thinning, a tree can have several hundred to several thousand mandarins, making manual counting not only labor-intensive but also exceedingly difficult. This study proposes a novel technology for automatically estimating the number of small, unripe mandarins before thinning operations using 360-degree videos of the trees. A deep neural network model leveraging Transformer architecture is designed to fuse features from various viewing angles. State-of-the-art (SOTA) unsupervised pre-training techniques, along with a custom-designed loss function, are employed to tackle the challenge of limited training data and to enhance the model's capability to capture global contextual features from images taken at multiple angles. Experimental results demonstrate that the proposed method significantly outperforms existing methods.