Transfer-learning based multi-class areca nut image classification under uncontrolled lighting on a conveyor system for automated sorting

Areca nut is an important cash crop with high cultural and economic significance to the Indian demography. The level of ripeness can be visually assessed based on the color and is an important deciding factor for the market value of the crop and subsequent post-harvest processing such as drying, de-husking, etc. The quality assessment and segregation task, when carried out by manual labor is subjective, inconsistent, and prone to be tedious with inadvertent errors and loss in the desired efficiency. The advent of computer vision (CV)--based segregation techniques along with deep learning (DL) neural network (NN) models can augment the manual efforts in the segregation process in the agri-food sector. A fast, low-cost conveyor-based system that performs multi-class classification of areca nuts based on color and a preliminary image dataset of areca nut images captured using the same is presented here, along with explorations of some pre-trained neural network models. A feeding mechanism, a conveyor system, and a sorter system were designed and fabricated using traditional machining and 3D printing processes. A dataset of RGB images of harvested areca nuts comprising around 1600 images of areca nuts on the conveyor belt under uncontrolled lighting conditions was generated to simulate the challenging industry-relevant sorting scenarios. Transfer learning using pretrained models like Inception-V3, Resnet50 and others was utilized to implement the classification. Using contour detection and Hough circle-based automated cropping, and manual cropping, four different versions of the original dataset created were used along with two variations in dataset split and two learning rates. Inception-V3 exhibited the highest testing accuracy of 91.98% for the 80-10-10 train-cross-validation-test percentage split with a 0.0001 learning rate, with Resnet50, Inception-Resnetv2 and VGG16 exhibiting 68.52%, 90.74% and 88.27% respectively. All methods of image cropping, including manual, adversely affected all models’ classification accuracy and performance metrics, possibly owing to loss of information during cropping or file conversion operations. VGG16 was found to be more sensitive to the cropping of RoI in the image than other models. Thus, for transfer learning, the dataset preprocessing needs careful consideration.