Plant leaf classification using the multiscale entropy of curvature and feature aggregation

This paper presents a methodology for classifying plant leaves on the basis of handcrafted features derived from the multiscale entropy of curvature and texture, as well as deep features obtained from convolutional neural networks (CNNs). We propose three object descriptors on the basis of the multiscale entropy of curvature. These object descriptors rely on the differential entropy of the probability distributions of multiscale curvatures to create a coarse-to-fine representation of the shape contour. Thus, we present a descriptor that aggregates the multiscale entropy of curvature, bending energy of curvature, and texture features to improve feature extraction of object signatures and subtle texture details of leaf images. The texture descriptor combines the statistics of the local binary pattern and gray-level co-occurrence matrix. We compare our handcrafted descriptors with deep features from various CNNs in multiclass classification using the random forest classifier, replacing the fully connected layer of the CNNs with this classifier. The experiments were conducted on four public leaf datasets: Plantscan, MED117, Flavia, and Swedish. The results of the F1-score and accuracy metrics, which exceed 99.50%, validate the aggregation strategy and show that it is competitive and powerful. The results also confirm that the proposed strategy outperformed six different sets of deep features according to the F1-score and accuracy. Moreover, the handcrafted descriptors achieved better results with 40 features than LeNet’s 50 features. The qualitative analysis of the multidimensional data visualization results prove that combining different shape features and texture details improved the description of the leaf images, as it provided better intraclass compactness and interclass separation in these datasets. • A novel descriptor based on multiscale entropy of curvature is introduced. • Differential entropy captures local and global shape complexity of leaf contours. • Shape and texture features are aggregated to enhance leaf image representation. • The handcrafted feature aggregation outperforms CNN-based features in multiclass classification tasks.