Determining Phenotypic Relationships among Flax (Linum usitatissimum L.) Genotypes Using Machine Learning and Deep Learning Guided by Phenotypic Core-Based Binary Seed Image Classification

Background/Aim: Accurate identification of flax (Linum usitatissimum L.) genotypes is essential for germplasm management, breeding programmes and seed certification. Although image-based phenotyping and deep learning have shown considerable potential for genotype classification, the relationships between phenotypic similarity, within-genotype variation and classification performance remain poorly understood. This study aimed to investigate binary genotype classification using phenotypic core seed sets and to examine the relationships among deep feature space, phenotypic distance and classification performance.Methods: Single-seed images representing five flax genotypes were used to construct phenotypic core datasets by selecting representative individuals from each genotype. Binary classification was performed using four convolutional neural network architectures. Deep feature space extracted by DenseNet121 was further used to quantify phenotypic distances, evaluate within-genotype variation, construct phenotypic similarity networks and visualise genotype relationships using hierarchical clustering and UMAP.Results: Significant differences were detected among genotypes in morphological, colour and texture characteristics (P < 0.001). Binary classification performance varied among genotype pairs, with F1 scores ranging from 0.706 to 0.951. DenseNet121 achieved the highest performance in six of the ten binary comparisons, while MobileNetV2 produced the highest overall F1 score (0.951). Phenotypic distances calculated in the deep feature space showed a significant positive correlation with CNN classification performance (Pearson r = 0.710, P = 0.021), whereas no significant relationship was observed for conventional machine learning models. Phenotypic similarity networks, hierarchical clustering and UMAP projections consistently visualised phenotypic relationships among genotypes and highlighted differences in within-genotype homogeneity.Conclusions: The findings demonstrate that image-based deep phenotyping can provide biologically interpretable information beyond classification accuracy. Integrating phenotypic core selection with deep feature space analysis offers a complementary framework for evaluating genotype relationships, within-genotype variation and representative germplasm, supporting future applications in flax breeding and digital phenotyping.