Beyond Optimal Conditions in Optical Sensor Arrays: Big Analytical Data Meets Multiway Deep Learning

Abstract Complex systems comprising variate samples are influenced by numerous interacting variables that are often overlooked under conventionally optimized experimental conditions, where parameters are typically tuned using one factor at a time or experimental design approaches. Although such optimized conditions may be adequate for small or well-defined sample sets, larger and chemically diverse data sets require comprehensive exploration of the full multidimensional data space to fully extract the underlying analytical information. In this work, we considered the chemical features influencing the system by constructing a comprehensive data set that captures the intrinsic chemical complexity of the sample species rather than relying on a single optimized condition. To this end, the discrimination of 42 metal ions with different valence states was investigated using a paper-based colorimetric sensor array composed of 23 sensor elements, evaluated under six conditions comprising two paper substrates and three pH levels (3.0, 7.0, and 11). The optimization process did not achieve a cross-validated accuracy exceeding 57.94%. The resulting high-dimensional data were analyzed in three structural modes to compare two-way, three-way, and deep learning strategies: (i) the two-way tensor mode (two-dimensional matrix), in which the concatenated data of all conditions satisfied during optimization were evaluated using principal component analysis (PCA) and PCA coupled with linear discriminant analysis (PCA–LDA); (ii) the three-way tensor mode (three-dimensional), where parallel factor analysis (PARAFAC) and Tucker decomposition were applied; and (iii) the big data mode, which was analyzed using a convolutional neural network (CNN). Among these approaches, the two-way and three-way chemometric models achieved cross-validated accuracies of 72.22% and 49.21%, respectively, while the CNN model provided complete discrimination of all 42 metal ions, reaching an accuracy of 99.7% on a prediction set. Furthermore, to investigate chemical relationships and multivariate behavior, metal ions were grouped into seven chemically meaningful categories reflecting their hardness–softness, redox flexibility, bioactivity, and toxicity. Feature selection using the Davies–Bouldin index (DBI) identified 14 metal ions that could be selectively recognized by a single sensing element, demonstrating strong discriminative potential. Overall, this study highlights how moving beyond “optimal conditions” and embracing full multidimensional data sets enables deep models such as CNNs to achieve unprecedented accuracy in complex chemical samples.