Scalable Machine Learning Pipelines for Big Telemetry Data inSemiconductor Manufacturing

The semiconductor industry faces increasing challenges in maintaining high yields and reducing costs asmanufacturing processes become more complex. A new and effective tool for optimising processes is big dataanalytics, enabling manufacturers to extract valuable insights from vast amounts of production data and make data-driven decisions. This study proposes a comprehensive machine learning (ML) pipeline tailored for analyzingtelemetry data using the SECOM dataset from the UCI repository. The methodology includes data cleaning, missingvalue imputation, feature scaling via Min-Max normalization, dimensionality reduction, and Synthetic MinorityOversampling Technique (SMOTE) to handle class imbalance. A Decision Tree Classifier (DTC) is utilized to classifygood and defective products, achieving an accuracy of 88% in addition to excellent results in terms of recall, F1-score,ROC-AUC, and accuracy. Based on a comparison, the offered DTC model performs much better than populartraditional and deep learning techniques and can be trusted for spotting and addressing faults in real life.