Empirical Analysis of Model Selection and Augmentation Limits on Small Tabular Datasets

Machine learning pipelines often fail when applied to small tabular datasets (N <= 150) with high feature-to-sample ratios (P/N >= 0.1). While practitioners frequently default to tree ensembles like Random Forests or oversampling techniques like SMOTE, these methods degrade under severe sample sparsity. In this work, we systematically evaluate model stability across three distinct benchmark datasets using Stratified 5-Fold Cross-Validation. Our experiments show that margin-based classifiers (specifically RBF SVMs) outperform Random Forests by 5.7% in weighted F1-score when P/N = 0.600. Additionally, we show that standard SMOTE oversampling introduces label noise in sparse feature spaces, whereas local Gaussian jittering maintains boundary stability. We synthesize these findings into a practical Data Geometry Scanner that inspects basic dataset properties to recommend suitable model families prior to training.