EdgeEval: A model-agnostic rapid prototyping framework for image processing at the edge

Advances in AI image classification are driving edge-deployment and on-device validation research, yet tooling hasn’t kept pace: evaluating a new architecture, quantization scheme, or domain means hardcoded label maps, fixed input shapes, and pipelines rebuilt from scratch, undermining reproducibility and cross-study comparability. A model-agnostic runtime, one binary that loads any model without code changes, enables researchers to prototype and validate rigorously while remaining deployable in real-world settings. We present EdgeEval, a Flutter toolkit that imports any TFLite model, auto-detects shape, dtype, and quantization via a Dynamic Metadata Mapping protocol, auto-resolves labels, retargets to new domains via JSON manifest with zero code changes, and profiles on-device CPU, GPU, memory, and power while verifying integrity via SHA-256. To test this, we trained three architectures (ResNet50, DenseNet121, EfficientNetB0) across four domains (rice disease, dermoscopy, satellite land cover, playing-card recognition) under three quantization variants (36 models, offline 10-fold cross-validated, paired Wilcoxon tests p < 0.05 in 9 of 12), then ran all 36 on Android hardware through the same unmodified app, on the same images. On-device int8 quantization produced a significant accuracy drop (paired McNemar, p < 0.05 ) in 9 of 12 domain-architecture pairs, matching the offline pattern, evidence that a deployment toolkit can verify whether offline characterization transfers, rather than assume it. EdgeEval gives researchers a toolkit for rigorous, reproducible prototyping and benchmarking: the same binary that validates a lab hypothesis deploys in the field.