SignFlow: Real-Time Multi-Modal Sign Language Recognition via 12-Stream Hybrid Architecture on the Full-Vocabulary ASL Citizen Benchmark

Isolated sign language recognition (ISLR) across comprehensive natural vocabularies is constrained by signer heterogeneity, subtle phonemic contrasts, and high computational latency on edge hardware. We present SignFlow, an end- to-end, on-device sign language recognition system evaluated on the complete official ASL Citizen benchmark (2,731 glosses, 32,940 test clips, evaluated across strictly held-out signers). SignFlow introduces an asymmetrical 12- stream hybrid architecture integrating: (i) four Landmark Transformers initialized via self-supervised masked-frame pretraining; (ii) three 1D-CNN temporal convolutional experts with dilated multi-round receptive fields; and (iii) five VideoMAE appearance models capturing full-frame dynamics and hand- crop details. Fusion weights and class-conditional temperature parameters are fit strictly on the official validation split via L-BFGS log-linear cross- entropy optimization. Evaluated on the 32,940-clip signer-independent test split, SignFlow achieves 87.04% Top-1 (28,670/32,940 correct), 98.27% Top-5, and 98.89% Top-10 accuracy. A single 1D-CNN expert (conv3) attains 85.16% Top- 1 solo. Signer-clustered bootstrap resampling (B=2,000) establishes a 95% confidence interval of [85.61%, 88.35%]. The entire pipeline operates locally on consumer hardware with an inference latency of 72 ms per clip on an NVIDIA RTX 3070 Ti laptop GPU.