Redesigning Mobile Systems for Foundation Models with model- and system-level orchestration

Transformer-based speech and language models deliver high-quality transcription and context-aware responses but require significant resources, complicating on-device deployment. Our work aims to build efficient mobile systems for real-time, accurate on-device model processing through system- and runtime-level innovations, thus eliminating cloud dependency and enhancing privacy.