BanglaAudioNative: Toward an End-to-End, Text-Free, Full-Duplex Speech Model for Bangla

Traditional conversational AI systems rely on a cascaded pipeline of automatic speech recognition (ASR), a text-based large language model (LLM), and text-to-speech (TTS) synthesis. Although functional, this text-mediated design adds latency at every stage, discards acoustic and emotional cues such as prosody and tone, and handles natural turn-taking poorly, including interruptions and concurrent backchanneling. We propose BanglaAudioNative, a research plan to build an end-to-end, audio-to-audio speech architecture for Bangla that requires no intermediate text transcription at inference. The system would model dual-stream conversational audio through a neural audio codec and an auto-regressive multi-stream transformer, mapping Bangla phonetics and prosody directly from speech to speech. To support training, we plan to build a dual-channel conversational speech dataset of spontaneous, multi-dialect Bangladeshi speech, targeting approximately 50 hours, that includes natural overlaps and Bangla–English (“Benglish”) code-switching. We will adapt an open-source full-duplex speech framework using parameter-efficient fine-tuning, with target objectives of sub-second response latency and real-time interruption handling. If successful, this work would address a gap in end-to-end voice intelligence for Bangla and provide an open blueprint for South Asian full-duplex speech systems.