A Lightweight Audio-LLM Framework for Infant Cry Behavior Recognition Using Wav2Vec2 and TinyLlama

Infant crying is a primary form of non-verbal communication through which infants express physiological and behavioral needs. However, accurately interpreting infant cries remains challenging because cry signals exhibit substantial acoustic and temporal variability, particularly under noisy conditions. To address this challenge, we propose a lightweight Audio-Large Language Model (Audio-LLM) framework for infant cry behavior recognition by integrating a pre-trained Wav2Vec2-base audio encoder with the TinyLlama-1.1B language model. Wav2Vec2-base extracts high-level acoustic representations directly from the infant cry waveform, which are temporally aggregated and transformed through a linear projection and LayerNorm into an LLM-compatible audio token. The resulting audio representation is combined with a task-specific text prompt and processed by TinyLlama, while Low-Rank Adaptation (LoRA) enables parameter-efficient adaptation of the language model. The framework performs five-class infant cry classification, including belly pain, burping, discomfort, hunger, and tiredness, using the Donate-a-Cry Corpus dataset. The model achieved 99.34\% validation accuracy under the clean training condition. To further evaluate robustness, controlled noise experiments were conducted at 0.1\%, 0.2\%, 0.5\%, and 2\% noise levels, achieving validation accuracies of 95.33\%, 86.00\%, 71.92\%, and 67.42\%, respectively. These results demonstrate the effectiveness of pre-trained acoustic representations combined with a compact LLM for infant cry behavior recognition and provide a quantitative assessment of performance degradation under increasing noise conditions. The proposed framework offers a lightweight alternative to conventional approaches and establishes an Audio-LLM-based direction for infant cry analysis.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-19
DOI
https://doi.org/10.21203/rs.3.rs-11338456/v1
Primary Topic
Infant Health and Development
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

A Lightweight Audio-LLM Framework for Infant Cry Behavior Recognition Using Wav2Vec2 and TinyLlama

Muhammad Sheraz Khan, Yeong Min Jang, Jehad Ali, Ijaz Ahmad Khan et al.
Research Square
Infant Health and Development
preprint

A Lightweight Audio-LLM Framework for Infant Cry Behavior Recognition Using Wav2Vec2 and TinyLlama

Muhammad Sheraz Khan, Yeong Min Jang, Jehad Ali, Ijaz Ahmad Khan, Irshad Ahmad, Falak Niaz
preprint en

Abstract

Infant crying is a primary form of non-verbal communication through which infants express physiological and behavioral needs. However, accurately interpreting infant cries remains challenging because cry signals exhibit substantial acoustic and temporal variability, particularly under noisy conditions. To address this challenge, we propose a lightweight Audio-Large Language Model (Audio-LLM) framework for infant cry behavior recognition by integrating a pre-trained Wav2Vec2-base audio encoder with the TinyLlama-1.1B language model. Wav2Vec2-base extracts high-level acoustic representations directly from the infant cry waveform, which are temporally aggregated and transformed through a linear projection and LayerNorm into an LLM-compatible audio token. The resulting audio representation is combined with a task-specific text prompt and processed by TinyLlama, while Low-Rank Adaptation (LoRA) enables parameter-efficient adaptation of the language model. The framework performs five-class infant cry classification, including belly pain, burping, discomfort, hunger, and tiredness, using the Donate-a-Cry Corpus dataset. The model achieved 99.34\% validation accuracy under the clean training condition. To further evaluate robustness, controlled noise experiments were conducted at 0.1\%, 0.2\%, 0.5\%, and 2\% noise levels, achieving validation accuracies of 95.33\%, 86.00\%, 71.92\%, and 67.42\%, respectively. These results demonstrate the effectiveness of pre-trained acoustic representations combined with a compact LLM for infant cry behavior recognition and provide a quantitative assessment of performance degradation under increasing noise conditions. The proposed framework offers a lightweight alternative to conventional approaches and establishes an Audio-LLM-based direction for infant cry analysis.

Research Square
Kookmin University (KR), Islamia College University (PK), Ajou University (KR)
Infant Health and Development
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon