Multi-criteria ranking of open automatic speech recognition models under alternative deployment priorities

Abstract Automatic speech recognition (ASR) models differ in accuracy, throughput, and parameter count, so choosing a model involves trade-offs. This study ranks 69 license-eligible open ASR model configurations from the July 2026 open ASR leaderboard, which covers eight short-form English test sets. Criterion weights were derived with the fuzzy analytic hierarchy process (fuzzy AHP) under three illustrative scenarios, each prioritizing one criterion: word error rate (WER), inverse real-time factor (RTFx), or parameter count. The models were then ranked with the technique for order preference by similarity to ideal solution (TOPSIS). NVIDIA Parakeet-TDT-CTC-110M ranked first in the WER-prioritized scenario, and NVIDIA FastConformer-CTC-large ranked first in the other two. The sensitivity analysis tested three further ranking methods (VIKOR, MABAC, and ARAS), classical AHP and entropy weights, stronger dominance ratings, min–max normalization, 171 weight combinations, and alternative WER measures. Across the four ranking methods and three scenarios, the top rank was always held by one of three NVIDIA models: Parakeet-TDT-CTC-110M, FastConformer-CTC-Large, or Conformer-CTC-Small. Classical AHP weights produced nearly identical rankings (Spearman rank correlation of at least 0.998), and Parakeet-TDT-CTC-110M or FastConformer-CTC-Large ranked first in 163 of the 171 TOPSIS weight combinations. However, under the strongest WER priority tested, the more accurate but larger NVIDIA Parakeet-TDT-0.6B-v2 ranked first in the WER-prioritized scenario, and ARAS favored the smallest models. The rankings are specific to this benchmark rather than validated recommendations for particular sectors, and any model’s suitability must be verified on representative audio and target hardware before deployment. The framework offers a transparent, reproducible way to compare ASR models under different deployment priorities.

Authors

Institutions

Publication Details

Journal
Scientific Reports
Published
2026-11-12
DOI
https://doi.org/10.1038/s41598-026-78831-5
Primary Topic
Speech Recognition and Synthesis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Multi-criteria ranking of open automatic speech recognition models under alternative deployment priorities

Omer A. Bafail, Adnan Miski
Scientific Reports
Speech Recognition and Synthesis
article

Multi-criteria ranking of open automatic speech recognition models under alternative deployment priorities

Omer A. Bafail, Adnan Miski
article en

Abstract

Abstract Automatic speech recognition (ASR) models differ in accuracy, throughput, and parameter count, so choosing a model involves trade-offs. This study ranks 69 license-eligible open ASR model configurations from the July 2026 open ASR leaderboard, which covers eight short-form English test sets. Criterion weights were derived with the fuzzy analytic hierarchy process (fuzzy AHP) under three illustrative scenarios, each prioritizing one criterion: word error rate (WER), inverse real-time factor (RTFx), or parameter count. The models were then ranked with the technique for order preference by similarity to ideal solution (TOPSIS). NVIDIA Parakeet-TDT-CTC-110M ranked first in the WER-prioritized scenario, and NVIDIA FastConformer-CTC-large ranked first in the other two. The sensitivity analysis tested three further ranking methods (VIKOR, MABAC, and ARAS), classical AHP and entropy weights, stronger dominance ratings, min–max normalization, 171 weight combinations, and alternative WER measures. Across the four ranking methods and three scenarios, the top rank was always held by one of three NVIDIA models: Parakeet-TDT-CTC-110M, FastConformer-CTC-Large, or Conformer-CTC-Small. Classical AHP weights produced nearly identical rankings (Spearman rank correlation of at least 0.998), and Parakeet-TDT-CTC-110M or FastConformer-CTC-Large ranked first in 163 of the 171 TOPSIS weight combinations. However, under the strongest WER priority tested, the more accurate but larger NVIDIA Parakeet-TDT-0.6B-v2 ranked first in the WER-prioritized scenario, and ARAS favored the smallest models. The rankings are specific to this benchmark rather than validated recommendations for particular sectors, and any model’s suitability must be verified on representative audio and target hardware before deployment. The framework offers a transparent, reproducible way to compare ASR models under different deployment priorities.

Scientific Reports
King Abdulaziz University (SA)
Speech Recognition and Synthesis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon