TD-interpreter: enhancing the understanding of timing diagrams with visual-language learning

Abstract We introduce TD-Interpreter, a specialised machine learning (ML) workflow that assists engineers in understanding complex timing diagrams (TDs), originating from a third party, during their design and verification process. The outcome of this workflow is a visual question-answer environment which allows engineers to input a set of TDs and ask design and verification queries regarding these TDs. To address limited training data availability, we developed a synthetic data generation pipeline that aligns visual information with its textual interpretation. We then implemented TD-Interpreter using a lightweight 7B Multimodal Large Language Model (MLLM), guided by both supervised fine-tuning (SFT) and reinforcement learning (RL). We also developed extensive TD-reasoning benchmarks from simple retrieval tasks to complex reasoning problems in the real world. Our experimental results show that TD-Interpreter substantially outperformed both the untuned base model, and commercial MLLMs (GPT-5.2), on the evaluated benchmarks, demonstrating the effectiveness of the proposed workflow.

Authors

Institutions

Publication Details

Journal
Computing
Published
2026-10-16
DOI
https://doi.org/10.1007/s00607-026-01737-7
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

TD-interpreter: enhancing the understanding of timing diagrams with visual-language learning

Radu Grosu, Ezio Bartocci, Dejan Ničković, Jie He et al.
Computing
Multimodal Machine Learning Applications
article

TD-interpreter: enhancing the understanding of timing diagrams with visual-language learning

Radu Grosu, Ezio Bartocci, Dejan Ničković, Jie He, Zhantao Yang, Vincent Theo Willem Kenbeek
article en

Abstract

Abstract We introduce TD-Interpreter, a specialised machine learning (ML) workflow that assists engineers in understanding complex timing diagrams (TDs), originating from a third party, during their design and verification process. The outcome of this workflow is a visual question-answer environment which allows engineers to input a set of TDs and ask design and verification queries regarding these TDs. To address limited training data availability, we developed a synthetic data generation pipeline that aligns visual information with its textual interpretation. We then implemented TD-Interpreter using a lightweight 7B Multimodal Large Language Model (MLLM), guided by both supervised fine-tuning (SFT) and reinforcement learning (RL). We also developed extensive TD-reasoning benchmarks from simple retrieval tasks to complex reasoning problems in the real world. Our experimental results show that TD-Interpreter substantially outperformed both the untuned base model, and commercial MLLMs (GPT-5.2), on the evaluated benchmarks, demonstrating the effectiveness of the proposed workflow.

ComputingVol. 108(11)
AIT Austrian Institute of Technology GmbH (AT), TU Wien (AT), Shanghai Jiao Tong University (CN)
Key Digital Technologies Joint Undertaking, Technische Universität Wien, Österreichische Forschungsförderungsgesellschaft
Openalex Percentile: Top 99%
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.