TD-interpreter: enhancing the understanding of timing diagrams with visual-language learning
Abstract We introduce TD-Interpreter, a specialised machine learning (ML) workflow that assists engineers in understanding complex timing diagrams (TDs), originating from a third party, during their design and verification process. The outcome of this workflow is a visual question-answer environment which allows engineers to input a set of TDs and ask design and verification queries regarding these TDs. To address limited training data availability, we developed a synthetic data generation pipeline that aligns visual information with its textual interpretation. We then implemented TD-Interpreter using a lightweight 7B Multimodal Large Language Model (MLLM), guided by both supervised fine-tuning (SFT) and reinforcement learning (RL). We also developed extensive TD-reasoning benchmarks from simple retrieval tasks to complex reasoning problems in the real world. Our experimental results show that TD-Interpreter substantially outperformed both the untuned base model, and commercial MLLMs (GPT-5.2), on the evaluated benchmarks, demonstrating the effectiveness of the proposed workflow.
Authors
- Radu Grosu (ORCID: https://orcid.org/0000-0001-5715-2142)
- Ezio Bartocci (ORCID: https://orcid.org/0000-0002-8004-6601)
- Dejan Ničković (ORCID: https://orcid.org/0000-0001-5468-0396)
- Jie He (ORCID: https://orcid.org/0000-0002-9308-8886)
- Zhantao Yang (ORCID: https://orcid.org/0000-0002-3259-4400)
- Vincent Theo Willem Kenbeek
Institutions
- AIT Austrian Institute of Technology GmbH (AT)
- TU Wien (AT)
- Shanghai Jiao Tong University (CN)
Publication Details
- Journal
- Computing
- Published
- 2026-10-16
- DOI
- https://doi.org/10.1007/s00607-026-01737-7
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Key Digital Technologies Joint Undertaking
- Technische Universität Wien
- Österreichische Forschungsförderungsgesellschaft