LLM - TG: Towards Automated Test Case Generation for Processors Using Large Language Models

Design verification (DV) has existed for decades and is crucial for identifying potential bugs before chip tape- out. Hand-crafting test cases is time-consuming and error-prone, even for experienced verification engineers. Prior work has attempted to lighten this burden by rule-guided random test case generation. However, this approach does not eliminate the manual effort required to write rules that describe detailed hardware behavior. Motivated by advances in large language models (LLMs), we explore their potential to capture register transfer level (RTL) behavior and construct prompts for test case generation based on RTL behavior. First, we introduce a prompt framework, LLM - Driven Test Generation (LLM - TG), to generate test cases, thereby enhancing LLMs' test generation capabilities. Additionally, we provide an open-source prompt library that offers a set of standardized prompts for processor verification, aiming to improve test generation efficiency. Lastly, we use an LLM to verify a 12-stage, multi-issue, out-of-order RV64GC processor, achieving at least an 8.34 % increase in block coverage and at least a 5.8 % increase in expression coverage compared to the state-of-the-art (SOTA) methods, LLM4DV and RISCV- DV. The prompt library is available at https://github.com/LLM-TGIPrompt_Library.

Authors

Institutions

Publication Details

Published
2024-11-18
DOI
https://doi.org/10.1109/iccd63220.2024.00066
Citations
4
Primary Topic
Software Testing and Debugging Techniques
Type
article
Field-Weighted Citation Impact
1.94
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

LLM - TG: Towards Automated Test Case Generation for Processors Using Large Language Models

Jingyue Zhao, Chao Xiao, Zhijie Yang, Yuanfeng Luo et al.
4 citations
Software Testing and Debugging Techniques
1.94
article

LLM - TG: Towards Automated Test Case Generation for Processors Using Large Language Models

Jingyue Zhao, Chao Xiao, Zhijie Yang, Yuanfeng Luo, Renzhi Chen, Na Li, Zhong Wan, Lei Wang, Huadong Dai, Yifei Deng, Yongbao Ai
article en
4 citations

Abstract

Design verification (DV) has existed for decades and is crucial for identifying potential bugs before chip tape- out. Hand-crafting test cases is time-consuming and error-prone, even for experienced verification engineers. Prior work has attempted to lighten this burden by rule-guided random test case generation. However, this approach does not eliminate the manual effort required to write rules that describe detailed hardware behavior. Motivated by advances in large language models (LLMs), we explore their potential to capture register transfer level (RTL) behavior and construct prompts for test case generation based on RTL behavior. First, we introduce a prompt framework, LLM - Driven Test Generation (LLM - TG), to generate test cases, thereby enhancing LLMs' test generation capabilities. Additionally, we provide an open-source prompt library that offers a set of standardized prompts for processor verification, aiming to improve test generation efficiency. Lastly, we use an LLM to verify a 12-stage, multi-issue, out-of-order RV64GC processor, achieving at least an 8.34 % increase in block coverage and at least a 5.8 % increase in expression coverage compared to the state-of-the-art (SOTA) methods, LLM4DV and RISCV- DV. The prompt library is available at https://github.com/LLM-TGIPrompt_Library.

National University of Defense Technology (CN), Academy of Military Medical Sciences (CN)
Openalex Percentile: Top 11%
Software Testing and Debugging Techniques
1.94
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.