Prompt-Guided Radiology Report Generation Utilizing SAM

Report generation models can effectively reduce the workload of radiologists by automatically analyzing images and generating reports. However, most reports generated by current model frameworks have many problems such as lack of completeness and misunderstanding, which affect radiologists’ willingness to adopt them. We develop a report generation model consisting of a target detection module, a feature extraction module, and a text generation module, which are used to detect lesions in images and generate lesion prompt boxes, fuse prompt features, and generate reports, respectively. In a two-stage generation way, local descriptions and complete reports are generated successively. Compared with the general frameworks, this method has advantages in both quantitative metrics and human scoring, which can provide radiologists with initial diagnosis reports in clinical applications, improve radiologists’ work efficiency and reduce missed diagnosis and misdiagnosis rates.