How Developers Read and Use Source Code When Summarizing APIs

The study investigates how developers use source code to generate API summaries. Eyetracking data is analyzed from 30 participants performing summarization tasks. The source code (no comments) was their only information source. Participants’ summaries were evaluated for correctness and visual attention was mapped to specific code elements in the Eclipse IDE. Results show that developers who correctly summarized code spent significantly less time (half, on average) on the task. More experienced developers generally took longer, though not always resulting in a correct summary. Semantic similarity tools were used to assess whether correct participant summaries better reflected what they read, with Sentence-BERT showing moderate alignment but no statistically significant differences overall. These findings suggest that summarization accuracy is influenced more by reading strategy than experience and that current similarity models struggle to fully capture human summarization behavior. This work informs future improvements in developer tools and automated summarization systems.