CodeQG: Automated Multiple Question Generation for Source Code Comprehension

During software maintenance and evolution, developers spend more than half of their time on code comprehension activities. In order to understand an unfamiliar code base, they would naturally ask different types of questions related to code snippets and try to find the answers. In this paper, we conduct an initial work to explore the possibility of automatic question generation for program comprehension. We construct a large-scale data set containing pairs of source code and questions that are automatically transformed from inline comments based on dependency analysis and semantic role labeling. We also build a comprehensive taxonomy of question types so as to generate questions concerning different aspects of code snippets, such as purpose, implementation details and so on. Then, we propose a deep learning-based prototype CodeQG to automatically generates multiple types of questions for code snippets. We evaluate CodeQG by using both typical performance metrics and manual evaluation. The results show that (1) we can achieve a value of 42.02 on BLEU4 and 60.81 on ROUGE-L for the generated questions; (2) overall, the questions are very correct in grammatical, semantic and format; (3) the questions are related to the corresponding code snippet and are helpful for developers in source code comprehension activities. Our work gives insights into automatically generating multiple types of questions for code comprehension. We expect this exploration will improve the applicability and generality of machine code comprehension.