Analysis of Accelerating Ideation Process with Large Language Model

Abstract. Large Language Models (LLMs) have rapidly advanced, demonstrating near-human intelligence in language comprehension, problem-solving, and autonomous decision-making, which can be leveraged in scientific discovery. Previous research of LLM for science often focuses on experiment execution part and the first stage of research, i.e., idea generation with large language models is lack of research. This paper fills this gap by surveying the use of LLM-based agent systems in research idea generation. This research first proposes three stages in research idea generation pipeline, pre-ideation (knowledge preparation), ideation (generation and iteration), post-ideation (evaluation), and provide detailed summary in different methods in each stage, then it summarizes different metrics and angles of evaluating generated research ideas, finally it discusses limitations and ethical concerns of existing works and suggest potential solutions and future directions. These results aim to provide a solid foundation for future research in improving LLM-based ideation systems and fostering responsible AI usage in scientific discovery.