Scalable Extraction of Author-Stated Research Gaps: Towards Automated Idea Generation

Papers state their own research gaps: limitations, future work and open problems. These statements could seed automated idea generation, but they are rare, scattered and easily confused with criticism of prior work. Asking a large language model (LLM) to judge every sentence is costly, and giving it the whole paper buries a few relevant sentences. We instead narrow each paper with a funnel: structural rules discard unlikely text, a natural-language-inference classifier proposes candidates, and the LLM verifies them without rewriting. Every output is a sentence the authors wrote, and one instruction sets the verifier to favour precision or recall. We evaluate on human-labelled limitation (SAL) and future-work (FWS) benchmarks and on 230 papers labelled by two LLM annotators and an LLM judge. The instruction controlled the trade-off on every dataset, and verification more than doubled SAL precision at 85% recall. Because the LLM only checks a short candidate list, it reads about 7-8 times fewer input tokens than whole-paper prompting on full papers, while matching or improving F1 on the human-labelled benchmarks at every model size. The gain is largest for small models: with a 4B LLM, F1 rose from 0.30 to 0.48 on arXiv papers and from 0.66 to 0.71 on SAL. Code, prompts, labels and all LLM verdicts: https://github.com/Yazangthb/Gap2Idea