Can Large Language Models Improve SE Active Learning via Warm-Starts?

Software engineering now demands the navigation of vast configuration spaces, not only for software systems but also for the processes used to build them. Misconfigurations threaten performance, stability, and security. Worse, our ability to manually reason across large design spaces is limited. Hence we say (a) configurability is a liability without tool support and (b) as a priority, SE needs to explore better configuration tools. Finding good configurations is a black art, requiring much experience. When that experience is scarce, “active learners” can build configuration models by labeling the most informative examples (so configurations can be generated using very little data). Active learning can benefit from good initial guesses (known as “warm starts”). This paper tries using Large Language Models (LLMs) for creating warm-starts. Results are compared against Gaussian Process Models and Tree of Parzen Estimators. For 49 SE tasks, LLM-generated warm starts significantly improved the performance of low- and medium-dimensional tasks. However, LLM effectiveness diminishes in high-dimensional problems, where Bayesian methods like Gaussian Process Models perform best. In order to support open science, our scripts and data are online at github.com/timm/moot and github.com/lohithsowmiyan/lazy-llm.