Brainstorming with a Generative Language Model: The Role of Creative Ability and Tool-Support for Brainstorming Performance

Generative language models (GLMs) are used for creative tasks such as brainstorming. While studies show GLMs’ creative potential, research on who benefits from support in creative tasks is inconclusive, and concerns are raised on whether humans reduce their effort when working with GLMs. Therefore, we explored how creative ability and GLM support affect the two brainstorming performance metrics fluency and originality. In our experiment with a GPT-4-based brainstorming app, we randomly assigned the 58 participants to one of two conditions: with or without GLM-support. Additionally, we split participants into a low- and a high-creativity group based on a creativity ability pre-test. Our findings reveal that low-creativity participants show increased fluency when being GLM-supported, while the effect on originality was positive for both groups. We contribute to the literature on human-GLM brainstorming and individual creativity support systems, offering a more nuanced perspective on GLMs’ (potentially undesired) effects on behavior and performance.