More AI Assistance Reduces Cognitive Engagement: Examining the AI Assistance Dilemma in AI-Supported Note-Taking

As AI tools become increasingly integrated into cognitively demanding tasks, like note-taking, questions remain about whether they enhance or compromise cognitive engagement. This paper examines the ''AI Assistance Dilemma'' in note-taking, investigating how varying levels of AI support affect user engagement and comprehension. In a within-subject experiment, we asked participants (N=30) to take notes during lecture videos under three conditions: Automated AI (high assistance with structured notes), Intermediate AI (moderate assistance with real-time summary, and Minimal AI (low assistance with transcript). Results reveal that Intermediate AI yields the highest post-test scores and Automated AI the lowest. Participants, however, preferred the automated setup due to its perceived ease of use and lower cognitive effort, suggesting a discrepancy between preferred convenience and cognitive benefits. Our study provides insights into designing AI assistance that preserves cognitive engagement, offering implications for designing moderate AI support in cognitive tasks.