Elicitation Strategies for Capturing Information Visualization Affordances

Understanding how data visualizations shape reader takeaways is critical for designing effective displays, but measuring these affordances remains a challenge. While free-response studies provide a rich source of human interpretations, they are costly to analyze and often contain ambiguities. We investigate alternative elicitation methods, including ranking charts, ranking conclusions, and rating salience, to determine their effectiveness in capturing visualization affordances. Alternative approaches varied in their sensitivity to chart familiarity and specific affordance factors. Salience ratings aligned well with gold-standard affordances collected from free-responses but failed to capture chart-specific insights, while ranking methods overemphasized familiar chart types. Additionally, we compared human responses across all elicitation methods to outputs from GPT-4o to evaluate the extent to which large language models (LLMs) could replicate human-derived affordances. These findings underscore the importance of evaluating multiple elicitation methods and clarify the potential and limitations of LLMs as proxies for human interpretation.