From Buttons to Bots. Exploring the Impact of Enhanced Features in Voting Advice Applications on Desktops and Smartphones

Theoretical framework<br/>Voters using Voting Advice Applications (VAAs) often struggle with comprehension issues, such as understanding unfamiliar terms. Instead of seeking clarification, they expose so-called satisficing behavior making assumptions about the statements’ meaning and providing an answer nonetheless. These answers are disproportionally frequently neutral and no-opinion answers (Kamoen &amp; Holleman, 2017). <br/><br/>To reduce this satisficing behavior, Conversational Agent Voting Advice Applications (CAVAAs) have been introduced, which allow users to request additional information through a chatbot while completing a VAA (Kamoen &amp; Liebrecht, 2022). While early research indicates that users perceive CAVAAs as more useful than traditional VAAs, previous comparisons have focused only on basic VAAs without supplementary features. Many VAAs now incorporate clickable buttons for additional information, making it both practically and theoretically relevant to compare CAVAAs to enhanced VAAs (VAA+). Such comparisons can clarify whether reducing satisficing behavior is driven by the chatbot’s interactive features or simply the availability of additional information.<br/><br/>Design<br/>This research involved two experimental studies to compare four VAA versions: (1) a traditional VAA without additional features, (2) a CAVAA with a chatbot, (3) a VAA+ with clickable supplementary information, and (4) a CAVAA+ combining a chatbot with an enriched web environment. In Study 1, 145 participants used the VAAs on a desktop in a laboratory setting, while in Study 2, 201 participants used the tools on mobile devices. Participants were randomly assigned to one of the four conditions in the run-up to the European Parliament election in the Netherlands (June 2024).<br/><br/>The standard VAA was based on the Dutch Stemwijzer and included 30 political attitude statements. In the chatbot conditions, users could request information on definitions, implications, pros and cons, or party stances by clicking predefined buttons or typing questions. To ensure neutrality, chatbot responses were pre-written by researchers. The enriched VAA+ and CAVAA+ versions included clickable definitions for difficult words and tabs providing information on the status quo, pros and cons, and party stances.<br/>Participants evaluated the tool’s usability and effectiveness in a follow-up survey. Chatbot conversations and the proportion of non-substantive responses were analyzed to assess satisficing behavior.<br/><br/>Results<br/>Both studies indicate that the 4 versions were rated similarly in ease of use and playfulness. However, VAA+, CAVAA, and CAVAA+ were rated significantly higher in perceived usefulness, political knowledge, and voter preparedness compared to the basic VAA. Additionally, both chatbots and enriched web environments reduced non-substantive responses, demonstrating their potential to improve the validity of voting advice. Importantly, these findings were consistent across both desktops and mobile devices, suggesting that the observed benefits of enriched tools are robust regardless of the platform used.