Use of generative artificial intelligence to improve output message effectiveness in decision support systems for prosumers

The object of this study is the use of generative artificial intelligence (GenAI) to create output messages in a decision support system (DSS) for prosumers. The research addresses the challenge of improving user experience (UX) by enhancing the effectiveness of DSS messages. A prototype DSS was developed for a specific private household equipped with solar panels. A rule-based message generation system was created as a baseline for comparison. An evaluation was conducted through surveys in Ukrainian and English. GenAI models from OpenAI and Anthropic were compared. Messages were assessed along two key dimensions of UX quality: usefulness and ease of comprehension. The results indicate that GenAI can enhance the effectiveness of DSS recommendations for specific user groups without adverse effects. The Sonnet 3.5 model (Anthropic) generated messages that were rated as statistically more useful (p < 0.05) by female users in Ukrainian. Users preferred shorter messages in English, and Sonnet 3.5 outperformed GPT-4 (OpenAI) in terms of usefulness in both languages (p < 0.05). The higher usefulness ratings can be attributed to more detailed recommendations while maintaining natural language. The English-language results were likely influenced by the fact that respondents were not native speakers. Differences between the models are associated with the specifics of their integration into the DSS. The results prove the hypothesis that GenAI can improve the efficiency of DSSs by generating more useful but not more complex messages. These results also indicate that GenAI’s main advantage is in tailoring the DSS output to the needs of different user groups. The difference in results between the models highlights the need for proper testing of the developed AI solutions in specific contexts. The results will be used to develop a more efficient DSS for electricity prosumers.