Newly Improved Intuitionistic Fuzzy EDAS with Interdependent Criteria Weights for Comparing Large Language Models in Text Summarization Tasks

Despite advances in using multi-criteria decision-making (MCDM) methods and their fuzzy set extensions for human evaluations of large language models (LLMs), several gaps remain in the literature, particularly in task-specific evaluations that offer a more tractable and interpretable approach. Thus, this work develops a generalized intuitionistic fuzzy MCDM approach that bridges methodological gaps by outlining two contributions. First, the integration of SWARA (Stepwise Weight Assessment Ratio Analysis) and WINGS (Weighted Influence Non-linear Gauge System) is demonstrated to compute the priority weights of the evaluation criteria, thereby augmenting the independence limitation in prior relevant studies. Second, we introduce a newly improved IF-EDAS (intuitionistic fuzzy Evaluation based on Distance from Average Solution) that preserves more uncertain information and provides a more natural extension of the canonical EDAS framework, starting with the adoption of the IFWAM (intuitionistic fuzzy weighted arithmetic mean) operator for a more intuitive approach in generating the intuitionistic fuzzy average solution vector. Also, the proposed IF-EDAS variant employs three decision rules and the Hamming distance metric in its novel computational approach. The proposed hybrid approach was deployed in two case studies evaluating five popular LLMs for text summarization across seven interdependent criteria. Results show that SWARA initially prioritizes accuracy, coherence, and consistency, but these were revised when accounting for criteria interdependence, with coherence and language quality emerging as the most preferred criteria. Both case studies suggest that Gemini may perform favorably, while Copilot may consistently rank last. The findings of the case studies share similar insights with those of three other similar IF-EDAS variants, although our claims may have limited external validity, which requires more case studies and experts in future task-specific human evaluations. The proposed approach, along with its deployment in two case studies, demonstrates human evaluations of LLMs with greater computational interpretability, which contribute to the general MCDM literature.