A multi-dimensional approach to improve validation practices for qualitative models of marine social-ecological systems

Current modeling practices for social-ecological systems (SES) are often qualitative and use causal loop diagrams (CLDs), as these models promote an evaluation of the systems loops and variable connectivity. Our literature review demonstrated that quality assurance of these models often lacks a consistent validation procedure. Therefore, a guide to improving the validation of qualitative models is presented. The presumed utility protocol is a multi-dimensional protocol with 26 criteria, organized into four dimensions, designed to assess specific parts of the modeling process and provide recommendations for improvement. This protocol was applied to three demonstration cases, located in the Arctic Northeast Atlantic Ocean, Macaronesia, and the Tuscan archipelago. The “Specific Model Tests” dimension, which focuses on the structure of the model, revealed positive evaluations of its structure, boundaries, and capacity to be scaled up. “Guidelines and Processes”, which focuses on the meaning and representativeness of the process, showed positive results regarding purpose, usefulness, presentation, and meaningfulness. “Policy Insights and Spillovers”, a dimension focused on the policy recommendations, revealed a high number of “not apply”, indicating that several criteria are too advanced for the status of the models tested. The “Administrative, Review, and Overview” dimension, which focused on the managerial overview, showed the models needed improvement in the documentation and replicability, while time and cost constraints were positively evaluated. The presumed utility protocol has shown to be a useful tool providing quantitative and qualitative evaluations for an intermediate evaluation of the model-building process, helping to substantiate confidence, with recommendations for improvements and applications elsewhere.

Authors

Institutions

Publication Details

Journal
Current Research in Environmental Sustainability
Published
2024-12-18
DOI
https://doi.org/10.1016/j.crsust.2024.100273
Citations
3
Primary Topic
Complex Systems and Decision Making
Type
article
Field-Weighted Citation Impact
1.47

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A multi-dimensional approach to improve validation practices for qualitative models of marine social-ecological systems

Andrea Z. Botelho, Manuela I. Parente, Sandra Rybicki, Unn Laksá et al.
3 citations
Current Research in Environmental Sustainability
Complex Systems and Decision Making
1.47
article

A multi-dimensional approach to improve validation practices for qualitative models of marine social-ecological systems

Andrea Z. Botelho, Manuela I. Parente, Sandra Rybicki, Unn Laksá, Jesús P. García, Candelaria Cecilia-Ruano, Alexander Harry McGrath, Chiara Ravaglioli, Anna H. Ólafsdóttir, Pamela J. Woods, Paola Parretti, Daniela Casimiro, Ana Dinis, Furqan Asif, Ricardo Haroun, Gustavo M. Martins, Catherine Chambers, Maria A. Ventura, Sarai Pouso, Caterina Mintrone, João Canning‐Clode, Marvin E. Olsen, Bruno Meirelles de Oliveira, Ana C. Costa, Nuno V. Álvaro, Ángel Borja, Talea Weissang
article en
3 citations

Abstract

Current modeling practices for social-ecological systems (SES) are often qualitative and use causal loop diagrams (CLDs), as these models promote an evaluation of the systems loops and variable connectivity. Our literature review demonstrated that quality assurance of these models often lacks a consistent validation procedure. Therefore, a guide to improving the validation of qualitative models is presented. The presumed utility protocol is a multi-dimensional protocol with 26 criteria, organized into four dimensions, designed to assess specific parts of the modeling process and provide recommendations for improvement. This protocol was applied to three demonstration cases, located in the Arctic Northeast Atlantic Ocean, Macaronesia, and the Tuscan archipelago. The “Specific Model Tests” dimension, which focuses on the structure of the model, revealed positive evaluations of its structure, boundaries, and capacity to be scaled up. “Guidelines and Processes”, which focuses on the meaning and representativeness of the process, showed positive results regarding purpose, usefulness, presentation, and meaningfulness. “Policy Insights and Spillovers”, a dimension focused on the policy recommendations, revealed a high number of “not apply”, indicating that several criteria are too advanced for the status of the models tested. The “Administrative, Review, and Overview” dimension, which focused on the managerial overview, showed the models needed improvement in the documentation and replicability, while time and cost constraints were positively evaluated. The presumed utility protocol has shown to be a useful tool providing quantitative and qualitative evaluations for an intermediate evaluation of the model-building process, helping to substantiate confidence, with recommendations for improvements and applications elsewhere.

Current Research in Environmental SustainabilityVol. 9
University of Pisa (IT), Universidade dos Açores (PT), Universidad de Las Palmas de Gran Canaria (ES), Madeira Tecnopolo (PT), Stefansson Arctic Institute (IS), CoNISMa (IT), Aquatic Systems (United States) (US), Care Resource (US), Hospital de Santo Espirito de Angra do Heroísmo (PT), Marine Technology Unit (ES), Universidade da Madeira (PT), Aalborg University (DK)
National University of Ireland, European Commission, University College Cork, Università di Pisa, HORIZON EUROPE Framework Programme
Life below water
Openalex Percentile: Top 15%
Complex Systems and Decision Making
1.47
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.