A Multi‐Criteria Decision Framework for Performance Evaluation of Large‐Scale Research Infrastructure: A Fuzzy Borda Approach With IoT‐Enabled Case Study

ABSTRACT This study presents a multi‐criteria decision framework for quantitatively evaluating and enhancing the utilization efficiency of large‐scale instruments (LSIs) within academic institutions. Facing the complex socio‐technical challenge of balancing multi‐stakeholder needs, including researchers, facility managers, and institutional administrators, we developed an integrated evaluation model based on the fuzzy Borda method. The model systematically prioritizes four key system performance indicators: Total operating hours, Basic management capacity, Testing revenue, and Research output. Deployed in a real‐world case at Shanghai University involving 350 LSIs, indicators of Testing revenue and Research output show high sensitivity to Internet of Things (IoT) implementation in the presence of remote management and improved user access from the IoT‐based technology. This further leads to non‐uniform efficiency distributions under both part‐time and full‐time management regimes. Therefore, this framework was used to assess the impact of a pervasive IoT intervention. A significant 15.00% average improvement in overall system efficiency was achieved through implemented IoT. Monte Carlo simulation subsequently supports its robustness (between 14.87% and 14.98%). This work contributes a transferable decision‐support methodology for the operational performance evaluation of complex shared infrastructure, highlighting how the core challenges in system sustainment and optimization could be addressed via systematic evaluation and digital integration. By separating the measurement methodology contribution (a fuzzy Borda‐based evaluation framework) from the empirical IoT impact assessment, the study provides both a general‐purpose evaluation tool and concrete evidence of IoT's efficacy. Significance and Practitioner Points This study contributes a validated multi‐criteria decision framework by integrating the fuzzy Borda method with Monte Carlo‐based robustness analysis, capable of quantifying the utilization efficiency of shared research infrastructure. The framework systematically translates multi‐stakeholder preferences into weighted performance metrics, covering encompassing operational throughput, financial return, research output, and management quality; it also addresses inherent uncertainty and subjectivity in expert judgments. Empirical findings reveal that IoT‐driven efficiency gains are significantly moderated by underlying management regimes (full‐time versus part‐time staffing), offering new insights into the nature of technology‐enabled performance improvement in socio‐technical systems. For practitioners, the framework offers a transparent and actionable tool to assess high‐value shared assets. It converts initially ambiguous stakeholder priorities into clearly defined performance indicators, providing a solid basis for investment decisions and policy development. The Shanghai University case study illustrates how IoT deployment, including automated usage logging, QR‐code access, and dual‐sensor validation, can reduce administrative workload and improve instrument accessibility, thereby improving the utilization efficiency of shared research infrastructure. The framework can be adapted to various institutional settings, regardless of whether they operate under full‐time or part‐time management models, and allows stakeholder preferences to be updated periodically as institutional priorities shift over time.

Authors

Institutions

Publication Details

Journal
Systems Engineering
Published
2026-08-24
DOI
https://doi.org/10.1002/sys.70087
Primary Topic
IoT and Edge/Fog Computing
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A Multi‐Criteria Decision Framework for Performance Evaluation of Large‐Scale Research Infrastructure: A Fuzzy Borda Approach With IoT‐Enabled Case Study

Haoliang Li, Xinru Liu
Systems Engineering
IoT and Edge/Fog Computing
article

A Multi‐Criteria Decision Framework for Performance Evaluation of Large‐Scale Research Infrastructure: A Fuzzy Borda Approach With IoT‐Enabled Case Study

Haoliang Li, Xinru Liu
article en

Abstract

ABSTRACT This study presents a multi‐criteria decision framework for quantitatively evaluating and enhancing the utilization efficiency of large‐scale instruments (LSIs) within academic institutions. Facing the complex socio‐technical challenge of balancing multi‐stakeholder needs, including researchers, facility managers, and institutional administrators, we developed an integrated evaluation model based on the fuzzy Borda method. The model systematically prioritizes four key system performance indicators: Total operating hours, Basic management capacity, Testing revenue, and Research output. Deployed in a real‐world case at Shanghai University involving 350 LSIs, indicators of Testing revenue and Research output show high sensitivity to Internet of Things (IoT) implementation in the presence of remote management and improved user access from the IoT‐based technology. This further leads to non‐uniform efficiency distributions under both part‐time and full‐time management regimes. Therefore, this framework was used to assess the impact of a pervasive IoT intervention. A significant 15.00% average improvement in overall system efficiency was achieved through implemented IoT. Monte Carlo simulation subsequently supports its robustness (between 14.87% and 14.98%). This work contributes a transferable decision‐support methodology for the operational performance evaluation of complex shared infrastructure, highlighting how the core challenges in system sustainment and optimization could be addressed via systematic evaluation and digital integration. By separating the measurement methodology contribution (a fuzzy Borda‐based evaluation framework) from the empirical IoT impact assessment, the study provides both a general‐purpose evaluation tool and concrete evidence of IoT's efficacy. Significance and Practitioner Points This study contributes a validated multi‐criteria decision framework by integrating the fuzzy Borda method with Monte Carlo‐based robustness analysis, capable of quantifying the utilization efficiency of shared research infrastructure. The framework systematically translates multi‐stakeholder preferences into weighted performance metrics, covering encompassing operational throughput, financial return, research output, and management quality; it also addresses inherent uncertainty and subjectivity in expert judgments. Empirical findings reveal that IoT‐driven efficiency gains are significantly moderated by underlying management regimes (full‐time versus part‐time staffing), offering new insights into the nature of technology‐enabled performance improvement in socio‐technical systems. For practitioners, the framework offers a transparent and actionable tool to assess high‐value shared assets. It converts initially ambiguous stakeholder priorities into clearly defined performance indicators, providing a solid basis for investment decisions and policy development. The Shanghai University case study illustrates how IoT deployment, including automated usage logging, QR‐code access, and dual‐sensor validation, can reduce administrative workload and improve instrument accessibility, thereby improving the utilization efficiency of shared research infrastructure. The framework can be adapted to various institutional settings, regardless of whether they operate under full‐time or part‐time management models, and allows stakeholder preferences to be updated periodically as institutional priorities shift over time.

Systems Engineering
Shanghai University (CN), Shanghai Research Institute of Materials (CN)
Industry, innovation and infrastructure
Openalex Percentile: Top 7%
IoT and Edge/Fog Computing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.