ECN AI Baseline Index 2024: A Stable and Accessible AI Performance Indicator Across Difficulty Shifts

Abstract We present the 2024 edition of the ECN AI Baseline Index (EAII), an accessible benchmark evaluating large language models in competitive programming under contest conditions. The universities division of the 2024 Sapientia–ECN (Efficiency, Challenge, Networking) programming contest included 11 teams and 16 university-track problems and introduced a judging platform restricted to C/C++/Pascal. Using a simple C++ prompting protocol with limited feedback rounds, the AI fully solved 13 of 16 problems (81.25%), partially solved two, and failed on one interactive task. Student teams achieved a median of nine and a maximum of twelve solved problems. The resulting 2024 EAII value is 125%. We provide per-problem and team-level analyses, along with figures summarizing score distributions, problem-specific solve rates, and year-over-year comparisons with 2023. Despite a markedly easier contest and a shift in the AI language channel (C++ versus Python), EAII remains stable relative to 2023, supporting the robustness of the normalization procedure. Interpretation is reserved for the Discussion; this paper focuses on methodology, descriptive analyses, and numerical results.

Authors

Institutions

Publication Details

Journal
Acta Universitatis Sapientiae Informatica
Published
2026-06-09
DOI
https://doi.org/10.1007/s44427-026-00033-8
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

ECN AI Baseline Index 2024: A Stable and Accessible AI Performance Indicator Across Difficulty Shifts

Zoltán Kátai, Dávid Iclănzan
Acta Universitatis Sapientiae Informatica
Artificial Intelligence in Healthcare and Education
article

ECN AI Baseline Index 2024: A Stable and Accessible AI Performance Indicator Across Difficulty Shifts

Zoltán Kátai, Dávid Iclănzan
article en

Abstract

Abstract We present the 2024 edition of the ECN AI Baseline Index (EAII), an accessible benchmark evaluating large language models in competitive programming under contest conditions. The universities division of the 2024 Sapientia–ECN (Efficiency, Challenge, Networking) programming contest included 11 teams and 16 university-track problems and introduced a judging platform restricted to C/C++/Pascal. Using a simple C++ prompting protocol with limited feedback rounds, the AI fully solved 13 of 16 problems (81.25%), partially solved two, and failed on one interactive task. Student teams achieved a median of nine and a maximum of twelve solved problems. The resulting 2024 EAII value is 125%. We provide per-problem and team-level analyses, along with figures summarizing score distributions, problem-specific solve rates, and year-over-year comparisons with 2023. Despite a markedly easier contest and a shift in the AI language channel (C++ versus Python), EAII remains stable relative to 2023, supporting the robustness of the normalization procedure. Interpretation is reserved for the Discussion; this paper focuses on methodology, descriptive analyses, and numerical results.

Acta Universitatis Sapientiae InformaticaVol. 18(1)
Sapientia Hungarian University of Transylvania (RO)
Quality Education
Openalex Percentile: Top 9%
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.