Holding an open validation engine to outside references: five ECB statistics, a credit-portfolio engine, an IRRBB engine and twelve planted defects

Validators of bank risk models need tests that supervisors recognise, reference calculations to compare with, and evidence that both are correct. We describe an open engine with 134 registered tests, five of the 17 ECB statistics for validating internal credit models, a copula and CreditRisk+ portfolio engine, an interest rate risk engine for the banking book, and a protocol that holds each component to references outside it; two references share the engine's arithmetic. The five ECB statistics agree with an independent transcription of the instructions within 4.4 x 10^-11 on 500 random inputs each; the Jeffreys test differs from the exact binomial test in 1 of 70 grades at the 5 % level. The portfolio engine matches four analytic anchors, an independent simulator (largest |z| below one in 10 runs of 200,000 scenarios), an independent recursion and its committed one-million-scenario rung (relative 2.9 x 10^-9); the interest rate engine matches QuantLib (relative 2.4 x 10^-14, 72 leg and shock pairs). Not every hypothesis held: 11 of the 17 ECB statistics are not implemented, and the maximum-likelihood interval of the asset correlation covers below nominal in 16 of 24 simulator cells, outside the binomial band in 1. Planting 12 defects shows that the five verification layers are not interchangeable: 10 of 10 real defects are found, 8 by one layer only. The hypotheses were not pre-registered. That no open toolkit offers these tests is checked only against 34 package descriptions and 106 installed distributions. Repository and reproducible pipeline (MIT): https://github.com/fsantibanezleal/CAOS_Contraste .

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-08
DOI
https://doi.org/10.5281/zenodo.24053177
Primary Topic
Credit Risk and Financial Regulations
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Holding an open validation engine to outside references: five ECB statistics, a credit-portfolio engine, an IRRBB engine and twelve planted defects

Felipe A. Santibáñez-Leal
Zenodo (CERN European Organization for Nuclear Research)
Credit Risk and Financial Regulations
preprint

Holding an open validation engine to outside references: five ECB statistics, a credit-portfolio engine, an IRRBB engine and twelve planted defects

Felipe A. Santibáñez-Leal
preprint en

Abstract

Validators of bank risk models need tests that supervisors recognise, reference calculations to compare with, and evidence that both are correct. We describe an open engine with 134 registered tests, five of the 17 ECB statistics for validating internal credit models, a copula and CreditRisk+ portfolio engine, an interest rate risk engine for the banking book, and a protocol that holds each component to references outside it; two references share the engine's arithmetic. The five ECB statistics agree with an independent transcription of the instructions within 4.4 x 10^-11 on 500 random inputs each; the Jeffreys test differs from the exact binomial test in 1 of 70 grades at the 5 % level. The portfolio engine matches four analytic anchors, an independent simulator (largest |z| below one in 10 runs of 200,000 scenarios), an independent recursion and its committed one-million-scenario rung (relative 2.9 x 10^-9); the interest rate engine matches QuantLib (relative 2.4 x 10^-14, 72 leg and shock pairs). Not every hypothesis held: 11 of the 17 ECB statistics are not implemented, and the maximum-likelihood interval of the asset correlation covers below nominal in 16 of 24 simulator cells, outside the binomial band in 1. Planting 12 defects shows that the five verification layers are not interchangeable: 10 of 10 real defects are found, 8 by one layer only. The hypotheses were not pre-registered. That no open toolkit offers these tests is checked only against 34 package descriptions and 106 installed distributions. Repository and reproducible pipeline (MIT): https://github.com/fsantibanezleal/CAOS_Contraste .

Zenodo (CERN European Organization for Nuclear Research)
Credit Risk and Financial Regulations
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon