morisien-embed: Text Embedding Models and Evaluation for Mauritian Creole (Kreol Morisien)

Second version of the preprint. It describes two released models, morisien-embed and morisien-embed-v1.5; the version number here is the paper's, not a model's. Revision of the first version. The first version reported that a dedicated Mauritian Creole embedding model beat the general multilingual baselines. On a benchmark built to have headroom it does not, and this version says so. What is new: an out-of-domain retrieval pool built from FLORES+ with the released xSIM++ augmentation, 996 queries over 45,029 passages, on which a purely lexical baseline scores 0.3273 accuracy@1 and LaBSE 0.6928; a controlled ablation showing hard-negative mining carries the result while Matryoshka loss contributes nothing to retrieval quality; and the finding that the base model matters more than any training choice. Fine-tuning LaBSE instead of multilingual-e5-base is the only configuration measured that beats untrained LaBSE out of domain, released as morisien-embed-v1.5. Scoring follows the margin-based similarity xSIM++ specifies rather than plain cosine. Under it morisien-embed-v1.5 reaches a 0.2932 error rate against 0.3343 for untrained LaBSE, McNemar exact p = 0.00083, paired bootstrap 95% interval [-0.0653, -0.0181]. Both models fail on the same distinction: distractors that reverse a causal relation are 4.35% of the pool and cause between 43% and 50% of the errors. Also corrected: nine claims from the first version, including that the corpus was 'effectively all publicly available parallel text' (it excludes google/smol), that leakage filters removed evaluation data (they remove 0 of 69,525 rows because the splits were already disjoint), and a test-visibility optimism bound that no committed script produced, which is withdrawn

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-07
DOI
https://doi.org/10.5281/zenodo.21877805
Primary Topic
Natural Language Processing Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

morisien-embed: Text Embedding Models and Evaluation for Mauritian Creole (Kreol Morisien)

Singaraj B
Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
preprint

morisien-embed: Text Embedding Models and Evaluation for Mauritian Creole (Kreol Morisien)

Singaraj B
preprint en

Abstract

Second version of the preprint. It describes two released models, morisien-embed and morisien-embed-v1.5; the version number here is the paper's, not a model's. Revision of the first version. The first version reported that a dedicated Mauritian Creole embedding model beat the general multilingual baselines. On a benchmark built to have headroom it does not, and this version says so. What is new: an out-of-domain retrieval pool built from FLORES+ with the released xSIM++ augmentation, 996 queries over 45,029 passages, on which a purely lexical baseline scores 0.3273 accuracy@1 and LaBSE 0.6928; a controlled ablation showing hard-negative mining carries the result while Matryoshka loss contributes nothing to retrieval quality; and the finding that the base model matters more than any training choice. Fine-tuning LaBSE instead of multilingual-e5-base is the only configuration measured that beats untrained LaBSE out of domain, released as morisien-embed-v1.5. Scoring follows the margin-based similarity xSIM++ specifies rather than plain cosine. Under it morisien-embed-v1.5 reaches a 0.2932 error rate against 0.3343 for untrained LaBSE, McNemar exact p = 0.00083, paired bootstrap 95% interval [-0.0653, -0.0181]. Both models fail on the same distinction: distractors that reverse a causal relation are 4.35% of the pool and cause between 43% and 50% of the errors. Also corrected: nine claims from the first version, including that the corpus was 'effectively all publicly available parallel text' (it excludes google/smol), that leakage filters removed evaluation data (they remove 0 of 69,525 rows because the splits were already disjoint), and a test-visibility optimism bound that no committed script produced, which is withdrawn

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.