An End-to-End Framework for Optical Music Recognition and Jianpu Transliteration

Abstract — Optical Music Recognition aims to automatically convert music notation into digital representations. Recent deep learning-based OMR approaches have achieved significant improvements in music symbol detection; however, most studies remain focused on object detection without fully addressing musical interpretation and end-to-end score transliteration. This research proposes an end-to-end hybrid framework for converting Western staff notation into numbered musical notation using a Dual-Model YOLO11 architecture that separates structural and semantic symbol detection. The detected symbols are subsequently processed through a Geometric Projection Layer to estimate note positions and transform geometric information into musical representations, followed by Musical Context Reconstruction to establish spatial relationships and musical structure based on music theory rules. Experimental evaluation on six test scores yielded transliteration accuracies ranging from 18.81% to 100%, with performance varying according to the characteristics and structural complexity of each musical score. These results demonstrate the feasibility of integrating deep learning-based symbol detection with geometric processing and rule-based musical reconstruction for end-to-end music score transliteration. Keywords— Geometric Projection Layer; Jianpu; Musical Context Reconstruction; Optical Music Recognition; Symbol Detection; Transliteration. Abstrak — Optical Music Recognition (OMR) merupakan bidang penelitian yang bertujuan mengonversi notasi musik ke dalam bentuk digital secara otomatis. Sebagian besar penelitian OMR berbasis deep learning masih berfokus pada deteksi simbol musik, sementara proses interpretasi hubungan antar simbol dan transliterasi notasi secara utuh masih menjadi tantangan. Penelitian ini mengusulkan sebuah framework hybrid end-to-end untuk mentransliterasikan notasi balok menjadi not angka menggunakan pendekatan Dual-Model YOLO11 yang memisahkan proses deteksi menjadi model struktural dan model semantik. Hasil deteksi kemudian diproses melalui Geometric Projection Layer untuk memperoleh estimasi posisi nada dan mentransformasikan informasi geometris menjadi representasi musikal, dilanjutkan dengan Musical Context Reconstruction untuk membangun hubungan spasial dan struktur musikal berdasarkan aturan teori musik. Hasil pengujian pada enam partitur menunjukkan akurasi transliterasi antara 18.81% hingga 100%, dengan variasi performa yang dipengaruhi oleh karakteristik dan kompleksitas struktur musikal pada masing-masing partitur. Hasil tersebut menunjukkan bahwa integrasi deteksi simbol berbasis deep learning, pemrosesan geometris, dan rekonstruksi musikal berbasis aturan dapat digunakan untuk mendukung proses transliterasi partitur musik secara end-to-end Kata kunci — Geometric Projection Layer; Jianpu; Musical Context Reconstruction; Optical Music Recognition; Symbol Detection; Transliteration.

Authors

Institutions

Publication Details

Journal
E-JOURNAL TEKNIK ELEKTRO DAN KOMPUTER
Published
2026-11-16
DOI
https://doi.org/10.35793/8yf8zy31
Primary Topic
Handwritten Text Recognition Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

An End-to-End Framework for Optical Music Recognition and Jianpu Transliteration

Vecky Canisius Poekoel, Arthur M. Rumagit, Christian Cleavy Sunkudon
1 links
E-JOURNAL TEKNIK ELEKTRO DAN KOMPUTER
Handwritten Text Recognition Techniques
article

An End-to-End Framework for Optical Music Recognition and Jianpu Transliteration

Vecky Canisius Poekoel, Arthur M. Rumagit, Christian Cleavy Sunkudon
article en

Abstract

Abstract — Optical Music Recognition aims to automatically convert music notation into digital representations. Recent deep learning-based OMR approaches have achieved significant improvements in music symbol detection; however, most studies remain focused on object detection without fully addressing musical interpretation and end-to-end score transliteration. This research proposes an end-to-end hybrid framework for converting Western staff notation into numbered musical notation using a Dual-Model YOLO11 architecture that separates structural and semantic symbol detection. The detected symbols are subsequently processed through a Geometric Projection Layer to estimate note positions and transform geometric information into musical representations, followed by Musical Context Reconstruction to establish spatial relationships and musical structure based on music theory rules. Experimental evaluation on six test scores yielded transliteration accuracies ranging from 18.81% to 100%, with performance varying according to the characteristics and structural complexity of each musical score. These results demonstrate the feasibility of integrating deep learning-based symbol detection with geometric processing and rule-based musical reconstruction for end-to-end music score transliteration. Keywords— Geometric Projection Layer; Jianpu; Musical Context Reconstruction; Optical Music Recognition; Symbol Detection; Transliteration. Abstrak — Optical Music Recognition (OMR) merupakan bidang penelitian yang bertujuan mengonversi notasi musik ke dalam bentuk digital secara otomatis. Sebagian besar penelitian OMR berbasis deep learning masih berfokus pada deteksi simbol musik, sementara proses interpretasi hubungan antar simbol dan transliterasi notasi secara utuh masih menjadi tantangan. Penelitian ini mengusulkan sebuah framework hybrid end-to-end untuk mentransliterasikan notasi balok menjadi not angka menggunakan pendekatan Dual-Model YOLO11 yang memisahkan proses deteksi menjadi model struktural dan model semantik. Hasil deteksi kemudian diproses melalui Geometric Projection Layer untuk memperoleh estimasi posisi nada dan mentransformasikan informasi geometris menjadi representasi musikal, dilanjutkan dengan Musical Context Reconstruction untuk membangun hubungan spasial dan struktur musikal berdasarkan aturan teori musik. Hasil pengujian pada enam partitur menunjukkan akurasi transliterasi antara 18.81% hingga 100%, dengan variasi performa yang dipengaruhi oleh karakteristik dan kompleksitas struktur musikal pada masing-masing partitur. Hasil tersebut menunjukkan bahwa integrasi deteksi simbol berbasis deep learning, pemrosesan geometris, dan rekonstruksi musikal berbasis aturan dapat digunakan untuk mendukung proses transliterasi partitur musik secara end-to-end Kata kunci — Geometric Projection Layer; Jianpu; Musical Context Reconstruction; Optical Music Recognition; Symbol Detection; Transliteration.

E-JOURNAL TEKNIK ELEKTRO DAN KOMPUTERVol. 15(3)
Sam Ratulangi University (ID)
Handwritten Text Recognition Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon