Preference-Based Optimization with Automatically Constructed Preferences for Low-Resource Kazakh-to-Kazakh Sign Language Gloss Translation: A Diagnostic Study of PPO with LoRA

Kazakh Sign Language (KSL) remains a low-resource language with limited parallel data for automatic translation. This study examines written Kazakh-to-KSL gloss translation with supervised models and preference-based optimization. A Transformer baseline and mBART-large-50 were trained on a parallel Kazakh–KSL corpus of 97,330 sentence pairs whose targets are derived from the source by rule-based reordering and tense-marker insertion. A reward model was trained on 79,215 automatically constructed preference pairs (corpus references as chosen, LLM-generated candidates as rejected) and a 27B-parameter Qwen3.6 policy adapted with low-rank adaptation (LoRA) was optimized with Proximal Policy Optimization (PPO). All systems were evaluated on a common test set of 8661 sentences disjoint from every training stage. PPO improved its own starting policy (sentence-level chrF 0.705 to 0.774; exact match 0.069 to 0.096), but the resulting policy remained below the supervised Transformer (BLEU 0.723) and mBART (sentence-level chrF 0.937; exact match 0.493). The reward model achieved 0.971 pairwise accuracy on the preference pairs (an in-sample estimate), but a classifier using surface features alone reached 0.984, and accuracy on pure word-order negatives was 0.635. These results show that, in this setting, held-out metrics largely reward reproduction of the source wording, which 24.3% of the references copy exactly, and that the reward signal largely reflects surface properties of the constructed pairs; we report a data-flow audit and shortcut analyses as a basis for future preference-based work on low-resource sign language translation.