Hybrid Prompt-Optimering utan Finjustering : Förbättrad Informationsutvinning och Översättning i Fordonsunderhållsregister

Unstructured, multilingual maintenance logs hamper automated fault analysis, yet fully fine-tuning large language models (LLMs) is prohibitively costly. We therefore adapt a 70 B-parameter LLaMA-33 model without gradient updates using a two-step Hybrid Prompt Optimization (HPO) scheme that merges expert-written instructions with DSPy’s automatic prompt search. The approach is trained and tested on 1000 real, Volvo Trucks records while running only on CPU. For evaluation we draw 10 disjoint test sets of 50 records each; scores reported here are the mean of those ten runs. Relative to a zero-shot baseline, manual prompts raise structured-field extraction accuracy by 23 %; DSPy adds 27 %; and HPO supplies a further 5 %, yielding a cumulative gain of ≈ 60 %. For full-text translation the same steps deliver 55 %, +2 %, and +2 %, respectively. Paired two-tailed t-tests across the 10 × 50 predictions confirm that all improvements remain significant at α = 0.05. Under the best prompt, the lightweight LLaMA-33-70B reaches 92 % of GPT-4o’s extraction quality and slightly surpasses it in translation—yet consumes only a fraction of the compute budget. These findings show that prompt engineering alone can unlock near–state-of-the-art multilingual extraction and translation, delivering a low-cost alternative to full model fine-tuning for industrial after-sales data.

Publication Details

Journal
KTH Publication Database DiVA (KTH Royal Institute of Technology)
Published
2025-01-01
Primary Topic
Software System Performance and Reliability
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Hybrid Prompt-Optimering utan Finjustering : Förbättrad Informationsutvinning och Översättning i Fordonsunderhållsregister

KTH Publication Database DiVA (KTH Royal Institute of Technology)
Software System Performance and Reliability
article

Hybrid Prompt-Optimering utan Finjustering : Förbättrad Informationsutvinning och Översättning i Fordonsunderhållsregister

article en

Abstract

Unstructured, multilingual maintenance logs hamper automated fault analysis, yet fully fine-tuning large language models (LLMs) is prohibitively costly. We therefore adapt a 70 B-parameter LLaMA-33 model without gradient updates using a two-step Hybrid Prompt Optimization (HPO) scheme that merges expert-written instructions with DSPy’s automatic prompt search. The approach is trained and tested on 1000 real, Volvo Trucks records while running only on CPU. For evaluation we draw 10 disjoint test sets of 50 records each; scores reported here are the mean of those ten runs. Relative to a zero-shot baseline, manual prompts raise structured-field extraction accuracy by 23 %; DSPy adds 27 %; and HPO supplies a further 5 %, yielding a cumulative gain of ≈ 60 %. For full-text translation the same steps deliver 55 %, +2 %, and +2 %, respectively. Paired two-tailed t-tests across the 10 × 50 predictions confirm that all improvements remain significant at α = 0.05. Under the best prompt, the lightweight LLaMA-33-70B reaches 92 % of GPT-4o’s extraction quality and slightly surpasses it in translation—yet consumes only a fraction of the compute budget. These findings show that prompt engineering alone can unlock near–state-of-the-art multilingual extraction and translation, delivering a low-cost alternative to full model fine-tuning for industrial after-sales data.

KTH Publication Database DiVA (KTH Royal Institute of Technology)
Openalex Percentile: Top 45%
Software System Performance and Reliability
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.