Self-Healing Web Data Extraction Pipelines: Combining Lightweight LLMs and Deterministic Parsers for E-Commerce and Telecom Intelligence

Production web data extraction pipelines for e-commerce and telecommunications intelligence face a persistent operational trade-off: deterministic parsers (XPath/CSS via lxml and parsel) offer near-zero marginal cost and millisecond latency but break frequently under frontend DOM mutations, whereas routing raw HTML through Large Language Models (LLMs) incurs prohibitive token costs, high latency, and exposure to indirect prompt injection. In this technical note (Report ID: UNAL-CS-2026-TN02), we present a self-healing extraction architecture deployed on serverless cloud workers that decouples routine parsing from LLM-assisted recovery. Under normal operation, extraction relies strictly on deterministic selectors validated against strict Pydantic schemas. When structural drift or schema violations trigger an anomaly circuit breaker, the system isolates the mutated HTML region via Lowest Common Ancestor (LCA) DOM pruning, encapsulates the untrusted snippet within cryptographic XML nonce boundaries to mitigate indirect prompt injection, and invokes a lightweight LLM out-of-band solely to synthesize a patched XPath/CSS selector. Once verified against historical schema invariants in a sandbox, the new selector hot-swaps into the production registry. Evaluated across Latin American and Colombian retail and telecommunications portals, this hybrid pipeline resolves layout breaks autonomously while reducing token consumption by over 98% compared to full-page LLM scraping.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-09
DOI
https://doi.org/10.5281/zenodo.24064915
Primary Topic
Web Data Mining and Analysis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Self-Healing Web Data Extraction Pipelines: Combining Lightweight LLMs and Deterministic Parsers for E-Commerce and Telecom Intelligence

Julian David Urrego Lancheros
Zenodo (CERN European Organization for Nuclear Research)
Web Data Mining and Analysis
article

Self-Healing Web Data Extraction Pipelines: Combining Lightweight LLMs and Deterministic Parsers for E-Commerce and Telecom Intelligence

Julian David Urrego Lancheros
article en

Abstract

Production web data extraction pipelines for e-commerce and telecommunications intelligence face a persistent operational trade-off: deterministic parsers (XPath/CSS via lxml and parsel) offer near-zero marginal cost and millisecond latency but break frequently under frontend DOM mutations, whereas routing raw HTML through Large Language Models (LLMs) incurs prohibitive token costs, high latency, and exposure to indirect prompt injection. In this technical note (Report ID: UNAL-CS-2026-TN02), we present a self-healing extraction architecture deployed on serverless cloud workers that decouples routine parsing from LLM-assisted recovery. Under normal operation, extraction relies strictly on deterministic selectors validated against strict Pydantic schemas. When structural drift or schema violations trigger an anomaly circuit breaker, the system isolates the mutated HTML region via Lowest Common Ancestor (LCA) DOM pruning, encapsulates the untrusted snippet within cryptographic XML nonce boundaries to mitigate indirect prompt injection, and invokes a lightweight LLM out-of-band solely to synthesize a patched XPath/CSS selector. Once verified against historical schema invariants in a sandbox, the new selector hot-swaps into the production registry. Evaluated across Latin American and Colombian retail and telecommunications portals, this hybrid pipeline resolves layout breaks autonomously while reducing token consumption by over 98% compared to full-page LLM scraping.

Zenodo (CERN European Organization for Nuclear Research)
Universidad Nacional de Colombia (CO)
Web Data Mining and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon