Structure-preserving segmentation and duplicate folding for German official tax-law texts: design, verification, and negative results
Retrieval over German tax law depends on how the source texts are cut into retrievable units. A fixed character window of about 1,200 characters is the common default. I measured that this default starts a fragment in the middle of a sentence in 64–91 % of cases, depending on the source, and that list items lose their heading and lead-in phrase. Structure-aligned chunking has already been studied for German statutes on the BGB (Prior, Milanova & Schultz 2026). This preprint applies the idea to German tax law in the LeckerMath working database: every official document is segmented along its own structure (statutes: § › Abs. › Satz/Nr. › Buchst.; court decisions: Leitsatz, Tenor, Tatbestand, Gründe › Rn.; administrative instructions: Abschnitt › Rz./Tz.). The structural path and the lead-in phrase of an enumeration are stored in a separate field. Quality is measured by reassembling each document from its fragments: 331,958 of 333,369 documents (99.58 %, 25 Oct 2026) reassemble without loss. Identical text is folded into groups with one vector per group (734,657 groups, 1.53 million copies). Repairs are organised as a cascade (method → check → freeze what passed → next method on the remainder) rather than as one algorithm re-run over the whole corpus. Version 1.0.2 holds 281,126 documents and 7.53 million fragments. I also report negative results: retrieval was not better than with the old segmentation (96.7 % vs 97.1 %), finer parent/child segmentation scored lower in blind-judged games, and a blind sample found defects in 34 of 75 documents of the working database. Part of a series; overview: https://doi.org/10.5281/zenodo.23949487. Files: English version (authoritative) and German version (Deutsche Fassung). Not legal or tax advice.
Authors
- Sergei Sevriukov
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-04
- DOI
- https://doi.org/10.5281/zenodo.23949519
- Primary Topic
- Artificial Intelligence in Law
- Type
- preprint