Benchmarking thermoelectric property extraction from scientific text

We compare five methods for extracting thermoelectric property records from paragraphs of the literature (regular expressions, the ChemDataExtractor thermoelectric parsers, MaterialsBERT named-entity recognition with rule-based linking, a single-prompt large language model (LLM) and an LLM agent) on one corpus of 311 papers with one scoring rule, with all methods frozen before the 200-paragraph test sample was annotated. A record counts as correct only if property, value, temperature and material all match the gold standard. On the 160 in-scope test paragraphs (241 core records), the single-prompt LLM reached F1 = 0.79 (95% CI 0.71–0.87) and the agent 0.73 (0.64–0.83), against 0.22 for the regular-expression baseline and 0.06 for the ChemDataExtractor and NER pipelines; both LLM methods recover property, value and temperature with F1 above 0.94, and most of their remaining errors concern the material. We found no evidence of an accuracy improvement from the tested agent configuration, and we release the protocol, code, gold standard and all method outputs.