Can Reasoning LLMs Synthesize Complex Climate Statements?

Accurately synthesizing climate evidence into concise statements is crucial for policy making and fostering public trust in climate science.Recent advancements in Large Language Models (LLMs), particularly the emergence of reasoning-optimized variants, which excel at mathematical and logical tasks, present a promising yet untested opportunity for scientific evidence synthesis.We evaluate stateof-the-art reasoning LLMs on two key tasks:(1) contextual confidence classification, assigning appropriate confidence levels to climate statements based on evidence, and (2) factual summarization of climate evidence, generating concise summaries evaluated for coherence, faithfulness, and similarity to expertwritten versions.Using a novel dataset of 612 structured examples constructed from the Sixth Assessment Report (AR6) of the Intergovernmental Panel on Climate Change (IPCC), we find reasoning LLMs outperform generalpurpose models in confidence classification by 8 percentage points in accuracy and macro-F1 scores.However, for summarization tasks, performance differences between model types are mixed.Our findings demonstrate that reasoning LLMs show promise as auxiliary tools for confidence assessment in climate evidence synthesis, while highlighting significant limitations in their direct application to climate evidence summarization.This work establishes a foundation for future research on the targeted integration of LLMs into scientific assessment workflows.Code and

Authors

Institutions

Publication Details

Published
2025-01-01
DOI
https://doi.org/10.18653/v1/2025.climatenlp-1.21
Primary Topic
Semantic Web and Ontologies
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Can Reasoning LLMs Synthesize Complex Climate Statements?

Yucheng Lu
Semantic Web and Ontologies
article

Can Reasoning LLMs Synthesize Complex Climate Statements?

Yucheng Lu
article en

Abstract

Accurately synthesizing climate evidence into concise statements is crucial for policy making and fostering public trust in climate science.Recent advancements in Large Language Models (LLMs), particularly the emergence of reasoning-optimized variants, which excel at mathematical and logical tasks, present a promising yet untested opportunity for scientific evidence synthesis.We evaluate stateof-the-art reasoning LLMs on two key tasks:(1) contextual confidence classification, assigning appropriate confidence levels to climate statements based on evidence, and (2) factual summarization of climate evidence, generating concise summaries evaluated for coherence, faithfulness, and similarity to expertwritten versions.Using a novel dataset of 612 structured examples constructed from the Sixth Assessment Report (AR6) of the Intergovernmental Panel on Climate Change (IPCC), we find reasoning LLMs outperform generalpurpose models in confidence classification by 8 percentage points in accuracy and macro-F1 scores.However, for summarization tasks, performance differences between model types are mixed.Our findings demonstrate that reasoning LLMs show promise as auxiliary tools for confidence assessment in climate evidence synthesis, while highlighting significant limitations in their direct application to climate evidence summarization.This work establishes a foundation for future research on the targeted integration of LLMs into scientific assessment workflows.Code and

New York University (US)
Climate action
Openalex Percentile: Top 90%
Semantic Web and Ontologies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.