Does AI write good science, and does it know whether it can? Evaluating the ability of Generative AI to write and evaluate scientific text

Our project assesses the ability of Generative AI in two key areas. 1. How well does Generative AI construct evidence based scientific text compared to humans? Our assessment includes the impact of different of AI tools, length of prose and few-shot prompting. 2. How well can AI grade and provide feedback against a set of criteria when compared to human assessors. This includes an evaluation of the ability of AI to assess its own product.

Publication Details

Journal
ORCA Online Research @Cardiff (Cardiff University)
Published
2025-01-09
Primary Topic
Immunotoxicology and immune responses
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Does AI write good science, and does it know whether it can? Evaluating the ability of Generative AI to write and evaluate scientific text

ORCA Online Research @Cardiff (Cardiff University)
Immunotoxicology and immune responses
article

Does AI write good science, and does it know whether it can? Evaluating the ability of Generative AI to write and evaluate scientific text

article en

Abstract

Our project assesses the ability of Generative AI in two key areas. 1. How well does Generative AI construct evidence based scientific text compared to humans? Our assessment includes the impact of different of AI tools, length of prose and few-shot prompting. 2. How well can AI grade and provide feedback against a set of criteria when compared to human assessors. This includes an evaluation of the ability of AI to assess its own product.

ORCA Online Research @Cardiff (Cardiff University)
Quality Education
Openalex Percentile: Top 45%
Immunotoxicology and immune responses
Open Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Does AI write good science, and does it know whether it can? Evaluating the ability of Generative AI to write and evaluate scientific text · ORCA Online Research @Cardiff (Cardiff University) (2025) | TGRS Research Map | TGRS