Identification of concepts of importance for the assessment of internal validity of <i>in vitro</i> toxicology studies using a modified Delphi technique

&lt;div&gt; &lt;h3&gt;Context&lt;/h3&gt; &lt;p&gt;&lt;em&gt;In vitro&lt;/em&gt;&nbsp;toxicity studies are increasingly being included as evidence in systematic reviews and chemical risk assessments. INVITES-IN, a tool for assessing the internal validity of&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;studies, is under development in a process consisting of four consecutive studies. Study One in the creation of INVITES-IN was the development of an &ldquo;item bank&rdquo; database of 405 concepts (&ldquo;items&rdquo;) of potential relevance for assessing the internal validity of&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;toxicity studies. The items were gathered from both focus group discussions and a purposive literature sample. In this paper we present the second study in the creation of INVITES-IN, i.e. the methods and results for identifying items for consideration when assessing the potential for bias in an&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;study.&lt;/p&gt; &lt;/div&gt; &lt;div&gt; &lt;h3&gt;Method&lt;/h3&gt; &lt;p&gt;A two-round digital Delphi survey, followed by online Delphi panel discussions guided by a moderator, was performed. The Delphi participants were experienced with both&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;models and systematic review methods.&lt;/p&gt; &lt;/div&gt; &lt;div&gt; &lt;h3&gt;Results&lt;/h3&gt; &lt;p&gt;Fifteen experts completed both Delphi rounds, and thirteen participated in a guided Delphi panel discussion. Of the 405 items in the bank, the experts agreed that 372 should be considered when assessing the potential for bias in an&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;study. Items gathered from both the literature sample and the focus group discussions (Study One) were considered to be important for the assessment of the potential for bias in an&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;study; 83&ndash;100% of the items collected from the literature sample were identified to be important and 91% (127) of the new items discovered in the focus group discussions of Study One were identified to be important.&lt;/p&gt; &lt;/div&gt; &lt;div&gt; &lt;h3&gt;Discussion&lt;/h3&gt; &lt;p&gt;The 372 retained items will be interpreted into a manageable set of study appraisal criteria and a supporting guidance that will constitute the INVITES-IN study appraisal tool. In terms of lessons for tool development, the high retention of items included in tools designed for assessment of human and animal studies to&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;studies suggests that many validity concepts are generally applicable across multiple study designs. Therefore, tool development processes should benefit from drawing on assessment tools outside the immediate domain of interest. Tool development would also likely benefit from supplementing literature reviews with focus group discussions, as our results demonstrate that the use of focus group discussions with domain experts was a pragmatic and valuable approach to increasing coverage of items in a tool development process.&lt;/p&gt; &lt;/div&gt; &lt;div&gt; &lt;h3&gt;Conclusion&lt;/h3&gt; &lt;p&gt;In conclusion, this study demonstrates the value of using rigorous methods to ensure a comprehensive dataset as the starting point for creation of an assessment tool, though the direct application of Delphi methods to item banks may be an unnecessary step in tool development.&lt;/p&gt; &lt;/div&gt;

Authors

Institutions

Publication Details

Journal
Evidence-Based Toxicology
Published
2025-09-05
DOI
https://doi.org/10.1080/2833373x.2025.2551013
Citations
1
Primary Topic
Delphi Technique in Research
Type
article
Field-Weighted Citation Impact
4.20
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Identification of concepts of importance for the assessment of internal validity of in vitro toxicology studies using a modified Delphi technique

Trine Husøy, Heather Ames, Miles P. Davenport, Gunn Elisabeth Vist et al.
1 citations
Evidence-Based Toxicology
Delphi Technique in Research
4.20
article

Identification of concepts of importance for the assessment of internal validity of in vitro toxicology studies using a modified Delphi technique

Trine Husøy, Heather Ames, Miles P. Davenport, Gunn Elisabeth Vist, Pilar Prieto, Lucina E. Lizarraga, Gro Haarklou Mathisen, Andrew A. Rooney, Amy Wang, Marc Audebert, Jennifer H. Olker, Paul Whaley, Camilla Svendsen, Joshua F. Robinson, Olga Tcheremenskaia, Ellen Bruzell, Ingrid L. Druwe, Ursula Gundert‐Remy, Eliana Spilioti, Sebastian Hoffmann, René Geci, Anastasia Spyropoulou, Carlijn R. Hooijmans, Adriano Sebollela, Johanna Zilliacus, Nicola M. Smith, Helena T. Högberg, Angela Bearth, A. Bernhard, Emanuela Testai, Anna Beronius, Thomas Härtung, Emma Di Consiglio, Erin E. Yost
article en
1 citations

Abstract

&lt;div&gt; &lt;h3&gt;Context&lt;/h3&gt; &lt;p&gt;&lt;em&gt;In vitro&lt;/em&gt;&nbsp;toxicity studies are increasingly being included as evidence in systematic reviews and chemical risk assessments. INVITES-IN, a tool for assessing the internal validity of&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;studies, is under development in a process consisting of four consecutive studies. Study One in the creation of INVITES-IN was the development of an &ldquo;item bank&rdquo; database of 405 concepts (&ldquo;items&rdquo;) of potential relevance for assessing the internal validity of&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;toxicity studies. The items were gathered from both focus group discussions and a purposive literature sample. In this paper we present the second study in the creation of INVITES-IN, i.e. the methods and results for identifying items for consideration when assessing the potential for bias in an&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;study.&lt;/p&gt; &lt;/div&gt; &lt;div&gt; &lt;h3&gt;Method&lt;/h3&gt; &lt;p&gt;A two-round digital Delphi survey, followed by online Delphi panel discussions guided by a moderator, was performed. The Delphi participants were experienced with both&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;models and systematic review methods.&lt;/p&gt; &lt;/div&gt; &lt;div&gt; &lt;h3&gt;Results&lt;/h3&gt; &lt;p&gt;Fifteen experts completed both Delphi rounds, and thirteen participated in a guided Delphi panel discussion. Of the 405 items in the bank, the experts agreed that 372 should be considered when assessing the potential for bias in an&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;study. Items gathered from both the literature sample and the focus group discussions (Study One) were considered to be important for the assessment of the potential for bias in an&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;study; 83&ndash;100% of the items collected from the literature sample were identified to be important and 91% (127) of the new items discovered in the focus group discussions of Study One were identified to be important.&lt;/p&gt; &lt;/div&gt; &lt;div&gt; &lt;h3&gt;Discussion&lt;/h3&gt; &lt;p&gt;The 372 retained items will be interpreted into a manageable set of study appraisal criteria and a supporting guidance that will constitute the INVITES-IN study appraisal tool. In terms of lessons for tool development, the high retention of items included in tools designed for assessment of human and animal studies to&nbsp;&lt;em&gt;in vitro&lt;/em&gt;&nbsp;studies suggests that many validity concepts are generally applicable across multiple study designs. Therefore, tool development processes should benefit from drawing on assessment tools outside the immediate domain of interest. Tool development would also likely benefit from supplementing literature reviews with focus group discussions, as our results demonstrate that the use of focus group discussions with domain experts was a pragmatic and valuable approach to increasing coverage of items in a tool development process.&lt;/p&gt; &lt;/div&gt; &lt;div&gt; &lt;h3&gt;Conclusion&lt;/h3&gt; &lt;p&gt;In conclusion, this study demonstrates the value of using rigorous methods to ensure a comprehensive dataset as the starting point for creation of an assessment tool, though the direct application of Delphi methods to item banks may be an unnecessary step in tool development.&lt;/p&gt; &lt;/div&gt;

Evidence-Based ToxicologyVol. 3(1)
Partners In Health (US), Environmental Protection Agency (US), National Center for Toxicological Research (US), Institut National de la Recherche Agronomique (MA), Thorlabs (Germany) (DE), Universidade de Ribeirão Preto (BR), Norwegian Institute of Public Health (NO), Johns Hopkins University (US), Radboud University Nijmegen (NL), Universidade de São Paulo (BR), University of California, San Francisco (US), Environmental Protection Agency (GH), University of Konstanz (DE), Nordic Institute of Dental Materials (NO), Istituto Superiore di Sanità (IT), European Commission (BE), Norwegian Institute of Marine Research (NO), University Medical Center (US), Radboud University Medical Center (NL), Karolinska Institutet (SE), UNSW Sydney (AU), Kleijnen Systematic Reviews (United Kingdom) (GB), National Institute of Environmental Health Sciences (US), Partners In Care (US), Toxicologie Alimentaire (FR), Molecular Partners (Switzerland) (CH), National Center for Environmental Assessment (EPA) (US), Joint Research Centre (IT), M&I/Partners (NL), National Institute of Health (AM), Benaki Phytopathological Institute (GR), Society of Environmental Toxicology and Chemistry (US), BH Consulting (Ireland) (IE), Clinics Hospital of Ribeirão Preto (BR), Environmental Protection Agency (IE), Brazilian College of Animal Reproduction (BR), National Institute of Nutrition (IN), Lancaster University (GB), Charité - Universitätsmedizin Berlin (DE), RWTH Aachen University (DE)
Openalex Percentile: Top 6%
Delphi Technique in Research
4.20
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.