Evaluation of Artificial Intelligence's Ability to Explore Literature on Orthopedic Navigation and Related Surgical Anatomy

Introduction Artificial intelligence has recently garnered increased research interest in orthopedics; yet, its role in the exploration of orthopedic literature remains unknown. We aimed to evaluate the ability of ChatGPT and ScholarGPT to identify and outline literature on orthopedic navigation and the visualization of related surgical anatomy. Methods We asked ChatGPT and ScholarGPT to list and summarize five studies: 1) about augmented reality-based navigation in orthopedic surgery, 2) about how well augmented reality-based navigation enabled anatomical accuracy in orthopedic surgery, and 3) which compared augmented reality-based navigation with another navigation technique in orthopedic surgery. Regarding each query, we evaluated how many studies were correctly detected and accurately summarized. Results ChatGPT scored excellently in identifying studies across all three queries. However, its performances in accurately summarizing these studies were 40%, 60%, and 60% respectively. On the other hand, ScholarGPT's performances in identifying were 60%, 40%, and 0%, respectively, while in summarization, they were 0%, 20%, and 0%, respectively. Both platforms exhibited bias in favor of augmented reality-based navigation across all queries. Conclusion ChatGPT and ScholarGPT are not yet able to provide researchers with reliable data from the literature on orthopedic navigation and related surgical anatomy. Ongoing artificial intelligence development may essentially reinforce these platforms' potential to play a more significant role in orthopedic research.