Evaluation of ChatGPT’s responses to frequently asked questions from osteosarcoma patients: A descriptive cross-sectional study

To evaluate the clinical appropriateness of ChatGPT's responses to questions frequently asked by osteosarcoma patients and their families. Ten questions frequently asked by osteosarcoma patients and their families were identified. Each question was submitted to OpenAI's GPT-5-based ChatGPT (August 2025 version) using separate user accounts. Two orthopedic oncology specialists independently evaluated the responses for clinical appropriateness using a 4-point Likert scale. Interrater agreement was analyzed with weighted Cohen kappa. Interrater agreement was found to be substantial (K = 0.667). One response was rated as an excellent response that did not require clarification, 5 responses were rated as satisfactory responses that required minimal clarification, and 4 responses were rated as satisfactory responses that required moderate clarification. There were no unsatisfactory responses requiring substantial clarification. ChatGPT's responses to osteosarcoma-related questions were found to be largely clinically appropriate. Nevertheless, given its limitations, artificial intelligence should be regarded as a supportive tool that requires physician oversight.