Comparison of ChatGPT-3.5 and Google Bard Performance on Turkish orthopaedics and traumatology national board examination

This study ia a cross-sectional study to evaluate and compare the responses of two chatbots to compare the performance of ChatGPT-3.5 and Google Bard on the Turkish Orthopaedics and Traumatology National Board Examination. The questions of the Turkish Orthopaedics and Traumatology National Board Examination were asked to the chatbots one by one to have them indicate what the correct answer was and determine the difficulty level of the questions. The examination consists of 100 questions; 92 were included in the study. It was found that ChatGPT-3.5 answered 54.3% of the questions correctly, while Google Bard answered 45.7% of the questions correctly. When the correlation of difficulty and accuracy between the two AI models was evaluated, it was found that both were poorly correlated between the two different AI models (r=0.290 and p=0.005 for difficulty; r=0.314 and p=0.002 for accuracy). Both language models showed about 50% success on the Turkish Orthopaedics and Traumatology National Board Examination. Both found similar levels of difficulty in the questions.