AI Chatbots in Pediatric Orthopedics: How Accurate Are Their Answers to Parents' Questions on Bowlegs and Knock Knees?

Area of Science:

• Pediatric Orthopedics
• Artificial Intelligence
• Health Informatics

Background:

• Large-language models (LLMs) provide quick health information but may lack accuracy and detail.
• Parents frequently seek information on pediatric knee deformities like bowlegs and knock knees.
• Chatbot reliability is critical for parental decisions regarding seeking medical treatment.

Purpose of the Study:

• To evaluate how AI chatbots address parental concerns about pediatric knee deformities.
• To compare the performance of ChatGPT, Gemini, and Copilot in providing orthopedic information.

Main Methods:

• Developed 20 standardized questions on bowlegs and knock knees.
• Queried three AI chatbots (ChatGPT, Gemini, Copilot) with the same questions.
• Five pediatric orthopedic surgeons rated chatbot responses for accuracy, clarity, comprehensiveness, and misleading information.

Main Results:

• ChatGPT and Gemini provided more accurate and comprehensive responses than Copilot (p < 0.05).
• No significant differences were observed in response clarity or the likelihood of providing incorrect information.
• ChatGPT and Gemini offered more detailed and precise answers, while Copilot was comparable in clarity but less thorough.

Conclusions:

• AI chatbots show varying performance in pediatric orthopedic information delivery.
• ChatGPT and Gemini demonstrated superior accuracy and comprehensiveness compared to Copilot.
• Continuous human oversight and validation are essential for using AI chatbots in pediatric healthcare.