Artificial intelligence in perioperative medicine education: A feasibility test of case-based learning

Area of Science:

• Medical Education
• Artificial Intelligence in Medicine

Background:

• Artificial intelligence (AI) is increasingly used in medicine, with large language models (LLMs) like ChatGPT offering potential benefits for perioperative medicine education and practice.
• Concerns exist regarding LLM accuracy, specifically the risk of generating hallucinations or factually incorrect information.

Purpose of the Study:

• To assess the feasibility of using an LLM-enabled platform for case-based education in perioperative clinical scenarios.
• To evaluate the accuracy and identify hallucinations in LLM responses within a perioperative medicine context.

Main Methods:

• Five core perioperative cases were developed and uploaded to a custom LLM platform.
• Anesthesiology trainees interacted with the AI, asking free-text questions about case details (history, examination, investigations).
• AI-generated question-and-answer pairs were independently reviewed for inappropriate responses, including hallucinations.

Main Results:

• The LLM demonstrated high accuracy, responding appropriately to 99.3% (543/547) of questions.
• No instances of hallucination were observed; in rare cases of inappropriate responses, the LLM refrained from answering rather than generating incorrect information.
• The LLM effectively provided information based on the provided case studies.

Conclusions:

• LLM-powered platforms show significant potential for accurately delivering case-based perioperative medicine educational content.
• The study indicates that LLMs can be a reliable tool for medical education, minimizing the risk of misinformation.