Has AI bitten off more than it can chew with emergency care?
Technology seems to be storming the medical sector, although new US research shows generative AI still needs to find the right balance between prescribing too little and too much care.
The new research, which was led by Christopher Williams, MD, from the University of California, focuses on the use of ChatGPT’s – an AI chatbot that was launched in 2022 – in emergency care settings. Experts found that the tech tends to recommend unnecessary medicine and care, including imaging.
Image: BoliviaInteligente
‘This is a valuable message to clinicians not to blindly trust these models,’ Williams said after he published his findings. ‘ChatGPT can answer medical exam questions and help draft clinical notes, but it’s not currently designed for situations that call for multiple considerations, like the situations in an emergency department.’
In the study, Williams had two ChatGPT’s provide recommendations a medical expert would make after an examination within the emergency department. This could include going for an X-ray, admitting the patient or prescribing antibiotics. Overall, researchers curated a set of 1,000 emergency department visits for these decisions. The sets had the same ratio of ‘yes’ to ‘no’ responses for decisions on admission, radiology and antibiotics.
In the study, Williams had ChatGPT’s give recommendations a medical expert usually would after an examination. Both machines were fed notes on appointments and could respond either ‘yes’ or ‘no’ to care methods such as X-rays, admitting patients or prescribing antibiotics. Overall, researchers curated a set of 1,000 emergency department visits for these decisions.
It is worth noting that two different versions of ChatGPT’s were used including the 3.5 version and 4.
However, the AI models tended to recommend services more often than what was actually needed. Williams remarked that this happened because models are trained on the internet – a system that wasn’t created to answer emergency medical questions, but to send readers to a doctor who can.
The results translate to ChatGPT-4 and ChatGPT-3.5 being 8% and 24% less accurate than resident physicians, respectively.
Concluding the study, Williams claimed ‘There’s no perfect solution’ when it comes to creating a successful AI technology for the medical sector. Although, he added: ‘Knowing that models like ChatGPT have these tendencies, we’re charged with thinking through how we want them to perform in clinical practice.’
The full results of the study can be found here.
In related news:
Opinion: The UK invented the Welfare State, now AI can help protect it
