GoKawiilAt least one clinical trial is in the works that aims to test the ability of an ‘AI doctor’ to take a patient’s information and propose a diagnosis.Credit: andrei_r/Getty
‘AI outperforms doctors in emergency room tasks’
‘Google AI has better bedside manner than human doctors’
Headlines such as these that tout the abilities of artificial-intelligence tools to surpass the skills of physicians are becoming more common.
An AI revolution is brewing in medicine. What will it look like?
But an advanced large language model (LLM) beating a physician at a single task doesn’t necessarily mean that AI is ready to take over medicine in the real world. Nature spoke to researchers studying the use of AI in health care to understand which ‘AI doctors’ have shown the most promise so far — and when such tools might take command of medical diagnoses. Some scientists point out that various AI systems are already handling simple medical tasks, such as taking notes and even renewing prescriptions, but they say that physicians can never fully be replaced by machines.
“Medicine is messy and patients don’t always have textbook stories to tell,” says David Wu, a resident physician who studies AI at Harvard Medical School in Boston, Massachusetts. “I don’t think we’ve proven that these systems can handle that mess.”
Up to the test
Still, a few demonstrations have got researchers excited about the AI revolution brewing in medicine. One study, published in April in the journal Science1, concluded that an advanced LLM performed better than physicians when evaluating the conditions of people visiting the emergency department at a Boston hospital. When the AI model — called o1 and developed by OpenAI in San Francisco, California — reviewed the information recorded by hospital staff members during a visit, it got the diagnosis correct or almost correct in 67% of cases, compared with around 50–55% for the two human doctors who participated in the experiment.
Google AI has better bedside manner than human doctors — and makes better diagnoses
... continue reading