Two AI Medical Tools — Amie and Mira — Outperformed Doctors In Diagnostics | AI in Education, Industry, and Government

IBL News | New York

Two AI tools have outperformed doctors in diagnostic and treatment decisions, showing the clinical value of LLMs, according to results published in Nature this month:

Google’s Amie produced more precise treatments and investigation plans than humans.
Amie used Google’s Gemini AI model to respond to data provided by actors role-playing patients. The scientists tested Amie against 21 primary care physicians across 100 multi-visit case scenarios in the United Kingdom.Amie matched real physicians in patient-management reasoning capabilities, outperforming human professionals’ reasoning about medication in difficult cases.
German researchers at TUD Dresden University of Technology and Heidelberg University co-developed Mira, a tool that outperformed physicians in analyses of diseases such as pancreatic cancer and pneumonia.
Mira draws on patient data from an electronic health record system and can choose from more than 85,000 options, including ordering diagnostic tests, prescribing medication, and scheduling procedures. The researchers tested it using information from more than 500 emergency department clinical cases, which were provided via chats with AI agents acting as patients.Mira notched a diagnostic accuracy of 87.1 percent across eight conditions, including appendicitis and lung embolism, according to the Nature paper. That compares with 78.1 percent achieved by a panel of six physicians across specialties.

Now, tests are conducted in controlled simulations rather than in real-world clinical use, but independent experts suggest that specialist health AI tools and agents can provide better medical advice and can transform medicine.

IBL News | New York

Two AI tools have outperformed doctors in diagnostic and treatment decisions, showing the clinical value of LLMs, according to results published in Nature this month:

Google’s Amie produced more precise treatments and investigation plans than humans.
Amie used Google’s Gemini AI model to respond to data provided by actors role-playing patients. The scientists tested Amie against 21 primary care physicians across 100 multi-visit case scenarios in the United Kingdom.Amie matched real physicians in patient-management reasoning capabilities, outperforming human professionals’ reasoning about medication in difficult cases.
German researchers at TUD Dresden University of Technology and Heidelberg University co-developed Mira, a tool that outperformed physicians in analyses of diseases such as pancreatic cancer and pneumonia.
Mira draws on patient data from an electronic health record system and can choose from more than 85,000 options, including ordering diagnostic tests, prescribing medication, and scheduling procedures. The researchers tested it using information from more than 500 emergency department clinical cases, which were provided via chats with AI agents acting as patients.Mira notched a diagnostic accuracy of 87.1 percent across eight conditions, including appendicitis and lung embolism, according to the Nature paper. That compares with 78.1 percent achieved by a panel of six physicians across specialties.