Chatbot's potential as a diagnostic partner
Speaking of AI research, a new letter in JAMA Network Open suggests large language models aren't necessarily always terrible at clinical reasoning, and could outperform doctors in certain limited circumstances.
Researchers at Beth Israel Deaconess Medical Center compared Chat GPT-4 — a publicly available tool — to clinicians by entering in a series of example medical cases along with an identical prompt 100 times, and asking for the likelihood that a patient had a certain diagnosis given patient's presentation. They also asked it to update its estimates given certain test results like mammography for breast cancer. Researchers compared the probabilistic estimates to those gathered from a national survey of more than 550 human practitioners. Depending on the case, the chatbot's estimates were sometimes more accurate than humans'.
"Humans struggle with probabilistic reasoning, the practice of making decisions based on calculating odds," author Adam Rodman said. But that's just one of many parts of the diagnostic process, he added, and authors "chose to evaluate probabilistic reasoning in isolation because it is a well-known area where humans could use support."
Interestingly, if the patients' test results were positive, the chatbot was sometimes more accurate, similarly more accurate or less accurate than humans. But when test results were negative, the chatbot was more accurate than humans in all test cases. Rodman said humans often associate negative test results with higher risks, potentially leading to over treatment or more tests. Thoughts on the research and its implications are welcome.
Lizzy's device digest
GE HealthCare CEO to chair AdvaMed
Medical device lobbying giant AdvaMed has named Peter Arduini, who leads GE HealthCare, the chair of its board of directors, Lizzy Lawrence writes. The announcement coincides with the establishment of the lobbying group's new division focused on policy issues for medical imaging technology; that group was formed just a couple months after it launched a digital health task force. The moves come as FDA heightens its evaluation of digital health tools, Lizzy writes. Read more.
No comments