Closer Look
ChatGPT in medicine misses the mark in early tests. Here's why
Photo illusration by Leon Neal/Getty Images
ChatGPT is a pretty good student. The AI tool got more than 80% of questions on a board exam right, something even the best students need years to achieve. But moving into the real world of medicine was much different, a Stanford study found. Almost 60% of ChatGPT's answers either disagreed with human specialists or offered information that wasn't clearly relevant. That's not surprising given specialists' reliance on medical records and ChatGPT's internet-sourced knowledge.
That highlights a bigger problem: The early testing of the model examined only its textbook knowledge, not its ability to help doctors make faster, better decisions in real-life situations, particularly messier ones when information is incomplete. "We still need to figure out what the evidence bar is to decide where they are useful and where they are not," said Philip Payne of Washington University in St. Louis. STAT's Casey Ross explains.
insuranceMedicare Advantage insurers get more time Health insurance companies got a big win Friday. After the companies' intense lobbying, the Biden administration said it would phase in changes to Medicare Advantage's risk adjustment program over three years, rather than deploying them all at once. What's at stake are the higher payments insurers get from the government by coding more illnesses. Medicare officials say insurers sometimes exploit the system by saying that their members are sicker than they are.
To fix that problem, Medicare is removing or consolidating codes for certain conditions that lead to inflated payments. Insurers said the administration's initial proposal amounted to slashing Medicare benefits, but CMS Administrator Chiquita Brooks-LaSure disagreed. "Any reductions in benefits would be decisions made by companies, not because of [Medicare] policies." STAT's John Wilkerson has more.
PHARMAFDA cites multiple failures at eye-drop manufacturer linked to serious infections The reports in February were horrifying: Drug-resistant infections were being traced to over-the-counter eye drops that caused serious illness from the bacteria Pseudomonas aeruginosa, eventually including systemic infections, permanent vision loss, and three deaths. Now, STAT's Ed Silverman writes in an exclusive story, an FDA inspection report says the manufacturer of the eye drops failed to follow numerous procedures to make sure its products did not become contaminated.
During a recent visit to Global Pharma Healthcare in Tamil Nadu, India, FDA inspectors found the company failed to take basic steps during sterilization, relied on deficient manufacturing processes and methods to ensure and verify sterility, and did not have adequate systems to create aseptic conditions, among other issues, the inspection report noted. The case follows the deaths of dozens of children linked to cough syrup made in India, renewing long-standing concerns about the Indian pharmaceutical industry. Read more.
No comments