Washington's top health IT official talks AI guardrails

Screenshot 2024-03-13 at 5.22.25 PM

I've covered health tech for years, and I've never witnessed as much anxiety and excitement about budding regulations as I have in recent months as Washington commits to curbing bias and error in health-focused AI. There's dramatic tension between health care's appetite for new technology and fears of patient harm if it's misused, and the regulatory panels at industry conventions are packed with startup founders, investors and hospital leaders alike eager for clarity as disparate federal agencies and industry groups cobble together guardrails before the technology outpaces them. My colleague Casey Ross sat down with National Coordinator for Health Information Technology Micky Tripathi, one of the most influential health data regulators, about what's ahead.

"I will say the emergence of ChatGPT surprised the heck out of everyone," Tripathi says. "You talk to Microsoft, and they were like, 'We had no idea it was going to explode like it did.' As you started to see that, and then recognize that there are going to be doctors who are absolutely going to alt-tabbing out of their EHRs and typing stuff into [ChatGPT], then you start to realize this is bigger than this little inventory of [AI models] we have in our department." Read more from their conversation.

AI clinical notes are taking off. Validation is a challenge.

Hospitals buying large language model tech to listen in to doctors' appointments and automatically populate electronic medical records are weighing those products' potential for saving time and easing documentation burdens against the risk of introducing error or even hallucinating, my colleagues Katie Palmer and Casey report. And though they're racing to adopt the technology, health systems are still not sure how to measure their accuracy or safety, or even approve them for use in medical settings, they write. Read more, including perspectives from informatics experts as well as documentation tech vendor Abridge.

What does human in the loop actually mean?

I always ask AI founders how they ensure their products don't harm patients. Their defense, they typically say, is keeping a human in the loop — often that means doctors must review the AI model's suggestions before acting on them, whether it's dispatching an email derived from an AI template or following up with certain patients based on abnormal radiological scans. But I've also seen wide variation in how closely humans are involved, depending on the use case; we also don't have a handle on how heavily humans are influenced by an AI model's output, even if it's wrong.

There's no federal standard for what "human in the loop" should mean; it's also not yet clear who's liable (whether it's the hospitals or AI developers) if there's harm. Amid the uncertainty, "'human in the loop' is a stop-gap solution to make progress and punt questions around trust, liability, cross population performance, and a full accounting of the consequences of algorithm-guided work to a later date," Stanford Health Care's chief data scientist Nigam Shah told STAT. "It's basically a get out of jail free card in my view." Read more, and reach out with your thoughts.

Startups

Catching up with Proscia's David West

This week I chatted with David West, chief executive of Philadelphia-based AI-guided pathology company Proscia. He told me the company's capitalizing on the pathology industry's gradual move toward digitization; the company also recently earned FDA clearance to market its software for primary diagnosis.

When Proscia approaches customers — typically independent or hospital-based diagnostic labs — they're often just beginning to transition away from analog lab tools. "The biggest competitor right now is the microscope," West told me. "We think this is the early days of a market transformation, in a field that's stuck on glass and microscopes."

Lizzy's device digest

Researchers build throat patch for voice disorders

Screenshot 2024-03-12 at 2-11-02 PM-png

A team at the University of California, Los Angeles has designed an adhesive patch that can turn throat movements into speech — a step toward addressing challenges faced by patients with dysfunctional vocal folds, Lizzy Lawrence writes. Detailed in a new paper in Nature Communications, the patch converts throat muscle movements into electrical signals and sends them to a machine-learning model that matches them to specific words. Eventually, the technology and adhesive material — developed by UCLA researchers in 2021 — could "help people speak without vocal folds," lead researcher Jun Chen told Lizzy. Read more.

conference circuit

#HIMSS 2024 news roundup: Deals and contracts

The health IT convention in Orlando appears to be relatively quiet, raising questions among conference regulars about whether its proximity in time to ViVE may have thinned out its attendee list. Still, a few small newsmakers: AI documentation startup Abridge announced a new contract with the University of California, Irvine's health system; Arkansas Heart Hospital is expanding its contract with Oracle Health's medical record and patient accounting software across more clinics and hospitals; and Siemens Healthineers has a new Apple Vision Pro app to let clinicians, students and patients explore holograms of the human body. What did I miss? Drop me a line.