As HIMSS24 Ends, Talk Shifts to AI Accountability

AI adoption is fast outpacing governance, and with memories (or nightmares) of EMR adoption and the Change Healthcare outage on everyone’s mind, the pressure is on to set up guardrails

With HIMSS24 in the rear-view mirror, one of the biggest takeaways from the conference was the energy and attendance. Healthcare’s movers and shakers were here, at least for a day or two, and they were making deals and forging partnerships.

And while AI tools and programs were dominating the discussion and deals, a lot of talk was centered on Ai accountability.

While the year-old Coalition for Health AI (CHAI) made early news with its announcement of advisory boards and partnerships, Microsoft unveiled its own Trustworthy & Responsible AI Network (TRAIN), an intriguing collaboration of health systems and federal representatives that aims to create more structure in the move toward governance.

“When it comes to AI’s tremendous capabilities, there is no doubt the technology has the potential to transform healthcare. However, the processes for implementing the technology responsibly are just as vital,” David Rhew, MD, Microsoft’s  global chief medical officer and vice president of healthcare, said in a press release. “By working together, TRAIN members aim to establish best practices for operationalizing responsible AI, helping improve patient outcomes and safety while fostering trust in healthcare AI.”

Rhew also commented on LinkedIn on how CHAI and TRAIN will co-exist.

“One way to look at it is that CHAI focuses on ‘the what,’” he wrote. “What is responsible AI (RAI)? What are the RAI principles and standards? ... while TRAIN focuses on ‘the how.’ How does one operationalize RAI? How can organizations collaborate in a privacy-preserving manner, such that data and IP are not exposed? How can we ensure that low-resource settings are able to apply RAI?
Simply put, CHAI helps develop the RAI standards, while TRAIN helps organizations implement them, through the use of technology-based RAI tools and guardrails.”

That post hints at the understanding that while everyone agrees the industry needs guardrails, how we get there will be a challenge. And with all the announcements and talk at HIMSS24 of health systems and hospitals launching AI programs, the urgency to create standards is growing by the day.

Many also don’t want the process to mirror EMR adoption two decades ago. The healthcare industry wasn’t prepared to embrace electronic medical records, leading to more than a few nightmares in implementation and the need for meaningful use guidelines. Now AI adoption is fast outpacing governance, and executives are in some ways nostalgic for the guidance offered by meaningful use.

Add to that the concern around the recent Change Healthcare cybersecurity attack some three weeks ago. The outage filtered into many a conversation at HIMSS24, creating a stir around the section of the exhibit hall given over to cybersecurity companies. While the incident has been tied back to a ransomware gang, more than a few people have wondered whether unchecked use of AI could lead to more data breaches – or whether AI could be used as a tool against attacks.

For example, Nordic Consulting announced at HIMSS24 a partnership with Microsoft Azure and Amazon Web Services to launch a Cloud Innovation Lab, addressing requests from healthcare organizations for help in expanding and managing their data storage capabilities, in part because of the increase in AI programs. Brijeet Akula, a Principal Architect at Nordic Consulting, said company was seeing a lot of business from hospitals and health systems spooked by the outage.

“They have sped up their desire to explore more security methods,” he noted.

So while security may be an outlier, it will add to the urgency to create guardrails around AI. The Biden Administration has set its course with an October 2023 Executive Order on AI and announcements of actions to come, but CHAI and TRAIN offer proof that there will be more players in the game.

Like CHAI, TRAIN has an impressive (and in some cases overlapping) roster. Along with healthcare technology company TrueBridge, participating health systems include AdventHealth, Advocate Health, Boston Children’s Hospital, Cleveland Clinic, Duke Health, Johns Hopkins Medicine, Mass General Brigham, MedStar Health, Mercy, Mount Sinai Health System, Northwestern Medicine, Providence, Sharp HealthCare, University of Texas Southwestern Medical Center, University of Wisconsin School of Medicine and Public Health and Vanderbilt University Medical Center, and the organization will be working with OCHIN, a non-profit innovation center with strong ties to Epic.

So even as these organizations are deploying AI in several, if not hundreds, of use cases, the talk in the background is around setting up accountability as soon as possible—especially before those use cases start involving patients.

“Even the best healthcare today still suffers from many challenges that AI-driven solutions can substantially improve,” Peter J. Embí, MD, MS, a professor and chair of the Department of Biomedical Informatics (DBMI) and senior vice president for research and innovation at Vanderbilt University Medical Center, said in the press release.”However, just as we wouldn’t think of treating patients with a new drug or device without ensuring and monitoring their efficacy and safety, we must test and monitor AI-derived models and algorithms before and after they are deployed across diverse healthcare settings and populations, to help minimize and prevent unintended harms. It is imperative that we work together and share tools and capabilities that enable systematic AI evaluation, surveillance and algorithm vigilance for the safe, effective and equitable use of AI in healthcare.”