With previous clinical decision support tools, she said, there was a better understanding of the algorithms being implemented and clearer data sources. But with the newer generation of AI tools, “these are somewhat of a black box in the clinical setting. So, if I’m seeing a patient and I have some sort of decision support pop up, I may not know what the algorithm is in the background. I may not know if the data that was put into the algorithm is accurate,” she said.
She also questioned what sort of feedback loops were in place for these tools. “When things go wrong, are we going to have a way to report them? Are we going to get them fixed if we report them? What are those mechanisms? And I think having processes in place is going to be so important for us to feel comfortable using these tools in the clinical setting. We absolutely recognize the value of them. We know that they can be really useful in a clinical setting. But there has to be that trust that is gained as we continue to roll them out.”
Lozovatsky added that organizations must monitor for hallucinations and drift if these models are meant to impact patient care.
Dr. Kathryn King, chief medical information officer and co-director of the national telehealth center of excellence at the Medical University of South Carolina Health, said that adopting AI in clinical settings will likely require a reconfiguration of workflows that start with the electronic health records system.
“The immaturity that I’m seeing in AI solutions in the clinical space is really that they are a point solution, if you will, at a single point in a long clinical workflow, typically within the EHR,” King said. “That one point in that workflow doesn’t really solve a clinician’s whole problem.”
Though the HITECH Act of 2009 was integral to pushing healthcare organizations to adopt EHR systems, the process at its crux was about digitizing paper records. A number of the hallmarks of the EHR are relics of paper charting.
“I think the next iteration of this has to be asking the questions, why do we do things the way we have for all of these years, and how do we move toward a different workflow that perhaps is more voice utilization but in a way that makes sense to each individual? And to me, the focus has been so much on, ‘this is the tool, this is the cool thing,’ instead of taking a step back and really reinventing the way that we deliver care,” Lozovatsky said.
