Healthcare’s Data Deluge is Intensifying Misdiagnosis

Healthcare's Data Deluge is Intensifying Misdiagnosis
Eli Ben-Joseph, co-founder and CEO of Regard

In 2019, my aunt was admitted to the hospital with heart failure. It was a harrowing experience for everyone involved. 

My family, desperate for answers, waited for hours for a doctor to check-in. When they finally did, their time was limited and felt rushed. But the whole time, we couldn’t shake the uncomfortable feeling that we were nuisances, that our welcome had run its course. We were stuck in that cycle for days until my aunt was deemed stable enough to be discharged. Except she wasn’t. 

Amidst all the mayhem of the hospital, my aunt’s physicians missed a critical diagnosis. Buried under the mountain of data within her electronic medical record was an up-trending creatinine level – an indicator that she had a serious kidney injury. Her care team never caught it. She left the hospital with an active, potentially life-threatening medical condition.

We were incredibly upset. Who was to blame? Pointing the finger at her clinicians felt intellectually dishonest. People become doctors to heal, because they care about the health of the people they treat. They dedicate years of their lives to learning how to do it well. How then, could they miss critical diagnoses, especially when so much clinical data is on the computer screens they seem to look at more than their patients? 

Therein lies the problem. It’s not the doctors. It’s not even their computer screens. It’s all the information they’re asked to process in such a small window of time. 

Information overload and its impact on healthcare

My aunt’s experience, I’ve come to learn, is far from unique. Most American families have a similar story. According to recent research published in JAMA, nearly 800,000 people die or are permanently disabled each year due to diagnostic errors made in clinical settings. Stroke, sepsis, pneumonia, blood clots, and lung cancer account for 40% of those misdiagnoses. All can be treated promptly and efficiently when doctors have the right diagnostic information handy. 

They often have the information they need, but it’s far from handy. It’s buried in the electronic health record (EHR).

As the EHR has stepped into its role as the digital hub for patient care, it’s accumulated a massive amount of clinical content. Notes, labs, imaging studies – it’s comprehensive. In theory, that’s a good thing. In practice, it’s an obstacle to delivering quality care. 

The sheer amount of medical information in the EHR for a given patient has become paralyzing. According to recent research, information overload can hinder the ability of clinicians to diagnose and treat their patients. This problem is only going to get worse as data snowballs: it took 50 years for the amount of clinical data to double between 1950 and 2000. As of 2020, it was a mere 73 days

Faced with a mountain of clinical data, even the most skilled, experienced, and knowledgeable doctors don’t have the time or bandwidth to dig in and find what they need. They’re searching for needles in a haystack. Tragically, it’s become far too easy to miss a critical diagnosis. 

Practical applications lost in the AI hype

I wish I’d known this when my aunt was in the hospital. Putting myself in the shoes of her doctors, I understand why our visits were short, and so few and far between. It wasn’t in hospitality my family and I were sensing. Rather, my aunt’s care team was likely just as frustrated as we were.  

Hospitals and clinics are frantic, high-pressure environments. That’s not going to change in the near future. Every provider organization is grappling with unfavorable supply-demand economics, and they will continue to do so for some time. Fewer clinicians seeing more patients – and having more medical information in front of them than they can reasonably put to use – is only going to make care worse. 

Clinicians have found some relief through ambient scribes. These solutions have reduced hours of “pajama time” and helped alleviate some administrative burden in hospitals and clinics. But clinical notes are just a drop in clinicians’ bucket of burdens. The unfortunate reality is, even if health systems were to magically hire their way out of their economic paradigm – and even if recording conversations were completely automated – clinicians would still be spending hours upon hours sorting through monstrous amounts of data. 

Producing documentation takes time, but making sense of years of data has become an inhuman feat. In fact, it’s no longer a job for a human. It should be a job for AI.

In all the excitement over generative AI’s usefulness in reducing tasks like scribing, we’ve overlooked its potential for helping clinicians analyze data and documentation, for putting the insights they need in front of them when they need it. Automating all that analysis makes the bucket of burdens much easier for clinicians to carry. 

The current regulatory environment allows AI to go beyond analysis. Right now, it is possible for AI to alert doctors to potential diagnoses based on the information it’s analyzed in the EHR – in record time. 

AI-enabled clinical decision support (CDS) is not a novel idea, but so far, CDS tools have largely targeted hyper-specific conditions in specific care settings and have seen low adoption. Their application is too limited, and their regulatory stature too cumbersome, to make sense for most providers outside of specialties. Clinicians need clinical intelligence technology that can be applied much more broadly to support decision-making on a population level. That’s where most care is happening, and it’s where most diagnoses are missed. The future of medicine is AI-enabled clinical insights systems that can analyze the EHR and prompt clinicians with potential diagnoses and treatment options in real time.

The benefit of a clinical insights system that can help care teams catch & document diagnoses like stroke, sepsis, and blood clots is clear for busy clinicians. But it’s even clearer for patients who deserve peace of mind knowing no diagnosis is being missed and that they’re getting the best possible care. 

The status quo is jeopardizing the lives of patients, as it did with my aunt. Technology should be able to clear the fog of data standing between them and their patients, shining a light on diagnoses they otherwise might miss.


About Eli Ben-Joseph

Eli Ben-Joseph is the co-founder and CEO of Regard, the leading AI clinical platform. The concept for Regard was born out of Stanford while Eli was a graduate student, alongside his co-founders Nate Wilson and Thomas Moulia. Eli received his bachelor of science in bioengineering and biology from MIT and a master of science in computer science and management from Stanford University. Prior to Regard, he worked at the MIT Media Lab on special projects.