Stunning jao1

Saria said that TREWS’ high adoption rate shows that providers will trust AI tools, but Fei Wang, an associate professor of health informatics at Cornell University, is more skeptical about how these findings will hold up if TREWS is deployed more broadly. Although he calls these studies first-of-a-kind and said he thinks their results are encouraging, he notes that providers can be conservative and reticent to change: “It’s just not easy to convince physicians to use another tool they are not familiar with,” Wang said. Any new system is a burden until proven otherwise. Trust takes time.

TREWS is further limited because it only knows what’s been inputted into the electronic health record — the system is not actually at the patient’s bedside. As one emergency department physician put it in an interview for the third study, the system “can’t help you with what it can’t see.” And even what it can see is filled with missing, faulty, and out-of-date data, according to Wang.

But Saria said that TREWS’ strengths and limitations are complementary to those of health care providers. While the algorithm can analyze massive amounts of clinical data in real-time, it will always be limited by the quality and comprehensiveness of the electronic health record. The goal, Saria added, is not to replace physicians, but to partner with them and augment their capabilities.

“This is a colleague telling you, based upon data and having reviewed all this person’s chart, why they believe there’s reason for concern,” Saria said. “We very much want our frontline providers to disagree because they have ultimately their eyes on the patient.” And TREWS continuously learns from these providers’ feedback. Such real-time improvements, as well as the diversity of data TREWS considers, is what distinguishes it from other electronic records tools for sepsis.

In addition to these functional differences, TREWS doesn’t alert providers with incessant pop-up boxes. Instead, the system uses a more passive approach, with alerts arriving as icons on the patient list that providers can click on later. Initially, Saria was worried that this might be too passive: “Providers aren’t going to listen. They’re not going to agree. You’re mostly going to get ignored.” Instead, clinicians responded to 89 percent of the system’s alerts. As the third study revealed via in-depth interviews, TREWS was seen as less “irritating” than the previous rules-based system.

Machine learning, algorithms work by looking for patterns in data to predict a particular outcome, like a patient’s risk of sepsis. Researchers train the algorithms on existing datasets, which helps the algorithms create a model for how that world works and then make predictions on new datasets. The algorithms can also actively adapt and improve over time, without the interference of humans.

TREWS follows this general mold. The researchers first trained the algorithm on historical electronic records data of 175,000 patient encounters, so it could recognize early signs of sepsis. After this testing showed that TREWS could have identified patients with sepsis hours before they actually got treatment, the algorithm was deployed inside hospitals to influence patient care in real-time.

Saria and Wu published three studies around TREWS. The first tried to determine how accurate the system was, whether providers would actually use it, and if use led to earlier sepsis treatment. The second went a step further to see if using TREWS actually reduced patients’ mortality. And the third described what 20 providers who tested the tool thought about machine learning, including what factors facilitate versus hinder trust.