Improving usefulness of acute care algorithms

By Robin Blythe, AusHSI PhD Scholar

Having given my final seminar in February 2024, I’m writing this as an update to my initial PhD research blog feature in 2021. At the time, there was something of a predictive analytics boom, where many start-ups and major multinationals were promising artificial intelligence could instantly and accurately diagnose, triage, and even suggest treatments for patients.

The motivation for my thesis topic Real-time clinical decision support systems for the prevention of deterioration in acute care was driven by, if not scepticism, at least caution – although many prediction models were already highly accurate, they weren’t achieving better outcomes for patients. I wanted to see what was missing from that process to make clinical decision support systems useful.

One of my research objectives early on was to understand how much time clinicians spent doing routine tasks that could potentially be automated by decision support, freeing them up to spend more time with patients. Unfortunately, I never managed to get the data I needed to study this idea. When speaking to acute care clinicians, however, it soon became obvious that decision support systems were adding to their workloads rather than alleviating them. Many alerts, those created by machine learning models especially, tended to overestimate the degree of risk. The alerts were not necessarily any better than the instincts of a skilled nurse, because at least the nurse can articulate why they were worried and what to do next.

Robin presenting his PhD research with AusHSI and the DHCRC at Digital Health Week NZ 2023 in Hamilton.

To make clinical decision support systems more useful, one of the main conclusions of my thesis was that the underlying model must be chosen carefully based on the outcome being predicted and the workflows it is designed to improve.

Simply replicating what clinicians already do, without allowing them to spend their time doing something meaningful, is not going to help patients. This is an important consideration both for aspiring model developers, who should spend time speaking with their end-users, and also for decision-makers like hospital administrators or funders, who should make sure the products they’re investing in are actually capable of living up to the hype.

Getting through to thesis submission was a slog but ultimately it was made far easier and more productive by the incredible opportunities served up to us by the project funder, the Digital Health CRC. We – the student cohort – benefitted from really motivating events where we got to pitch our work to industry experts and get the thoughts of our peers across Australia. When you’re doing a lot of remote work, having a cohort like this one, as well as the broader doctoral student group at AusHSI, is invaluable. It wouldn’t be much of a PhD journey without the chance to badger your colleagues about a cool and totally impractical idea you had, but that’s how breakthroughs are made.

As a partnering member of the Digital Health Cooperative Research Centre, AusHSI manages several research projects in the area of digital disruption in health environments. Hear Robin talk about his PhD work with AusHSI and the DHCRC with Peter Birch in this Talking HealthTech episode at Digital Health Week NZ 2023 (20:40 min).