AIDA blog: We need unremarkable AI

Thoughts from AIDA Director Claes Lundström

Are you dreaming about spectacular AI making a big splash as it revolutionizes diagnostic imaging? I’m not. I think we have reason to aim for the opposite, AI changing healthcare for the better without making a big fuss.

The message from the radiologists and pathologists engaging in AIDA is that new IT technology must be very carefully tailored to fit the clinical scenarios where it is to be used. AI solutions are no different in that respect. Even a stellar AUC can be trumped by mundane flaws in the user experience. Seemingly small issues can and will prevent adoption, such as having to login to another application or being offered assistance when you don’t need it.

Or to put it in other words:

“There is a real need to design decision support tools not only as a functional utility but as an integrated experience. Their effectiveness should be measured not only by prediction accuracy, but by effectiveness when situated within its social and physical context such as workplace culture and social structures.”

The quote is from an insightful paper by Yang et al. of Carnegie Mellon University, to be presented at CHI this year. They study how to present outcome predictions to inform decisions on artificial heart implants. The key conclusion they make is that AI-based decision support should be unremarkable, that is, conveyed in an unobtrusive way.

I think this perspective is important to embrace in AI development. What we want is to augment workflows. Every time the AI component catches your attention without contributing value, it’s instead degrading the workflow. Working through false positives is the typical example. You don’t need to see that many irrelevant findings before you consider the “support” to rather be a nuisance.

But wait a minute. Isn’t disruption the holy grail in technology innovation? That doesn’t seem to fit with the idea of unremarkable augmentation. Well, if it were possible to replace an entire clinical process with an AI solution doing the job better, I would be all for disruption. The reality is, however, that AI solutions can for the foreseeable future only cover quite narrow tasks. And a clinical scenario, at least a challenging one, does not boil down to a distinct, isolated task but to a complex combination of assessments.

AI has enormous potential to assist with narrow tasks but only if it’s carefully put into the workflow context. As an AI developer, you should fight your instincts to put your specific contribution to the diagnostic work in the limelight. Instead, unremarkability is something to strive for. At least if you’re interested in putting AI to actual use in healthcare.