Whether new technology integrates smoothly into a clinician’s workflow is a crucial predictor of success for AI vendors.
Increasingly, GPs are being nudged into a business model that means they must see many patients a day and stay busy to stay financially viable. If your AI innovation can’t be learned and used in the time it takes them to make a cup of coffee, your chances of success are wickedly small.
That was the message to AI vendors from Dr Zachary Tan, a clinician and president of AI retinal imaging success story, Optain, on day two of the Australasian Institute of Digital Health’s AI.Care conference in Brisbane.
“The key question that we should all ask ourselves as we’re developing these new technologies is, who’s going to be using it? Will clinicians actually adopt it? What problem are you seeking to solve?” he said.
“My mother is a GP. I see how rushed she is every day, and she has to see a new patient every 10 minutes, and every year, our guidelines, our recommendations, ask her to do more in less time.
“So, the reality is if your clinical staff can’t operate the system in the same amount of time it takes them to make a coffee, it’s probably not going to work.
“You have to design for the person and their reality.”
Optain works closely with nurses and medical assistants in the United States, where the company’s retinal imaging technology is deployed across about 5% of the health system.
In parts of the health workforce where staff turnover is high, making technology that requires little training is crucial, said Dr Tan.
Optain recently joined forces with the Digital Health Cooperative Research Centre to apply advanced AI methods to analyse retinal images alongside linked health data from hundreds of thousands of participants.
Oculomics – the science of detecting systemic disease biomarkers through the eye – traditionally relies on manual analysis of large image datasets. However, by using AI methods, the team is hoping to build a multimodal foundational model that will deliver more comprehensive systemic disease detection than traditional single-disease approaches.
Dr Tan told conference delegates that Optain’s mobile retinal imaging technology was serving two purposes.
“AI, via Optain, can really transform eye care in two fundamental ways,” he said.
“The first is how AI can solve real access problems, extending the reach of eye care professionals to communities that face significant access barriers, irrespective of postcode.
“And then secondly … how can we use AI to unlock totally new diagnostic capabilities through this notion of oculomics.”
Optometrists and ophthalmologists have been looking at the retina for many years in order to detect eye diseases, he said, but oculomics was the next step along.
“This is where AI is able to unlock some new capability,” Dr Tan said.
“The systemic disease signals in retina, those really subtle cardiovascular changes … the amyloid if I have a neurodegenerative disease … the impact of kidney function – those are invisible to any human observer, irrespective of how much or how well trained or how experienced you may be.
“What the AI can unlock today with this notion of oculomomics is that we can extract and quantify those 1000s of very subtle, micro patterns that no human grader can perceive or measure.
“So, this isn’t simply around extending access. Here we are at a point in time with clinical AI, which is creating entirely new diagnostic capabilities which simply didn’t exist before.
“These insights don’t actually exist without AI. Oculomics as a diagnostic field and as a clinical field was born because of AI, and it’s really this dual proposition which is so exciting.
Related
“Not only can we use AI to extend access to existing services, solving a very important clinical problem. Today, we’re actually creating entirely new diagnostic capabilities for systemic disease detection which weren’t previously possible.
“That’s really the promise which is so exciting, and certainly what gets me out of bed in the morning.”
But, he said, having a great idea, and turning that idea into a scalable, economically viable and useable product was the challenge.
Points of failure
Venture capital investors in AI healthcare technology look at four possible points of failure when evaluating whether they will invest, said Dr Tan.
“This reality is truth. Healthcare AI very rarely fails because of model performance,” he said.
“It fails because of the systems around it, the processes and the organisational readiness. You have a technical breakthrough which produces a model – that’s necessary, but that alone in itself is not sufficient.
“It’s not enough for clinical impact.
“Failure point number one – a model works in the lab. It’s validated in the lab. It struggles in the clinic.
“It’s all around data representation, edge cases, real world data drift that exposes the model limitations that you don’t usually see in a very carefully curated data set.
“The second point [of failure] is, a model performs clinically well, getting some great results. It’s really accurate.
“But it doesn’t fit the workflow. It’s too hard to use, takes too much time, requires way too much training, disrupts appointment flow. What usually happens is your clinical staff find workarounds or simply stop using it.
“Failure point number three, your model fits a workflow, and seems to be accurate. But healthcare is a business.
“The technology lacks economic viability. There’s no reimbursement, there’s unclear ROI.
“And failure point number four, the model works economically. We can put a business case around it. But we can’t scale it.
“So often I see models dwell in a pilot purgatory. The technology is working beautifully. We can show the business case, but the support needed to implement the damn thing is just too hard, and you can’t hire or train fast enough to support hundreds of sites.”
All four problems have to be solved to have a successful healthcare AI, said Dr Tan.
“This is not an easy game,” he said.
“You really have to think about those four critical failure points up front.”
Three kinds of science
Dr Tan said standing an AI venture up in healthcare required three different types of science.
“You need science science, you need business science, and you need investor science,” he said.
“Science science – discovery, validation, publications, clinical trials, model, accuracy – that’s foundational. That’s essential. That’s where academic institutions and universities are really good. That’s where Australia is actually really good.
“We’re great discovery, publications and innovation. We have world-class researchers producing groundbreaking work, but that’s often not enough.
“Business science is where AI projects face their greatest challenges.
“It’s almost as important as the algorithm itself.
“Does your model reflect a real workflow which can adopt it human factors as well. How is someone interfacing with your product? Can they use it in a minute by pressing a button, or are they having to jump through 12 different screens to try and use your product?
“Are they integrating into these systems of record, the PMS or the EA chart?
“That work, again, is less visible than the Nature paper, but really does determine whether your innovation reaches a patient or stays on the lab.”
Australia, Dr Tan said, needed to do better at new business science.
“We also need to get better around investor science, because everything that I just walked through is really expensive,” he said.
“Optain has raised $35 million US dollars to date. And only at this point in the last six months have we been ready to commercially scale.”
Australia was good at handing out research and development grants, said Dr Tan, but start-ups needed to lean heavily on the private sector and venture capital to enable “that scaling gap”.
“My key takeaway here is in three types of science, science, business science, investor science – all three are essential for that translation,” he said.
“If you miss any one, your technology or research will stay online.”
The Australasian Institute of Digital Health’s AI.Care conference is being held in Brisbane on Monday 24 November and Tuesday 25 November.