Give us clear guidance, but don’t stifle innovation says the medical software industry in response to the regulator’s most recent review.
Software vendors have largely welcomed the TGA’s latest review of AI scribes, saying they welcomed more clarity and guidance as long as innovation was not “stifled”.
The report, Clarifying and Strengthening the Regulation of Medical Device Software including Artificial Intelligence, was quietly released earlier this month.
It said that digital scribes frequently did provide a diagnosis and treatment options, and that “[t]his functionality indicates digital scribes meet the definition of a medical device and require pre-market approval in the Australian Register of Therapeutic Goods and are potentially being supplied in breach of the Act.”
Jessica White, Best Practice’s group manager for partnerships, said the organisation welcomed pre-market approval for AI scribes.
The practice management software company works with over 200 partners, including those who produce embedded AI transcription products.
“We’re watching this space evolve, and we were hoping that there was some form of pre-market approval or a regulatory framework put in place around the use of AI,” Ms White told HSD.
“In saying that, we don’t want the regulations to stifle innovation – that’s really important, because the current process for software as a medical device is quite complex.
“It’s unclear to a lot of stakeholders around whether their software actually meets the requirements of software as a medical device.”
Best Practice They would like to see a framework applied, but “a different lens” was needed for AI, said Ms White.
“I don’t know if the TGA has been transparent or clear enough around how these AI scribes fit into their software as a medical device regulations and processes. We just need some clearer guardrails and guidelines.
“If you’re a SaMD [software as medical device] provider, it’s about whether you provide clinical decision support, and I think there’s a lot of ambiguity around that, because if I’m providing AI scribe recommendations, for example, around a path forward relating to a patient’s treatment path, that scribe is not actually deciding an outcome for you; it’s presenting you with options.
“There’s probably been a lack of engagement in the AI space from the government, and, from my perspective, this report is just addressing that.
“I look at it as: ‘We haven’t been clear in this space; we have watched AI evolve over the last six to 12 months [and] we’re now going to address this space and ensure that we can help these products deliver safe and compliant products to market’.”
In addition, said Ms White, not all AI scribes were used for clinical purposes but in fact provided a lot of value in the administration of practices.
“I’d just hate to see efficiency loss because we put this really stringent framework around something, and it stops innovation,” she said.
“Some of the partners we work with in this space are small start-ups. They’re doctors who have great ideas and concepts, and they don’t have $100,000 to put towards trying to meet these requirements and things like that.
“So there needs to be a middle ground.
Related
“But I certainly think pre-market approval is really important, and we need to be conscious of where these vendors are coming from, where their data is being stored, where their transcription is happening, what language models they’re using.
“There needs to be a lot more transparency for the end user around how these products do what they do.”
The idea that end users were driving the push towards regulation was echoed by Medical Software Industry Association CEO Emma Hossack.
Ms Hossack, who is a past director at the Australian Digital Health Agency and is on the ADHA Council for Connected Care, said the report was not indicative of “wholesale change” for the industry and that it was just about providing greater clarity.
“Even if you do find yourself captured by software as a medical device, you’re not going to be suddenly in class three. You’re going to be, perhaps, subject to the ARTG, and that’s not especially onerous,” she told HSD.
“I am really keen to make sure that we do engage closely with the TGA, to make sure that the detail is extremely clear.”
Ms Hossack said the report was likely “responding to the concerns of the medical profession, who are embracing the technology in droves because they’re finding it improves their ability to focus on patients and to get some less exciting, non-clinical work done in an efficient way”.
“A lot of software is used for clerical, administrative tasks, like automating payments, and they’re not actually impacting on a patient’s welfare, or prescribing or next steps for that patient,” she said.
Like Ms White, Ms Hossack emphasised the industry’s need for clarity.
“It’s in everyone’s interest that there is clarity,” she said.
“And smaller companies, start-up companies with a good idea, are at huge risk if they don’t have clarity, because they can invest enormous swags of time – and I’ve seen it happen and it’s heartbreaking – on something that isn’t clear.”
She said the big operators were “making an enormous effort to ensure that they’re going even above and beyond what is currently expected of them”.
Ms Hossack said she did not think the TGA was going to follow in the NHS’s footsteps in terms of defining AI scribes as medical devices.
“Under the TGA guideline of 2023 they excluded AI scribes on the basis that if they were not providing clinical decision-making support or services or not therapeutic guidance, then they’re excluded from software as a medical device,” she said.
“But then, of course, we’ve seen the NHS be a little bit more stringent with their definition. And I think there was probably a colonial reaction – if they’re doing it, we should too.
“But realistically, our software as a medical device regulation has been extremely successful today. It was co-designed with industry very closely, and case examples have been terrific.
“What we’re seeing here is the TGA suggesting that they have made the right decision with AI scribes.
“But where the scribes are going further than being a transcription service and where they’re integrated into clinical practice and could influence clinical decision-making, then regulation is appropriate under software as a medical device.
“The big risk for our industry is having fear, doubt and uncertainty among regulation, so it’s really important that we all get absolute clarity, and that’s what we’re doing as an industry.”
Professor Enrico Coiera, director of theAustralian Alliance for Artificial Intelligence in Healthcare and director of the Centre for Health Informatics at Macquarie University told HSD the latest TGA report “makes it pretty black and white clear that scribes are subject to TGA regulation”.
“[It] is a pretty major change with all sorts of consequences that I guess will need to be worked out.”
It was not about the class of scribe, but the function, said Professor Coiera.
“What does the scribe do? If you and I are having a chat and the scribe generates a readout of the meeting, that’s not decision support, that’s creating a record,” he explained.
“If it now takes a conversation and produces a letter, if it produces a summary, it’s therefore taken information, included something, excluded something else – that’s making decisions. And so that that function is subject to regulation.”
The technology was moving very quickly and regulation was difficult, said Professor Enrico.
“You know the old adage of ‘move fast and break things’? In healthcare, if you’re moving fast and breaking things, the things you’re breaking are people. So, we don’t want to do that.
“That’s why this is an important document, and the fact that there’s a lot of ‘we need to look at’ statements simply reflects the challenges the TGA faces.
“They’re not kicking that can down the road. The can is tumbling in the wind.”
Professor Coiera said the traditional medical software industry was familiar with safety and quality issues, but there were many new entrants into the market.
“If I was a scribe manufacturer and I wanted to be leader of the pack in the next round, I’d be publishing publicly my performance metrics,” he said.
“I’d love to see what error rates are like, what kind of errors we generate, because that helps the clinician know what to look for. I think that level of maturity and disclosure will separate out industry pretty quickly.
“If I was a purchaser or a user, what this says to me is that I would immediately start to ask questions about whether or not this [product] was compliant with TGA.”