Banning AI scribe use in public healthcare settings would have a significant impact on hospital-based clinicians.
Speaking as part of a panel for the AIDH’s Executives in Digital Health series earlier today, Dr Emily Kirkpatrick, managing director of EKology Health, said that SA Health last week banned the use of AI scribes in its public health system settings.
The revelation came in response to a question from an audience member about who would be legally responsible if a patient suffered harm as a result of an AI-generated consultation summary.
“What we have seen is a rapid exponential growth of something that we don’t have key governance control over,” Dr Kirkpatrick said.
“And we made a decision in our state, in South Australia, to ban all use of scribes across the public health system in South Australia last week.
“That was on the basis of the fact that we had uncontrolled governance occurring around information going into publicly available health tools.”
HSD reached out to SA Health today for confirmation of the ban and further details but it had not responded by publication deadline.
Leah Mooney, director at KPMG Law, said that a recent consultation process in this space proposed changes as to who would be legally liable in such a situation.
“But the important thing with AI is that it should not be used blindly without human oversight. As a doctor, you have a primary fiduciary duty to patients.”
Related
Tracey Duffey, first assistant secretary of the medical devices and product quality division at the federal Department of Health and Aged Care, concurred with Ms Mooney’s assessment that it was the healthcare professional’s role to check the content of AI-generated consultation notes.
SA Health’s move comes after the state government banned DeepSeek, the Chinese AI chatbot, from government networks and instructed public services to remove the app from any government-owned devices.
The South Australian government also banned TikTok from being installed on government devices in 2023, citing security and privacy risks.
A statewide ban of AI scribe technology in public health system settings would be a step beyond the state government’s guideline for the use of generative AI and large language model tools, which is intended to serve as a guide for SA government agencies when considering the use of such technology.