Ultimately, the future of AI in healthcare will be shaped by the quality of the leadership, governance and clinical judgement that surround it, not by the technology itself.
Artificial intelligence is no longer a future consideration for healthcare. Across Australia, AI is already moving from pilots into clinical practice – from ambient AI scribes that reduce documentation burden to tools that support clinical decision-making and surface population health risks.
Yet many organisations are still navigating how to deploy these technologies safely at scale. Recent Australian research found that while 60% of healthcare organisations are piloting AI technologies, only 12% have deployed them across multiple care or administrative functions.
AI is a medical leadership challenge
There is a misconception that AI is primarily a technology challenge. In my experience, it is fundamentally a clinical leadership challenge.
Imagine a rural hospital where an AI tool alerts clinicians to sepsis hours before it becomes clinically apparent. The first time it fires, the patient improves. The second time, it is wrong, and the patient deteriorates. Who is accountable – the vendor, the clinician or the organisation that deploys it? If responsibility cannot be clearly traced, the issue is not the technology; it is the governance around it.
Through my engagement with healthcare leaders across Australia, I have seen a common theme emerge: accountability becomes harder to define as AI moves into clinical practice.
At a recent Royal Australasian College of Medical Administrators (RACMA) event on AI and digital leadership, I spoke about three critical distances where accountability can disappear: between data and decisions, where bias or poor-quality data can influence outcomes; from vendor to governance, where real-world use can drift beyond a tool’s intended purpose; and between a tool to clinical judgement, where over-reliance on AI can erode critical thinking.
If those three distances are closed, the one question is answered.
The principles of clinical governance have not changed. We still need good data, capable people, clear accountability, defined escalation pathways and ongoing oversight.
What has changed is the speed, scale and complexity with which decisions can now be influenced. AI amplifies the consequences when governance fundamentals are not strong enough.
Also in today’s edition:
- EXCLUSIVE: SDPRIA review results due Friday as leak reveals 3500 unresolved rollout incidents
- Australia risks becoming ‘hostage to geopolitics’ over healthcare AI
- Physical assault rates among healthcare workers double in 10 years
- The withholding problem: how good doctoring can lead to institutional mistrust
- Northern Territory to come full circle on voluntary assisted dying
- Coatsworth slams public health and clinicians for ‘failed’ smoking policies
- Researchers map ‘pharmacy deserts’, call for more equitable distribution
Building trust through governance
Trust in AI is not created by algorithms alone. It is earned through strong governance, transparency and accountability.
AI learns from the systems and data we provide. When data is incomplete or unrepresentative, algorithms can reproduce existing biases at scale, which means that data quality is not simply an IT issue; it is a clinical governance and, ultimately, a clinical safety issue.
Healthcare has spent decades developing pharmacovigilance to monitor medicines after they enter clinical practice. We will need the same discipline for AI. More than 10,000 AI-related incidents have been recorded globally since 2014. Governance cannot stop at implementation. It must be continuous.
Related
The opportunity is significant. So is the responsibility
AI has the potential to improve outcomes, reduce inequity, support overstretched clinicians and help health and care systems meet growing demand. But technology alone will never deliver these benefits. Success depends on the governance, culture and accountability that surround it.
We should celebrate the progress artificial intelligence is making in health and care settings. But we should also recognise the leadership work happening behind the scenes – by clinicians, governance professionals, digital health leaders and health organisations – to ensure these technologies are safe and implemented responsibly.
Ultimately, the future of AI in healthcare will be shaped by the quality of the leadership, governance and clinical judgement that surround it, not by the technology itself.
The technology is ready. Our responsibility is to ensure our governance is, too.
Dr Monica Trujillo is Chief Health & Risk Officer at Telstra Health and has more than 20 years’ experience in medicine, clinical informatics and digital health. Formerly Chief Clinical Information Officer at the Australian Digital Health Agency, she is a leading advocate for safe, clinician-led digital transformation.