How much more evidence do the politicians need to see before they do what they’re best at – backflipping. Switch off the Support at Home integrated assessment tool and start again.
Mark Butler and Sam Rae are painting themselves into an increasingly small corner when it comes to the integrated assessment tool for Support at Home funding, and aged care in general, frankly.
Long-time readers of HSD will know that we’ve been banging on about the controversies around the IAT for months, starting less than a month after the implementation of the new Aged Care Act on 1 November, 2025.
Since then we have followed the trail of misery through multiple senate estimates hearings, aged care provider conferences, FOI documents revealing the testing (or lack of) done on the IAT, and endless stubborn refutations from the ministers.
At times it has felt very much like we have been screaming into the void.
That changed this week when, frankly, the ABC discovered the news that the IAT sucks. Monday night’s episode of Four Corners was a tour de force.
The moment when Greg Pugh, deputy secretary of aged care and seniors at the Department of Health, Disability and Ageing, was asked in senate estimates a series of questions that exposed the absolute fragility of government testing of the IAT was damning.
Who was consulted about taking away the human override?
“In terms of if we ever consulted one individual provider or older person about the removal of the override, I believe the answer to that is no,” said Mr Pugh.
Was the algorithm that has actually been used from 1 November ever live-trialled?
No: the final (i.e., current) algorithm was not the algorithm that formed part of the earlier trial. The Department’s defence was that it had evolved through earlier versions and had subsequently been tested and refined internally.
Did you take its outputs to clinicians and independently establish that it was producing the right clinical result?
“The department validates it,” said Mr Pugh. “The department ran the classification algorithm through a process of refinement on more than 200,000 completed assessments over 2024-25 and we did the same thing for the prioritisation algorithm over 100,000 completed assessments over 2024-25.”
Senator David Pocock then said:
“What [I] didn’t hear is the current algorithm being used has been clinically assessed.
“So again, which group of clinicians has actually looked at the algorithm that’s currently in place and said this is fit for purpose?”
There was no clear answer to that given in estimates. We didn’t hear one, and neither did Four Corners – “the department has not been able to provide names of those involved in vetting the final classification algorithm”.
That matters, because independent validation is not some optional bit of algorithmic housekeeping.
Sydney researcher Andrew Rochford argues that consequential automated decision-making should be continuously measured against an independent, qualified human benchmark – not simply tested and assessed by the organisation deploying it.
In fact, Rochford identifies precisely the problem exposed by Pugh’s answer.
Related
If the organisation deploying the system also sets the standard, measures the system against it and audits the result, it is ultimately marking its own homework. His proposed model instead uses independent experts, blinded to the AI’s decisions, to establish whether the machine continues to match professional human judgement.
Then there was Professor Kathy Eagar’s devastating and steely-eyed analysis of the algorithm itself.
At one branch of the algorithm’s decision tree, a difference of a single point could result in a person gaining or losing almost $40,000 a year in funding, she told Four Corners.
“It is fundamentally completely flawed, and the technical problem is a different one to the policy problem,” Professor Eagar said.
“It doesn’t pass the pub test, but it’s because of the design of the whole classification.
“The algorithm cannot be fixed by tinkering. It can only be fixed by ditching the whole algorithm and starting again.”
Professor Eagar is due to appear at the next public hearing of the Senate inquiry into the Support at Home program late on Monday afternoon. We’ll be watching.
If Four Corners wasn’t enough, the Senate sent a very clear message to the government on Tuesday evening, with the Greens winning the right to debate the IAT and subsequently calling on the government to “take immediate action to abolish the IAT” and replace it with a tested assessment system designed with clinical experts.
It also called for the replacement system to put the rights of older people first, restore the role of clinical professionals in assessments, and reinstate the ability for human override.
The government keeps insisting the IAT can be improved, but the accumulating evidence raises a more fundamental question:
Why is it still using an algorithm that was not live-trialled in its final form, whose independent clinical validation has not been demonstrated, and which experts say is structurally flawed?
And which, by the way, is leaving elderly Australians without critical supports, security and safety while all this obfuscation and bum-covering is going on.
There is, however, evidence this week that the government remains capable of changing its mind.
On Thursday, Mr Butler and Mr Rae finally abandoned the plan to fold the Commonwealth Home Support Program into Support at Home.
It was a significant backflip, and a welcome one.
Which rather begs the question: if the government can listen to the sector, look again at the evidence and change course on CHSP, why the hell can’t it do the same with the IAT?
Mr Rae, bless him, has decided it will be worth him turning up to the CHSP national providers conference on Monday morning at the MCG.
Little does he know a certain HSD journalist will be there to ask a couple of face-to-face questions. I’m tipping I won’t get within 30 feet of him, but hey, readers, I’ll give it a red-hot go.