The AI lesson 2025 leaves us with (again)

As everyone is sharing their Wrapped Year in Review for 2025, it’s a fitting time to look back on the year that was in the world of AI. Well at least what unfolded in Australian courts this year. 

If there is one consistent theme emerging from the judgments handed down this year, it is this: courts are still encountering the same fundamental problem when AI is misused — a failure to verify what the technology produces. This year has shown this no matter the physical jurisdiction, the factual circumstances, or the AI assistance used, the message from the courts is the same. 

From fabricated authorities, to inaccurate summaries, and over-reliance on AI outputs that were never independently checked (ie. by a human), the courts’ responses in 2025 have read less like isolated warnings and more like a refrain on repeat. Not because judges are resistant to technology (well, not all), but because professional obligations have not changed simply because the tool has. 

This article looks at 3 Australian cases from the year that was 2025 that illustrate this pattern, and why this repetition matters. Not as a scolding exercise, but as a reminder - subtle or otherwise -  that verification and validation are not optional extra steps in an AI-assisted workflow - they are the point at which human judgement must still firmly sit. What unites these cases is not the use of AI itself, but the breach of long-standing professional duties of accuracy, supervision and candour to the court.

Murray on behalf of the Wamba Wemba Native Title Claim Group v State of Victoria [2025] FCA 731

First let us start with the Native Title case of Murray where court documents were filed containing incorrect citations arising from the use of AI in the preparation of the documents. 

Perhaps a little-known fact outside the legal profession is that not all judgments are provided in written form, particularly where they do not concern the substantive proceeding. That’s one of the things that makes this Federal Court judgement particularly interesting because at paragraph [2] Justice Murphy notes that they originally provided ex tempore reasons (Latin for [orally] ‘at the time’) which were not published. It was following discussions with colleagues that cemented the ‘growing problem regarding false citations in documents prepared using AI’ that prompted their Honour to publish the reasons. 

In this matter the Applicant filed a summary document containing many footnotes referencing anthropological and historical reports. Upon being tasked to produce the documents referred to in the footnotes, it became apparent that most of the documents did not exist. 

In the legal practitioner’s submissions explaining what had occurred, blame was put onto the junior lawyer who prepared the document from a different location without the source material. The junior lawyer informed that Google Scholar was used to find the material, but when attempting to replicate the results it was thought that AI was the cause.

The bottom line proved costly: the lack of supervision and validation of AI outputs resulted in the firm being ordered to personally pay the respondents’ costs on an indemnity basis — ouch! Indemnity costs are awarded in exceptional circumstances to deter misconduct during proceedings. 

Re Walker [2025] VSC 714 

Unfortunately, the indemnity costs ordered in Murray did not deter the same conduct from occurring in the next case..

In the case of Re Walker, the Victorian Supreme Court saw opening submissions by a solicitor containing not 1, but 4 mysterious citations. For those fortunate enough never to have been involved with litigation, basically if you submit unfamiliar cases to the court, they will go and find them and read them. In this instance, the court could obviously not find the hallucinated cases. 

Perhaps even more embarrassing, is that the Victorian Supreme Court published guidelines in relation to the responsible use of AI in litigation in May 2024. These guidelines state that ‘Generative AI does not relieve the responsible legal practitioner of the need to exercise judgement and professional skill in reviewing the final product to be provided to the Court…’ (Guidelines for litigants - Responsible Use of Artificial Intelligence for Litigation - VSC). 

The solicitor in question informed the court that the defendant’s submissions were partially drafted with ‘an AI-assisted legal research software tool’, for which the employer firm paid a subscription for. It was also noted that drafts of the submissions were exchanged with counsel.  

‘The ability of the court to rely upon the accuracy of submissions made by counsel is fundamental to the due administration of justice.’ (Director of Public Prosecutions v GR [2025] VSC 490 per Elliot J).

So what happened? The court determined that it was ‘unacceptable’ for AI to be used in court documents without thorough independent verification.

In the courts’ reasoning it relied upon the statement by Justice Elliot in DPP v GR that the accuracy of submissions is fundamental to the due administration of justice. Providing submissions containing hallucinated cases amounted to the solicitor’s conduct being unprofessional conduct. As a consequence for breaching the courts’ guidelines, the solicitor received a formal reprimand. 

Mertz & Mertz (No 3) [2025] FedCFamC1A 222

If Murray was the pre-warning and Re Walker the warning shot, just 4 days later Mertz & Mertz confirms the message is not sinking in. Now let’s take a look at the family law appeal of Mertz & Mertz (No 3). 

This matter in the Federal Circuit and Family Court of Australia now enshrines what happens when a paralegal uses AI to draft a list of authorities which, you guessed it, contains hallucinated cases - and no-one checks it. 

The aptly titled section in the judgement: “The AI Issue” (from paragraph [2]) informs that not 3 weeks after filing a Summary of Argument and List of Authorities, the appellant filed an amended document. The court stated that ‘[i]t was obvious that the amendments were made to rectify the inclusion in the original of non-existent, inaccurate and mis-leading authorities.’ 

Shortly after the Amended Summary of Argument and List of Authorities was filed, the court’s legal case manager for the appeal judicial registrar wrote to counsel that they knew AI had been used and that they were considering referring the matter to the Legal Services Commissioner for further investigation. 

During the written submissions to the court regarding the ‘AI Issue’, the counsel involved confirmed AI had been used, but that they had not used it, whilst seemingly pointing the finger at their instructing solicitor. The court, being a poster child for natural justice, allowed the instructing solicitor to provide submissions. In turn they pointed the finger to the paralegal as the user of the AI (who then had their employment terminated).

However, the ‘AI Issue’ didn’t stop there. A week after the amended document (and the same day as the instructing solicitor’s submissions were filed) the instructing solicitor wrote to the registry providing an updated list of the footnotes contained within the amended document and a request they substitute ‘each and every footnote’. There was no indication in this correspondence as to whether AI could explain these further identified errors. 

The court quoted the Full Court in Helmold & Mariya (No 2) (2025) FLC 94-272: ‘[r]eliance upon unverified research generated by AI has the capacity to confuse, to create unnecessary complexity, to result in wasted time and to mislead the Court and other parties’. 

The professional responsibilities and obligations of legal practitioners was referenced in the courts’ deliberations. The court considered remarks by Acting Judge Humphreys in the 2024 case of Dayal of the significant risks of using AI tools in legal practice in ways which are likely to diminish public confidence in the administration of justice, or bring the legal profession into disrepute. 

The bottom line is that the court treated the matter as a serious breach of professional responsibility that cannot be ‘delegated’ to an AI tool. Additionally, the court took no interest in the finger pointing exercise, and reiterated that practitioners remain accountable for accuracy. 

What are the AI devs doing about this?

The answer, at least publicly, is limited. Despite these limitations, the year in which courts repeatedly addressed AI misuse also saw some policy movement from developers.

On 29 January 2025, OpenAI updated the Universal Usage policies to prohibit the use of OpenAI services for ‘provision of tailored advice that requires a license, such as legal or medical advice, without appropriate involvement by a licensed professional’. 

On 16 August 2025 Anthropic similarly updated their Usage Policy to ‘reflect the growing capabilities and evolving usage’ of their AI products. The policy provides that areas such as law, finance and medicine are treated as ‘high-risk domains’ where outputs must have a qualified human in the loop.

Even if more AI devs update their AI policies to include similar prohibitions, it ultimately remains up to the user to ensure they are using the tool as intended. 

So, what now?

What makes these decisions particularly instructive is that they do not turn on the novelty of AI, nor on the specific tools used. They turn on principles that pre-date generative AI entirely: accuracy, supervision, and responsibility for work put before a court. As AI tools continue to evolve, it is these principles — not the technology itself — that will determine whether their use strengthens or undermines professional practice.

In each of these matters, AI was not the root cause — but it amplified the consequences of existing failures in supervision, verification and professional judgment. The cases of Murray, Re Walker and Mertz & Mertz reinforce something that lawyers and other professionals seem to have forgotten with the emergence of mainstream AI: that humans remain accountable.

Re Walker and Mertz & Mertz were handed down a mere 4 days apart against a backdrop of earlier 2024 and 2025 judgements already sounding the same warnings. At this point, repetition across jurisdictions and practice areas makes it difficult to characterise these decisions as outliers and instead reinforces the need to address these pre-existing issues. What has changed is not the existence of error, but the speed and scale at which unverified error can now enter formal processes.

While these cases arise in legal proceedings, the lesson applies equally to any profession using AI to produce work relied on by others. Just because we have a fancy new tool that can create efficiencies and productivity gains doesn’t mean that we throw away all our governance and professional responsibility obligations. 

As we move into 2026, the lesson should already be clear: verification is not optional, no matter how familiar the warning has become.