Deloitte publishes report on AI hallucinations. When one of the world’s most prestigious consulting firms falls into the artificial intelligence trap, it’s worth stopping and asking: if hallucinations are so common, maybe it’s worth paying more attention to the results of AI content?
$440,000 report with made-up quotes
Deloitte Australia found itself at the center of a scandal after it provided the government with a $440,000 (some sources say $290,000) report that (to put it mildly) had some veracity issues 😉
The purpose of the document was to help the Australian government fight abuses in the social welfare system. Instead, the report became an illustrative case of how dangerous over-reliance on generative AI can be.
What went wrong? The report included fabricated references to nonexistent scientific papers, a fabricated quote attributed to a federal judge, and references to reports that were simply never written. These are not minor mistakes, but fundamental errors that undermine the credibility of the entire document.
The case was discovered by Chris Rudge, a health and social law researcher at the University of Sydney. As he himself admitted, the moment of revelation came when he came across a fragment indicating that Professor Lisa Burton Crawford (an expert in public and constitutional law) was to write a book on a subject completely foreign to her specialization.
I immediately knew it was either an AI hallucination or the best kept secret in the world because I had never heard of this book and it sounded absurd
Rudge told The Associated Press.
Billions of investments in AI and basic errors
The irony of the whole situation is even more acute when we realize the scale of Deloitte’s investment in artificial intelligence. In September, the company announced that it would allocate USD 3 billion to the development of generative AI by fiscal year 2030. The day before the publication of the revised version of the report, Anthropic (creator of the Claude model) announced a partnership with Deloitte, which will make the AI ​​assistant available to over 470,000 of the company’s employees.
This highlights a fundamental problem: large organizations are investing enormous sums in technology that they have not yet learned to properly control. Deloitte is no exception. All of the Big Four consulting firms and global giants like McKinsey are spending hundreds of millions of dollars on AI initiatives, often focusing on efficiency gains at the expense of due diligence.
In June, the UK Financial Reporting Council warned that the Big Four firms were not properly monitoring the impact of AI and automated technologies on the quality of their audits. The Australian case shows that these fears were justified.
Quiet corrections and not very convincing translations
The original version of the report was published in July on the Australian Department of Employment and Workplace Relations website. The revised version appeared quietly on Friday – only after the case came to light.
Interestingly, the updated version now includes a disclosure that it was built using OpenAI’s Azure (a service that provides access to OpenAI’s Advanced Models API). The original report did not contain such information. Fabricated quotes and non-existent references were also removed.
In the “Report update” section, Deloitte assures that the changes “in no way affect the substantive content, findings and recommendations contained in the report.” This statement, however, sounds quite bold when we consider that the report was to serve as a basis for making decisions about public policy, and its foundation was to be… non-existent research.
Importantly, this is not the first case in which AI hallucinations bring down a report, a company, a lawyer or a project. It is worth noting, however, that in each of these cases (including the one from Deloitte) the weakest link is… a human. Why? Because he is the one who does not perform proper fact-checking before publication/approval/upload.
Lesson costing (only partially) refunded
Deloitte has pledged to return the final installment of the payment to the government, although Senator Barbara Pocock of the Australian Green Party argues that the company should return the entire $440,000.
Deloitte misused AI and did it in a very inappropriate way: it misquoted a judge, it used non-existent sources. “That’s the kind of thing a freshman would get in real trouble for.”
Pocock told the Australian Broadcasting Corp.
He’s right. If a student submitted a paper with made-up sources, he or she would most likely be charged with academic fraud. When one of the world’s most powerful consulting firms does this, it receives… a partial refund?
What can we learn?
This case is not just a failure of one company, but a cold bucket and a warning to all people who use generative artificial intelligence in their work. Here are the key takeaways:
- Man must remain at the center. Generative AI can be a powerful tool to speed up work, but it cannot replace human verification. Every source, every quote, every reference requires manual checking. It’s not a matter of not trusting the technology – it’s a matter of understanding its limitations.
- Transparency is key. The fact that the original version of the report did not disclose the use of AI is a serious mistake. Customers (and especially public institutions) have the right to know what tools were used to prepare the materials they are paying for.
- Investments in AI without proper procedures are a recipe for disaster. You can spend billions on the latest technology, but without solid verification and quality control protocols, the risk of errors is enormous. The most ironic thing is that such control does not require much – just a sober mind and fact-checking every date and fact that appears in AI-generated content.
- Reputation is a matter of consistency. For consulting companies, credibility is everything. One report of AI hallucinations can destroy years of trust building. Deloitte emerged from this situation with a partial refund and an official statement that “the matter has been resolved directly with the client.” But is it really?
What if? Will AI lead to disaster?
What is most disturbing is that Rudge discovered these errors by accident during routine reading. How many other reports, analyzes and documents (not only from Deloitte) contain similar hallucinations that simply no one has noticed yet? What if it is, for example, an architectural design or a report influencing decisions regarding, for example, the health system?
Artificial intelligence can be an extraordinary tool. But as the Deloitte case shows, even the largest market players can become victims of their own self-confidence and rush to adopt technology. In the age of AI, a critical thinker like Chris Rudge is more valuable than ever.
Because AI can do a lot. But in fact-checking, humans are still at the top of the evolutionary chain 😉
