Skip to main content
    ← Back to blog

    AI hallucination: what it is and how to catch bad analysis

    November 19, 2026
    Inteligência Artificial
    AI hallucination: what it is and how to catch bad analysis

    You ask an AI tool which campaign drove the most sales in October. Three seconds later: “Cart Remarketing, with 312 orders and a 6.4 ROAS.” The answer names a real campaign, uses numbers that look checked and sounds sure of itself. The problem: Meta Ads recorded 187 orders for that campaign, and the ROAS was 3.1.

    Nobody lied on purpose. The model did what it was trained to do: produce the most likely continuation of your question. When it doesn’t have the data, or only has part of it, the most likely continuation is still a number that looks true.

    AI hallucination is when a language model generates false or unsupported information and presents it with the same confidence as a correct answer: an invented number, a source that doesn’t exist, a quote nobody said. It happens because the model predicts plausible text instead of checking the truth, and it gets worse when context or data is missing.

    Why AI hallucinates (and why it’s not a bug that will disappear)

    A large language model learns language patterns from billions of texts. When it answers, it picks, word by word, what is most likely to come next. That works well for structure, tone and general reasoning. For specific facts, like your store’s revenue last Tuesday or the CPA of an ad set, the model only gets it right if the data is in front of it: in the conversation, in an attached file or in a connected tool.

    In September 2025, OpenAI published Why language models hallucinate with an uncomfortable explanation: the very tests used to score models reward guessing. If the metric only counts correct answers, saying “I don’t know” scores zero, while a guess sometimes scores a point. The model learns to guess.

    The study’s example makes it concrete. On SimpleQA, a set of short factual questions, o4-mini almost never abstained and got 24% right. gpt-5-thinking-mini got slightly fewer right, 22%, but said “I don’t know” on more than half the questions. The result: the error rate fell from 75% to 26%.

    Models that admit uncertainty make fewer errors Answers to factual questions on SimpleQA, as % of total gpt-5-thinking-mini: says “I don’t know” when unsure 22% 26% 52% o4-mini: almost always takes a guess 24% 75% 1% abstention Correct Wrong (hallucination) I don’t know Source: OpenAI, “Why language models hallucinate”, September 2025
    Abstaining when the evidence is missing cuts the error rate from 75% to 26%, with almost the same number of correct answers.

    For anyone using AI in a business, the practical lesson is this: a model that admits it doesn’t know is more useful than one that gets a few more right and is confidently wrong the rest of the time. Accuracy leaderboards don’t tell that story.

    What are the types of AI hallucination?

    Not every hallucination is the same, and each type calls for a different check. These four show up most in marketing and management work:

    TypeWhat it looks likeExample in data analysis
    FactualStates something false about the world“Free GA4 keeps detailed data for 5 years” (the maximum on the free version is 14 months)
    NumericalInvents or miscalculates a numberA sales total that doesn’t match your ERP; a conversion rate with the wrong denominator
    SourceCites a study, link or author that doesn’t exist“According to research by firm X, 68% of retailers…”, and the research doesn’t exist
    ReasoningRight data, wrong conclusionSees sales and ad spend rising together and concludes one caused the other

    Numerical hallucinations are the most dangerous for data-driven decisions. A made-up source falls apart the moment you click the link. A ROAS three points too high goes straight into the budget spreadsheet, and nobody goes back to check.

    AI hallucination examples that cost real money

    The most cited case of the past year is Deloitte in Australia. An AU$440,000 report for the Australian government’s employment department included academic references that didn’t exist and a fabricated quote attributed to a Federal Court judgment. A University of Sydney researcher spotted the problems after publication. The firm refunded the final installment of the contract, according to The Register’s October 2025 report.

    The point isn’t that Deloitte used AI. It’s that nobody checked the references before delivery. At a small or mid-sized company, the equivalent is less dramatic and far more common:

    • An agency’s monthly client report with a cost per lead the AI “estimated” because the export was missing the leads column.
    • A churn analysis in which the model treated inactive customers as cancelled and doubled the rate.
    • A competitor comparison with prices the AI remembered from two years ago.

    In every case, the answer looked right. That’s the trap: hallucinations don’t come with a warning label.

    How to spot AI hallucination in data analysis

    You don’t need to distrust everything. You need to know where to look. Four questions settle most cases in a few minutes.

    Four questions before you use an AI number Takes under five minutes and catches most hallucinations 1. Source Where is it from? 2. Reproduce Does it match? 3. Magnitude Is it plausible? 4. Counter-check What would break it? Passed all four Decide with the number Failed one It is a hypothesis: go back The bigger the decision, the stricter each step
    Validation protocol: if a number fails any step, it’s still a hypothesis, not a fact.

    1. Where did this number come from?

    Ask for the explicit origin: which table, which date range, which filter. A tool that actually queries your data can show the query, whether SQL, a formula or a sheet name. If the answer is vague (“based on the available data”), treat the number as a hypothesis.

    2. Can you reproduce it?

    Take one number from the answer and recalculate it in the source platform. If the AI says October’s average order value was $186, open your store backend and divide the month’s revenue by the number of orders. If it matches, move on. If it doesn’t, stop, because the rest of the analysis probably used the same bad input.

    3. Is the order of magnitude plausible?

    You know your business better than any model. If your conversion rate has always been between 1% and 2% and the AI shows 8%, someone swapped sessions for users or orders for items. This common-sense test catches a lot of errors without opening a single spreadsheet.

    4. What’s the counter-evidence?

    Ask the AI: “what would make this conclusion wrong?” A good model points out the weak spots in its own analysis, such as a small sample, seasonality or a tracking change. If it can’t find any, be suspicious. The same care applies to the biases that distort decisions, which the AI inherits from whoever is asking.

    How to prevent AI hallucination before it happens

    Checking afterwards works. Preventing it is cheaper. Four habits make a visible difference.

    Hand over the data instead of relying on memory

    A bare question (“what’s the average CTR on Meta Ads?”) makes the model reach into its training memory. A question with data attached (“here’s the October export, calculate CTR by campaign”) makes it compute. That’s the principle behind RAG and connected tools: the model answers from the tables you provided, not from what it remembers.

    Give it permission to say “I don’t know”

    Write in the prompt: “if the data isn’t available, say so. Don’t estimate.” It sounds obvious, but it shifts the model’s behavior in exactly the direction the OpenAI study describes. You’ll find more instruction templates in our guide to prompts for data analysis.

    Require calculations to run in a tool

    Language models are bad at mental math once a table has hundreds of rows. Ask for the calculation to run in code or SQL and for the query to appear in the answer. An AI agent with access to your database does this by default: it queries, calculates and only then writes.

    Clean the data before you ask

    If your revenue column mixes gross and net values, the AI will add them up with total confidence. Hallucination and data quality go hand in hand: messy data in becomes a convincing, wrong analysis out.

    Can AI hallucination be fixed?

    Not completely, and anyone promising zero hallucinations is selling something. The rate drops with better models, well-supplied context and tools that calculate instead of remember. Responsibility for the decision still sits with whoever signs off on the budget.

    In practice, the rule that works scales with the risk. For a draft social caption, accept the occasional error and skim it. To move $10,000 of budget between channels, demand the query, reproduce one number and ask for the counter-evidence. Ten minutes of checking costs little next to reallocating money based on a number that never existed.

    Frequently asked questions

    What is hallucination in generative AI?

    It’s when a model like ChatGPT, Gemini or Claude produces false or unsupported content and presents it as fact. It can be a figure, a source, a quote or an event. The name comes from the resemblance to perceiving something that isn’t there.

    What is the AI hallucination rate?

    It depends on the model and the task. On hard factual questions, OpenAI’s 2025 study showed error rates between 26% and 75% depending on the model. When the model summarizes a document you supplied, errors are much lower because the source is right in front of it. There’s no single number.

    What are some examples of AI hallucination?

    Citing scientific papers that don’t exist, inventing case law in legal filings, attributing quotes to people who never said them and, in data analysis, presenting metrics calculated from numbers the model never had.

    How can you tell if AI is hallucinating?

    Ask where the data came from, reproduce at least one number in the original platform, check that the order of magnitude matches your history and ask for the counter-argument. If the tool can’t show where a number came from, treat it as a hypothesis.

    Does ChatGPT make up data?

    It can, especially when you ask about numbers it has no access to, such as your company’s metrics. With the file attached and an instruction not to estimate, the risk drops considerably, but checking is still necessary.

    Hallucination isn’t a reason to keep AI out of data analysis. It’s a reason to choose tools that work with your actual numbers and show the path to them. That’s how Sherlok works: you ask in plain language, it queries your connected Meta Ads, Google Ads, GA4, CRM and spreadsheets, and returns the analysis with the origin of every number so you can check it before you decide.

    Your AI agents already know what to do in your account.

    Turn them on free, no credit card.

    • No credit card
    • Read-only access
    • Revoke it any time

    Keep going

    Read next