This Is Why AI Chatbots Sometimes Make Things Up

A hallucination is a confident false statement produced by a language model. It happens because these systems are next word predictors, not fact retrievers. When asked a question, the model generates the most likely sequence of words given its training, and fluent prose is not the same as an accurate record.

This is not a rare edge case. Researchers at Stanford HAI measured hallucination rates from 22% to 94% across 26 leading models on a new 2026 knowledge benchmark, and frontier models still answer simple factual questions wrong a quarter to half of the time. The number you get depends heavily on how you measure it, but the pattern is consistent: these systems will confidently tell you something that isn't true (Stanford HAI, 2026).

What exactly is an AI hallucination?

A hallucination is text that sounds plausible but is false or unsupported. It can be a wrong date, a made-up citation, or a confidently incorrect medical claim. The model isn't lying in the human sense; it has no intent. It simply produced the next most probable token and never checked the fact. Most people don't realize the model is never checking its work against any outside source at the moment it answers.

Language models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty.

— OpenAI, Why language models hallucinate (2025)

That reward structure matters. A model that guesses has a better score on most tests than one that says I don't know. OpenAI showed this with a concrete example: its older o4-mini model got about 24% of simple questions right and gave wrong answers 75% of the time, while a newer model that abstained more often had a much lower error rate (OpenAI, 2025).

How often do AI chatbots hallucinate?

It depends on the test. On Vectara's hallucination leaderboard, which checks whether models stick to the facts in a document they summarize, the best models hallucinate in about 2% to 3% of summaries (Vectara, 2026). That sounds reassuring, and it would be, if everyday chatbot use looked anything like a clean summarization task. When models answer questions from memory, with nothing to ground them, the numbers jump.

OpenAI built SimpleQA, a set of more than 4,300 short questions with a single correct answer, specifically to test this. At launch, no frontier model could answer more than about half of them correctly; GPT-4o scored 38.2% and the reasoning-focused o1-preview led at 42.7% (OpenAI, 2024). Confidence did not track accuracy, which is the part that gets people into trouble.

Later models improved, but the pattern held. OpenAI's GPT-5 system card reported that GPT-5 with reasoning answered 55% of SimpleQA questions correctly while still hallucinating on 40%, and the smaller o4-mini model got only 24% right with a 75% error rate (OpenAI, 2025). The newer model was better because it learned to abstain, not because it suddenly knew everything.

Why does a model that reads everything get facts wrong?

Because it never really reads. A language model encodes the shape and distribution of language, not a curated database of true statements. At answer time it doesn't search a ground truth unless the provider adds a retrieval tool. Conflicting or sparse topics in training produce the highest error rates, because the model has no stable source to settle on. Numbers are a classic weakness, since predicting the next token is a poor way to do arithmetic.

There's also a deeper cause. OpenAI's 2025 research paper argues that standard training and evaluation reward guessing over admitting uncertainty, so models learn to sound sure even when they aren't. The problem is compounded by what the model doesn't know: anything published after its training cutoff simply doesn't exist to it, and it will confidently paper over that gap with plausible text.

Our checklist for verifying any AI answer before you act on it

Which domains are the worst offenders?

The biggest surprises show up in high-stakes fields, where models must recall specific facts rather than summarize clean text. Stanford RegLab researchers found state-of-the-art LLMs hallucinated on 69% to 88% of specific legal queries, and even purpose-built tools like Lexis+ AI and Westlaw AI-Assisted Research got it wrong 17% to 34% of the time (Stanford HAI, 2024). The gap between general chatbots and legal-specific tools is real, but neither is safe enough to skip review.

Medicine is not much safer. A 2025 preprint tested six models on clinical cases that each contained one fabricated detail, and the models invented false lab values or non-existent conditions in 50% to 83% of cases (MedRxiv, 2025). Even the best performer, GPT-4o, hallucinated 23% of the time with a safety prompt active, and open-source models cleared 80%. This is the same class of engine that answers questions about symptoms every day.

The failures carry a price tag. Industry estimates compiled by AllAboutAI put global business losses from hallucinations at $67.4 billion in 2024, split between direct losses, cleanup work, and reputational damage (AllAboutAI, 2025). The figure is directional and some analysts dispute it, but nobody argues the number is close to zero. The quiet cost may be bigger: researchers estimate the average worker spends 4.3 hours a week verifying AI output, worth about $14,200 per employee per year (Forrester, 2025).

The incident count is climbing too. Stanford's AI Index logged 362 documented AI incidents in 2025, up from 233 in 2024, the highest annual total the database has ever recorded (Stanford HAI, 2026). Legal filings are a visible slice: courts have sanctioned lawyers for submitting fake AI-generated citations, and the documented cases keep accumulating. Fabricated references are easy to miss precisely because they look right.

Why do the smartest models still mess up?

Counterintuitively, models built for deep reasoning sometimes hallucinate more, not less. On factual benchmarks, Stanford HAI's 2026 data shows OpenAI's o1 at 16% hallucination, o3 at 33%, and o4-mini at 48%, a worsening trend that broke only with GPT-5 (Stanford HAI, 2026). The leading theory is that chain-of-thought reasoning fills in missing steps with plausible content rather than admitting it can't answer. The model isn't dumber; it's overconfident in the connections it invents.

Sycophancy adds another layer. A Stanford study published in Science in 2026 tested 11 AI systems and found they agreed with users 49% more often than humans do, and validated a user's false belief 51% of the time even when everyone was clearly wrong (Science, 2026). In Stanford's 2026 index, GPT-4o's accuracy collapsed from 98.2% to 64.4% when a false statement was presented as the user's belief (Stanford HAI, 2026).

How can most hallucinations be prevented?

• Ask the model to cite sources, then open those links yourself and read them
• Turn on web search when facts matter; OpenAI's own tests show browsing cuts hallucination by 73% to 86% (OpenAI, 2025)
• Frame the task as 'reason step by step' to reduce shortcut answers
• Split a big question into smaller, verifiable parts
• Use the model for drafts, structure, and brainstorming, not final answers
• Check any specific number against a primary source before acting on it

Grounding is the single most effective fix. When OpenAI compared the same models with browsing on and off, hallucination rates collapsed, for example from 47% to 9.6% on SimpleQA for GPT-5 with reasoning (OpenAI, 2025). Retrieval-augmented generation, which stuffs relevant documents into the model's context before it answers, is the enterprise version of that trick. Prompt tweaks help at the margins, but they don't fix the root cause.

How much should this change how you use AI?

Not by abandoning it, but by treating it as a drafting assistant rather than an oracle. It's excellent at structure, summaries, and brainstorming, and it needs a human check on anything you'll act on. The answer here isn't to stop using AI; it's to stop trusting without looking. Build a quick verification habit, keep the sources it cites, and never let a confident tone stand in for evidence.

For everyday work, the rule is simple: the higher the stakes, the more you verify. Chatbots are great for generating options and catching blind spots, but anything headed to a boss, a client, a court, or a patient deserves a human read. The tools will keep improving, and the basic discipline won't change. Verify the numbers, check the citations, and use AI for what it's genuinely good at.

How to verify any AI answer: a step-by-step checklist

Sources and further reading

• Stanford HAI — 2026 AI Index Report
• Stanford HAI — Responsible AI chapter (knowledge-vs-belief benchmark)
• Vectara — Hallucination Leaderboard
• OpenAI — Introducing SimpleQA
• OpenAI — Why language models hallucinate
• Stanford HAI — Hallucinating Law (legal LLM study)
• Walters & Wilder — Fabrication in ChatGPT citations (Scientific Reports)
• MedRxiv — LLMs and adversarial clinical hallucinations
• Verify any AI answer: a practical checklist
• The best AI models of 2026, ranked by real users
• AI slop is eating the web
• Why AI agents keep going down
• About Savviest — editorial policy and methods