The Surprising Truth About AI Mistakes
Most people assume AI gives wrong answers because it does not know enough. The reality is nearly the opposite: AI hallucinates most often when it is confident, not when it is confused. Understanding this distinction changes how every business owner should approach the AI tools they use every day.
In May 2026, OpenAI reported that GPT-5.5 Instant produces 52.5% fewer hallucinated claims than its predecessor on high-stakes prompts covering medicine, law, and finance. That figure is an improvement — but it also confirms that hallucination is not solved. It is managed. And for a Hong Kong business owner relying on AI for customer communications, financial summaries, or legal document drafts, the difference matters enormously.
What Is AI Hallucination?
AI hallucination is when an AI system generates information that is factually incorrect, fabricated, or unverifiable — but presents it with the same confident tone as accurate content. The term comes from psychology, where a hallucination is a perception with no basis in reality. In AI, a hallucination is a response with no basis in fact.
A classic example: you ask an AI to summarise a company's financial results, and it confidently states a revenue figure that was never reported anywhere. Or you ask it to cite a source, and it invents a plausible-sounding academic paper that does not exist. Or it attributes a quote to a real person who never said it.
The critical thing to understand is that AI does not know it is hallucinating. The model generates text by predicting the most statistically plausible next word, given everything it has been trained on. When it has insufficient or ambiguous data on a topic, it does not stop and say "I'm not sure." It fills the gap with what sounds right — and presents the result as though it were verified fact.
Why Does AI Hallucinate?
AI hallucination occurs because large language models generate text by pattern completion, not by reasoning from verified facts. The model has been trained on an enormous amount of text, and when it encounters a prompt, it produces the statistically most likely continuation — whether or not that continuation is factually accurate.
The problem is compounded in several situations. When a topic is underrepresented in training data, the model has fewer reliable patterns to draw on and is more likely to extrapolate incorrectly. When the user's question is ambiguous, the model may resolve the ambiguity in a direction that seems plausible but is factually wrong. And when the model is asked to produce a specific format — such as a numbered list of sources, or a table of figures — it will sometimes generate plausible-looking entries to fill the expected structure, even if those entries are invented.
Newer model generations reduce hallucination through a combination of better training data, reinforcement learning from human feedback, and targeted fine-tuning for high-stakes domains. But they do not eliminate it. Hallucination is a structural property of how these models work, not a bug that can be patched away entirely.
What Are the Most Common Types of AI Hallucination?
There are four hallucination types that matter most to business owners: factual errors, source confabulation, number fabrication, and entity misattribution.
Factual errors are the most common. The AI states something as true that is simply incorrect — a wrong date, a wrong company name, a wrong regulation. These are often subtle enough to pass initial review but consequential enough to cause problems when they reach clients or regulators.
Source confabulation is when the AI invents a citation that looks real but does not exist. It might name a real journal, a real author, and a plausible-sounding paper title — none of which correspond to an actual publication. This is particularly dangerous in professional or research contexts where citations are expected to be verifiable.
Number fabrication occurs when the AI generates a specific figure — a percentage, a price, a quantity — that sounds authoritative but was not sourced from any actual data. Asking an AI for "the average gross margin for restaurants in Hong Kong" without providing a source may produce a figure that looks reasonable but was essentially estimated by the model.
Entity misattribution is when the AI attributes a statement, action, or characteristic to the wrong person, company, or organisation. This ranges from minor errors ("the wrong CEO name") to significant reputational risks ("incorrectly stating a company was involved in a legal case").
How Often Does AI Hallucination Happen?
Hallucination rates vary significantly by task type, model generation, and how the prompt is structured. According to OpenAI's own benchmarks released in May 2026, GPT-5.5 Instant produces 52.5% fewer hallucinated claims than GPT-5.3 Instant on high-stakes prompts — which means the error rate has been nearly halved, but not eliminated. A Gartner research note from 2025 estimated that AI systems then available produced unreliable outputs on 15 to 20 percent of factual queries in enterprise contexts.
The practical implication: if you are using AI for a task that requires factual accuracy — quoting a regulation, summarising a contract clause, citing a statistic — assume that roughly 1 in 5 to 1 in 10 responses may contain an error that needs verification. This does not make AI useless; it means AI requires a human checkpoint on factual claims, just as a junior staff member's first draft requires review.
Hallucination rates are considerably lower for tasks that do not require factual recall: writing tone improvements, reformatting data that you provide, translating between languages you can verify, or generating creative options for a brief you have written. For these tasks, the risk is much smaller.
Which Business Tasks Carry the Highest Risk?
High-risk tasks are those where a hallucinated fact would cause harm before anyone catches it: legal document drafting, regulatory compliance statements, financial figures that will be cited in client reports, medical or health information, product specifications that affect safety, and any content that includes a citation or named statistic.
For a Hong Kong SME owner, the specific high-risk scenarios are: using AI to answer customer queries about your product's specifications or compliance certifications; having AI draft terms and conditions or privacy policy language; asking AI to summarise a contract or tenancy agreement; and using AI to generate financial reports that will be shared with investors, banks, or government bodies.
Lower-risk tasks — where hallucination either does not apply or where the error would be immediately obvious — include writing marketing copy where claims are not sourced to third parties, reformatting your own data into a different structure, generating ideas or options from which you select, translating content from one language to another for review, and summarising documents where you hold the original and can spot a discrepancy.
How Can Hong Kong SMEs Minimise Hallucination Risk?
The most effective strategies for reducing hallucination risk are structural, not technical. You do not need a deeper understanding of AI to implement them — you need a consistent habit of treating AI output the way you treat a first draft from a new employee.
Provide the source, do not ask for it. Instead of asking "What is the GST rate for F&B businesses in Hong Kong?", paste in the relevant regulation and ask "Based on this text, what is the applicable rate?". When you give the AI the source material, it does not need to recall facts — it only needs to read and extract, which is far more reliable.
Ask for reasoning, not just conclusions. A prompt like "Tell me the answer and explain step by step how you arrived at it" forces the AI to expose its logic, making errors easier to catch before they reach the output.
Verify any specific figure before using it. A percentage, a year, a name, a regulation number — any specific datum produced by the AI should be confirmed against an original source before it appears in a client document, a presentation, or a public-facing statement.
Use AI for structure, humans for facts. The AI is excellent at building the framework of a document — headings, paragraph flow, professional language. Have it write the structure, then fill in the verified facts yourself. This approach gives you the efficiency gains of AI without exposing you to hallucination risk on the information that matters most.
Is AI Still Worth Using If It Can Make Up Facts?
Yes, for the vast majority of business tasks — provided you understand what you are asking it to do. Hallucination is a real limitation, but it is a tractable one. The alternative — not using AI at all — means a competitor who has learned to use AI effectively with proper verification will outpace you on every task that benefits from automation.
Think of it this way: every new employee you hire also makes factual errors, especially in their first few weeks. You do not stop hiring. You build a review process. AI requires the same adjustment: treat its output as a high-quality first draft that needs human verification on factual claims, and it becomes a legitimate productivity multiplier rather than a liability.
The businesses that struggle with AI are the ones that either avoid it entirely out of fear, or trust it entirely without review. The businesses that thrive are the ones that have found the middle ground: systematic, structured use with a clear verification checkpoint built into the workflow.
How to Build an AI-Safe Verification Habit in Your Team
A simple three-step habit covers the majority of hallucination risk in most SME workflows. First, flag the output: before any AI-generated content leaves your team, someone asks "Does this contain any specific facts, figures, names, or citations?" If yes, each one must be verified against an original source before the document moves forward.
Second, maintain a "trust-but-verify" list for your most common AI tasks. Over time, you will identify which tasks in your business produce reliable AI output and which carry higher hallucination risk. Codify this as a simple internal guide so every team member knows what needs a second check.
Third, use AI's own ability to self-review. Prompts like "Review what you just wrote and identify any claims that you are not fully certain about" often surface the AI's own uncertainty — which is a useful signal even if it does not catch everything.
With UD by your side, AI does not have to feel cold. Navigating AI confidently — knowing what to trust, what to check, and how to build reliable workflows — is exactly what separates businesses that benefit from AI from those that get burned by it.
Not Sure How AI-Ready Your Business Is?
Understanding hallucination is one part of using AI safely. The bigger question is whether your current AI setup — the tools you use, the tasks you automate, and the review processes you have in place — is actually protecting you and your clients. We will walk you through it step by step: assessing your AI readiness, identifying your highest-risk workflows, and building the right safeguards for your specific operation.