Glossary

Hallucination

In one line: Hallucination is when an AI generates information that sounds confident but is actually wrong or made up. The single biggest risk in deploying AI to customers.

What it looks like

A customer asks your AI assistant: “Is the X20 model still under warranty after 24 months?” The AI confidently replies: “Yes, the X20 has a 36-month warranty.” Except your X20 has a 12-month warranty — the AI invented the number. It sounds right. It’s wrong. Now the customer expects something you don’t offer.

Why it happens

Language models predict plausible-sounding text based on patterns in their training. They don’t have a built-in fact-checker. If your specific product isn’t in their training data and you don’t connect them to your actual product info, they fill the gap with whatever sounds reasonable.

How to prevent it

• Ground the AI in your real data (RAG — Retrieval-Augmented Generation). Don’t let it guess.
• Tell it explicitly to say “I don’t know” when uncertain — and reward that behavior in testing.
• Constrain its scope. Don’t let the AI answer about your warranty unless you’ve given it warranty data.
• Test it with adversarial questions before launch. Find the failure modes.
• Show sources when possible so customers (and reviewers) can verify.

A safe deployment pattern

Start AI assistants in “suggest mode” — they draft replies but a human approves before sending. Once you trust the failure rate, graduate them to fully automated for the safe categories of questions, while still escalating uncertain ones to humans. This is how careful retailers roll out AI customer support without blowing up trust.

Related terms

Large Language Model (LLM), Retrieval-Augmented Generation (RAG), Chatbot vs AI Assistant