Hallucination

A hallucination is output that is fluent, confident and false — an invented citation, a policy that doesn't exist, a fabricated order number. It's a direct consequence of models predicting plausible text rather than retrieving facts.

Hallucination can be reduced but not eliminated. The effective mitigations are structural: ground answers in retrieved documents, require the model to cite which passage it used, verify claims against a real system before acting, and design the interface so an unverified answer is visibly unverified.

The most dangerous cases are the plausible ones. An obviously absurd answer gets caught. A slightly wrong refund window, delivery estimate or price gets believed and acted on.

Why it matters

This decides where AI can be pointed. Drafting, summarising and classifying tolerate an occasional wrong answer with a human in the loop. Quoting prices, confirming policies and committing money do not, without a verification step against a real system.

Related terms

Large Language Model (LLM)A large language model is a system trained on very large amounts of text to predict likely continuations. That single ability underlies writing, summa…RAG (Retrieval-Augmented Generation)RAG is the technique of searching your own documents for passages relevant to a question, then giving those passages to a language model along with th…Human in the LoopHuman in the loop means an automated process pauses for a person to approve, correct or reject before it continues — typically before anything irrever…GuardrailsGuardrails are the constraints around an AI system that stop it doing things it shouldn't — restricting which tools it can call, validating its output…

More in AI Automation & Agents

AI AgentWorkflow AutomationLarge Language Model (LLM)RAG (Retrieval-Augmented Generation)EmbeddingVector DatabaseHuman in the LoopGuardrailsTool Use (Function Calling)Model Context Protocol (MCP)PromptTokenEvalsIdempotencyWebhookAPI IntegrationAI Voice Agent

Need this built rather than explained?

We publish every price we charge, and you get a quote in writing before anything starts.

See every price
All terms