Token

A token is the unit language models read and generate — roughly three-quarters of a word in English. Model pricing is per token, so tokens are the unit AI features are actually billed in.

Both input and output are charged, usually at different rates. A support bot that sends ten pages of retrieved documents with every question pays for those ten pages on every single question, which is how a feature that seemed cheap in testing becomes expensive at volume.

Cost control is mostly retrieval discipline: send the three relevant paragraphs rather than the whole manual, cache what repeats, and pick the smallest model that does the job.

Why it matters

If you're quoted a monthly AI running cost, tokens are what's being estimated. Ask what assumptions about volume and context size sit behind the number.

Related terms

Large Language Model (LLM)A large language model is a system trained on very large amounts of text to predict likely continuations. That single ability underlies writing, summa…PromptA prompt is the input given to a language model — the instructions, context and data it uses to produce output. In production, prompts are versioned a…RAG (Retrieval-Augmented Generation)RAG is the technique of searching your own documents for passages relevant to a question, then giving those passages to a language model along with th…

More in AI Automation & Agents

AI AgentWorkflow AutomationLarge Language Model (LLM)RAG (Retrieval-Augmented Generation)EmbeddingVector DatabaseHallucinationHuman in the LoopGuardrailsTool Use (Function Calling)Model Context Protocol (MCP)PromptEvalsIdempotencyWebhookAPI IntegrationAI Voice Agent

Need this built rather than explained?

We publish every price we charge, and you get a quote in writing before anything starts.

See every price
All terms