TL;DR: A token is a chunk of text, about three quarters of an English word on average. Providers bill per token, with output typically priced several times higher than input; models generate output one token at a time; and the context window caps how many tokens a model can consider at once. Prompt length, history, retrieved documents and answer length are therefore cost, speed and quality decisions.
What a token is
Models do not process letters or whole words. A tokenizer splits text into tokens: frequent words are often a single token, while rare words, numbers and many non-English words are split into several pieces. Spaces and punctuation are usually part of tokens.
Rough rules of thumb for English:
| Text | Approximate tokens |
|---|---|
| One word | 1.3 |
| One sentence | 15 to 30 |
| One page | 500 to 700 |
| A 300-page book | 120,000 to 180,000 |
These are approximations. The real count depends on the tokenizer, which differs between model families and generations.
Tokens as cost
Providers price per million tokens, separately for input (everything sent to the model) and output (what it writes). As of October 2026, Anthropic's published first-party list prices per million tokens are:
| Model | Input | Output |
|---|---|---|
| Claude Haiku 5.5 (prompts up to 100K tokens) | $0.10 | $0.50 |
| Claude Sonnet 5.5 | $2 | $10 |
| Claude Opus 5.5 | $4 | $20 |
| Claude Fable 5.1 | $10 | $50 |
Two patterns matter more than the exact figures: output costs about five times input on this rate card, and the cheapest and most expensive tiers differ by about 100 times. Model choice is one of the largest cost decisions a team makes.
Input is often the larger share of the bill despite its lower price, because every call resends the system prompt, conversation history and retrieved context. Ask a two-line question of an assistant whose instructions run to 3,000 tokens, and the bill counts about 3,050 input tokens, every time. Prompt caching discounts repeated prefixes heavily on most major platforms, which is why prompt structure is a cost decision.
Tokens as speed
Models generate output one token at a time, typically tens to a couple of hundred tokens per second depending on model and load. A 500-token answer at 80 tokens per second takes over six seconds to complete. Long input also slows the first token, because the model must process the whole prompt before generating. Shorter prompts and shorter answers are faster.
Tokens as memory: the context window
Each model has a context window: the maximum number of tokens it can consider in one request, counting instructions, history, documents and the answer. Windows have grown large, but three limits remain:
- Fit. Very long conversations or large document sets eventually exceed the window, so something must be summarised, dropped or retrieved selectively.
- Cost and latency. Filling a large window on every request is expensive and slow, even when it fits.
- Attention. Models can be less reliable about details buried in the middle of very long inputs. More context is not always better context.
When a request exceeds a limit, behavior depends on the API and application: it may be rejected, or an explicitly configured truncation/compaction policy may remove content. Do not promise that the model simply forgets the oldest message. Reserve output capacity and handle length-related finish states so a cut-off answer is not mistaken for a completed task.
Languages and tokenizers
Tokenizers trained mostly on English text tend to split other languages, particularly those in non-Latin scripts, into more tokens for the same meaning. The same answer in Hindi or Tamil can cost more and take longer to generate than in English; the size of the gap varies a lot by tokenizer. Tokenizers also change between model generations: Anthropic documents that its Claude 4.7 and later models produce about 30% more tokens for the same text than earlier ones, as of October 2026. Cost estimates should always be measured on real traffic, per language, on the model in use.
Product decisions that follow
| Decision | Token effect | Typical choice |
|---|---|---|
| System prompt | Billed every call | Keep it tight; cache the stable part |
| History | Grows every turn | Summarise older turns |
| Retrieval depth | More chunks, more tokens | Fewer, better-ranked chunks |
| Answer length | Output priced highest, slowest | Ask for concise answers; set limits |
| Model tier | Price per token varies 100x | Route simple tasks to small models |
Estimating with tokens
A quick cost estimate: input tokens per call × input price + output tokens per call × output price, times calls per task, times tasks per month, adjusted for retries and failures. Then divide by successful tasks to get cost per successful task, the number that actually matters for the product.
In the interview
Token fluency is assumed in AI PM loops. Candidates should be able to estimate costs from token counts, explain why output costs more, connect prompt and answer length to latency, and describe what happens when conversations exceed the context window. Mentioning language-dependent token counts or tokenizer changes between model versions signals hands-on experience with real bills.
Next: how LLMs generate · prompt and prefix caching
Price source: Anthropic rate card, checked 29 October 2026.