a place of mind

By Behnam Mo

How Language Models Work: Tokens, Context, Temperature

One customer message travels through this whole page: Is there any way to get a discount before I commit to the annual plan? We follow it from raw text to tokens, to numbers, to meaning, to a growing context, to a bill; and we end where every marketer should: cost per task, not cost per token.

MODULE 1

Tokens: How Text Becomes Pieces

A language model never reads words. It reads tokens: chunks of text, each mapped to an ID (identifier) number in a fixed vocabulary. Type anything below; this is the real GPT-5-family tokenizer running in your browser.

Tokenizer library could not load (offline?). Showing an approximation; counts are estimates.
Split by:
–
tokens
–
characters
–
words
–
tokens per word
Hover a chip to highlight every other chip with the same ID. In the discount sentence,  to appears twice: once in to get, once in to the. Same ID, different role; the model tells them apart only from the surrounding tokens.
The point. The model never sees the word discount. It sees a number. Characters would make sequences too long; whole words can't handle new or rare words (or a typo, or unmarketability); subword tokenization is the compromise most modern models use. Compare the Persian version: this tokenizer prices it similarly, while another model or phrasing may split it differently. Tokenization decides your unit costs before any pricing page does.
MODULE 2

Embeddings: From IDs to Meaning

An ID is just a label; number 11522 is not more than number 357. So each token ID is looked up in a giant table and becomes an embedding: a long list of numbers (a vector) that encodes meaning. Nearby vectors, similar meanings.

The point. Any discount available?, any promos?, and is there a deal? land in the same neighborhood of this space. A rule-based chatbot needed a human to script each equivalence; a large language model learned the relationship from data. That is why it generalizes, and also why it can only be as good as the neighborhoods it learned.
MODULE 3

Attention: Words Reshape Each Other

Embeddings alone give every word one fixed meaning. But bank in river bank and bank account should not mean the same thing. Attention is the mechanism that lets each position update its meaning from permitted context. In a causal, decoder-only model, a position can attend only to itself and earlier tokens.

The point. Attention is why The plan is not discounted and Is the plan discounted? use nearly the same tokens, yet mean opposite things to the model. Meaning became contextual, computed fresh for every input, with each later position able to consult earlier context. That foundational idea is central to modern language models.
MODULE 4

Context: What the Model Actually Receives

When a customer types one sentence into a chatbot, that sentence is not what the model receives. The provider assembles a whole package, the context, and the customer's words are usually the smallest part of it.

The point. Every call sends the entire package, priced by the token. The system prompt is your brand voice, your policy, and your guardrails; it rides along on every single message, and you pay for it every single time.
MODULE 5

The Growing Conversation

A bare model call is stateless: it remembers nothing between calls. To continue a conversation, prior context must be supplied again, either by your application or the provider. Step through a four-turn hardware-store chat and watch the input meter climb.

The point. Output tokens become input tokens on the next turn. The customer's fourth message is eight tokens long, yet it costs hundreds, because it drags the whole history behind it. That is why long conversations get expensive, and why the industry needed the trick in the next module.
MODULE 6

The Key-Value Cache: Don't Pay Twice for the Same Prefix

In attention, every token computes keys and values. A key-value cache stores those computations. If the start of your context is byte-for-byte identical to a recent call, the provider can reuse those computations instead of redoing them, and charges you a fraction of the price for the reused part.

The point. A cache hit is not the model remembering you. It is the provider skipping arithmetic it already did, on a prefix that matches exactly. Change one early word and everything after it is recomputed at full price. Stable system prompts are not just good engineering; they are a pricing strategy.
MODULE 7

Temperature and Friends: Why the Same Question Gets Different Answers

At each step the model produces a probability for every possible next token, then samples one. Temperature, top_k, and top_p reshape that distribution before the draw. These parameters control response diversity and which candidates remain eligible for selection.

The point. Temperature explains why the same chatbot, given the same input, can answer differently. It is not a bug; it is a dial, and the choice of temperature is a branding decision: it determines your chatbot's personality.
MODULE 8

Pricing: Dollars per Task, Not Dollars per Token

Pricing pages list rates for input, cached input, and output tokens. Your real cost also depends on the tokenizer, prompt and history length, output length, and number of calls: cost per task = (fresh-input tokens × input rate) + (cached-input tokens × cached-input rate) + (output tokens × output rate), summed across all calls. Model selection is a procurement decision, and it is workload-dependent. The figures below are illustrative and editable; verify current provider pricing before using them.

The point. Evaluate cost per task, not cost per token. A model can charge more per token, inflate your input count via its tokenizer, and still be cheaper, if it finishes the job in fewer, shorter outputs. The only honest answer to which model? is a measurement on your own traffic.