AI & LLMs
Token
A token is the small chunk of text (roughly a word-piece) that language models read and generate; usage and limits are counted in tokens, not characters.
A token is the unit a language model actually processes. Text is split into tokens - often sub-word pieces - before the model sees it. As a rough rule of thumb, one token is about four characters of English, or three-quarters of a word.
Tokens matter for two practical reasons: models have a maximum number they can handle at once (the context window), and most API pricing is per token.
Related terms
Last reviewed 2026-07-09