Frequently asked questions

What a token is, why the number moves between models, and how far you can trust each count.

What is a token?

A token is the unit a language model actually reads. It is usually a common word, a word fragment, a piece of punctuation, or a run of bytes. English prose averages roughly four characters per token, so 1,000 tokens is about 750 words, but that ratio moves a lot with code, with numbers, and with non-Latin scripts.

Why does the same text count differently on different models?

Every model family trains its own tokenizer with its own vocabulary. A word that is one token in one vocabulary can be three in another. Vocabulary size matters most for text the tokenizer was not optimised for: Chinese, Arabic, and heavily punctuated code are where the families diverge hardest, sometimes by more than fifty percent on the same input.

Why can Claude only be estimated?

Anthropic does not publish a tokenizer for Claude. The only exact count comes from its count_tokens API endpoint, which returns a single integer, requires an API key, and means sending your text to a server. This tool runs entirely in your browser and never transmits what you paste, so Claude is estimated per character class and shown as a range rather than a point value. Token IDs are disabled for Claude, because showing plausible but wrong integers would be worse than showing nothing.

Is my text sent anywhere?

No. Every tokenizer runs in your browser in a Web Worker. There is no server route, no API key, and no analytics on the text. The only network request the page ever makes is downloading a tokenizer file from Hugging Face when you pick a model that needs one, and the file size is shown before that download starts.

Does the token count match what an API call will charge me?

Not exactly, and it will always be lower. This counts the tokens in a piece of text. A real API request also spends tokens on message framing, the system prompt, any tool or function definitions you send, and images. Treat this number as the cost of your content, then add the overhead of the request that carries it.

What do the exact, proxy, and estimate labels mean?

Exact means the model's own published tokenizer is running on this page, so the count, the boundaries and the IDs are the real ones. Proxy means a published tokenizer from the same family is standing in, which is the case for Gemini, where Gemma 3 is used because Google does not publish Gemini's. Estimate means no public tokenizer exists at all, which today applies only to Claude.