AI Token Pricing

AI token pricing charges for language model use by the token rather than by seat, request or hour. Input tokens and output tokens carry separate rates. Cost rises with how much text you send in and how much the model sends back.

A token is a fragment of text, usually shorter than a word. Common words become one token — rare words, names and code split into several.

That fragmentation is why a page of plain English and a page of dense technical notes cost different amounts to process, even at the same word count.

Because rates are quoted per million tokens, a single call looks almost free — and a production workload does not. The arithmetic only becomes visible once volume, conversation length and retries are all in the same spreadsheet.

Key takeaways

• AI token pricing meters language model use by the token, with separate rates for input and output.
• Output tokens are consistently the more expensive side, so verbose answers cost more than long prompts.
• Conversation history is resent on every turn, so cost compounds as a chat gets longer.
• Caching repeated prompt prefixes and batching non-urgent work are the two standard cost controls.

How it works

Billing is metered per token in both directions. The provider counts the tokens you send, counts the tokens the model generates, applies a different rate to each, and adds modifiers for caching, batching and very long contexts.

Four things drive the bill, and only one of them is obvious.

Conversation length is the trap. A chat assistant resends the whole transcript on every turn to stay coherent, so turn thirty is paying for twenty-nine earlier turns as well as the new question.

Retries are the line item nobody budgets. A failed parse, a refused answer or a timeout all bill for the tokens already spent, so a five percent retry rate is a five percent surcharge.

Caching cuts the repeated part. When a long system prompt, policy document or product catalogue sits at the front of every request, providers bill those cached tokens at a lower rate than fresh input.

Batching cuts the urgent part. Work that does not need an answer in the next second gets queued and priced lower — which covers most back-office classification and enrichment.

Published rates move often, so a forecast built on today’s per-token figure dates quickly. Model the drivers instead: average prompt size, average response size, turns per session, sessions per month, and a retry allowance.

Model choice is the other lever. A smaller model handling routine classification and a larger one handling the hard minority is usually cheaper than sending everything to the expensive option — which is what AI vendor evaluation exercises are increasingly built to test.

Two guardrails keep the estimate honest. Set a maximum output length per call, and cap or summarise conversation history, because unbounded versions of either are how pilot budgets get overrun.

Examples

Published vendor documentation is the clearest place to see the mechanism working, and the research lineage behind tokenisation explains why the industry meters a text fragment at all rather than something simpler like a request or a word.

OpenAI’s published pricing separates input, cached input and output rates for the same model, quotes them per million tokens, and adds tiers: a standard tier, faster tiers priced higher for lower latency, and an asynchronous batch tier.

The batch route is the plainest discount. The documented batch tier costs 50% less than synchronous calls, with a 24-hour completion window and its own rate limit pool, which suits evaluations, classification and bulk embedding.

Google’s Gemini platform shows the caching side. Its documentation explains that a repeated prompt prefix is billed at a reduced rate, with large shared content placed at the start of the prompt.

Retrieval changes the shape of the bill. Pulling documents into the prompt to ground an answer is an input-token cost, so a system that retrieves ten passages per question pays for all ten every time.

The unit itself comes out of translation research. Rico Sennrich, Barry Haddow and Alexandra Birch’s 2016 paper on subword units for rare words in neural machine translation introduced the byte pair encoding approach most current models still tokenise with.

Related terms

AI token pricing sits inside the broader commercial vocabulary buyers use to compare and control AI spend. These neighbouring terms cover the units being metered and the frameworks used to judge them.

• Context Window: the maximum span of tokens a model can consider in one request.
• Inference Cost: the total expense of running a trained model to produce predictions.
• Consumption Pricing: the commercial model charging by measured usage rather than licence.
• Large Language Model: the class of model whose usage token pricing meters.
• Total Cost of Ownership: the full lifetime cost of a system including running and support.

FAQ

What is a token in AI pricing?

A token is a fragment of text, usually shorter than a word. Common words are one token; rare words, names and code split into several, which is why identical word counts can bill differently.

Why do output tokens cost more than input tokens?

Generating text is more computationally demanding than reading it, so providers price the generated side higher. Shortening answers therefore saves more than shortening prompts.

How do you reduce AI token costs?

Cap output length, trim or summarise conversation history, cache repeated prompt prefixes, batch non-urgent work, and route easy requests to a smaller model.

Does a longer conversation cost more?

Yes, and the increase is not linear. History is resent each turn, so a long session pays repeatedly for its own earlier messages.

How should a buyer forecast AI token spend?

Model the drivers rather than a published rate: prompt size, response size, turns per session, sessions per month and a retry allowance.

Search the Outsource Accelerator provider directory to compare firms that build and run AI-assisted services with transparent usage costs.