Context Window

A context window is the span of text — measured in tokens — that a large language model can read and reference when generating a response. It acts as the model’s working memory, holding the current conversation rather than its permanent training knowledge base.

The window includes everything the model sees during a single request: your prompt, any documents you attach, tool outputs, prior turns of a conversation, and the response it generates. Once tokens fall outside the window, the model treats them as if they never existed.

Sizes have grown fast. GPT-2 shipped with 1,024 tokens in 2019; today Anthropic‘s Claude Opus 5 and Google‘s Gemini 2.5 Pro both handle 1 million tokens — roughly eight novels held in one call.

Practically, the window sets the ceiling on what you can hand the model in one call.

Business teams use it to fit an entire quarterly earnings PDF, a customer’s full support-ticket history, or a codebase for the model to reason across in a single pass.

Key takeaways

• The context window is measured in tokens (roughly 4 characters or 0.75 words per token) and holds prompt, prior turns, and the response together.
• Larger windows aren’t automatically better; accuracy and recall degrade as more tokens fill the window, a pattern called context rot.
• Frontier models range from 128,000 tokens on OpenAI’s GPT-4 Turbo (2023) to 1 million tokens on Anthropic’s Claude Opus 5 and Google’s Gemini 2.5 Pro, both released in 2025.
• Once tokens exceed the limit, the model returns an error or drops the oldest content, so context management matters as much as raw window size.
• Sizing your context correctly reduces cost, latency, and error rates, three metrics that decide whether an AI feature ships or stalls in production.

How it works

A context window operates as a rolling buffer of tokens. When you send a request, the model tokenises every input into a single sequence (system prompt, chat history, attached documents, tool outputs), then generates its response inside the remaining budget.

Under the hood, the model applies self-attention across every token in the sequence, computing relationships between each pair. That is why longer windows raise compute cost quadratically in the worst case, even though vendors publish flat per-token pricing.

Tokens are not characters. English tokenisers pack 3–4 characters into one token, so 1,000 tokens equals about 750 English words. Numbers, code, and non-Latin scripts break into more tokens per character.

The model reads the whole sequence in one forward pass, so latency and cost scale with token count. System prompts, cached prefixes, tool definitions, images, and output all count. Prompt caching lowers price for repeat prefixes but does not save window space.

Accuracy also drifts as the window fills. Anthropic calls this context rot: recall and reasoning drop even when the buffer has spare room. Curating what enters the window beats stuffing every document into one request, especially past 100,000 tokens.

Examples

Context window size shapes what an artificial intelligence product can actually do. A 4,000-token model can hold a short email thread; a 1-million-token model can hold an entire legal brief or the full source code of a mid-size app.

Anthropic Claude Opus 5 (2025) ships with a 1-million-token window on the Claude API, Amazon Bedrock, Google Cloud, and Microsoft Foundry — enough to ingest 750,000 words in a single request per Anthropic’s documentation.

Google Gemini 2.5 Pro launched in 2025 with a 1-million-token context, marketed as the capacity to hold eight full-length novels or 200 podcast transcripts inside one prompt, according to Google’s Gemini API docs.

OpenAI GPT-4 Turbo shipped a 128,000-token window in November 2023, a 32x jump from the original GPT-4’s 4,096-token window earlier that year. That change opened long-document chatbot and legal-review use cases overnight.

Enterprise legal review, a 2024 BPO use case, moved from chunk-and-retrieve pipelines to single-shot 1M-token prompts. Firms using knowledge process outsourcing providers cut contract-review cycle time by more than half.

Related terms

A context window sits inside a broader vocabulary of large-language-model concepts. The terms below define adjacent ideas, from what a token is to how retrieval works around window limits, and appear in AI vendor documentation.

• Artificial Intelligence: the broader field of building systems that perform tasks requiring human-like reasoning and pattern recognition.
• Machine Learning: the subset of AI focused on algorithms that learn patterns from data rather than following hard-coded rules.
• Generative AI: AI systems that produce new text, images, or code from a prompt, powered by large models.
• Natural Language Processing: the discipline of teaching machines to understand and produce human language at scale.
• Chatbot: a conversational software agent that runs on top of a language model to answer user questions.
• Automation: the use of software or robots to complete tasks without direct human input, often paired with AI.

FAQ

How is a context window measured?

Context windows are measured in tokens, not characters or words. A token averages 4 characters of English text, so 100 tokens equals about 75 words. Different tokenisers count identical text slightly differently, which is why vendors publish exact numbers.

Why does a bigger context window matter?

Bigger windows let the model see more source material at once: full documents, long conversations, entire code repos. They reduce the need for retrieval systems that chunk content. They also make agentic workflows practical, since tool outputs stay in memory.

What happens when the context window is full?

If input exceeds the window, most APIs return an error. Some chat interfaces drop the oldest turns instead, which can degrade the model’s memory of earlier instructions. Server-side compaction summarises old turns to keep conversations going.

Do larger context windows always give better answers?

No. Accuracy and recall drop as tokens fill the window, a pattern known as context rot. Curating what goes into context, through retrieval or careful prompt design, often outperforms feeding the model every document you have.

How much does context length affect cost?

Longer prompts cost more because vendors bill per input token, and providers with 1-million-token windows charge higher rates once you exceed the standard band on their pricing tiers.

How do developers work around context window limits?

Common workarounds include retrieval-augmented generation (fetching only relevant chunks), summarisation (compressing old turns into memos), and server-side compaction (which the platform runs automatically to trade fidelity for room).

Explore how leading offshore business process outsourcing providers pair long-context AI models with skilled human review to cut document-heavy turnaround time at Outsource Accelerator.