Tokens and Context Windows Explained: Why Your Chat Forgets
What tokens are, what a context window holds, why a long conversation with ChatGPT, Claude or Gemini starts to forget, and simple ways to keep it on track.
Roman Shalabanov
Published
8 min read
A context window is the amount of text an AI model can take into account at one time, measured in tokens. It includes everything the model sees for a reply: the app's hidden instructions, your messages, its earlier answers, attached files and the reply it is writing. Anything outside the window does not exist for the model.
That is why a long chat seems to forget. When the conversation grows past the window, the app has to drop or compress older messages, and the model cannot see what was removed. Even inside a large window, details buried in the middle of a long conversation are easier for a model to miss. The rest of this article explains tokens, what fills a window and what you can do about it.
What a token is
Language models do not read letters or words. A tokenizer first cuts your text into tokens, pieces from a fixed vocabulary of common chunks. A frequent English word is usually one token. A rare word splits into several. Spaces usually attach to the token that follows them.
Here is how OpenAI's o200k_base tokenizer, used by the GPT-4o generation of models, splits a few examples. We ran them through the same tokenizer that powers our Token Counter:
| Text | Tokens | How it splits |
|---|---|---|
| The cat sat on the mat. | 7 | The · cat · sat · on · the · mat · . |
| Tokenization is surprisingly unintuitive. | 7 | Token · ization · is · surprisingly · unint · uitive · . |
| antidisestablishmentarianism | 6 | ant · idis · est · ablishment · arian · ism |
| 2026-10-19 14:35:07 | 12 | 202 · 6 · - · 10 · - · 19 · (space) · 14 · : · 35 · : · 07 |
| function add(a, b) { return a + b; } | 13 | function · add · (a · , · b · ) · { · return · a · + · b · ; · } |
A few patterns show up quickly:
- Ordinary prose is cheap. A 60-word paragraph of plain English came to 67 tokens, about 4.8 characters per token.
- Numbers, dates, code and URLs are expensive. The timestamp above is 19 characters and 12 tokens. A 69-character tracking URL took 16.
- Other languages vary. The same idea can take more or fewer tokens than in English, depending on how well the tokenizer's vocabulary covers the language.
OpenAI's rule of thumb is that one token is about four characters of common English text, or about three quarters of a word, so 100 tokens is roughly 75 words (source). Treat that as an estimate. Each model family uses its own tokenizer, and they differ a lot: Anthropic says its current tokenizer fits roughly 555,000 English words into 1 million tokens, while its older models fit about 750,000 (models overview).
Token Counter Count tokens, words and characters and see how much of a context window you use.What fills a context window
Think of the context window as the model's working memory for one reply. Anthropic's documentation describes it the same way and lists what counts toward it:
- System instructions. The app's own instructions, which you usually do not see.
- The whole conversation. Every earlier message and every earlier answer, resent with each new turn.
- Files and images. A pasted PDF or a screenshot is turned into tokens too.
- Tool definitions and results. Web search results, code output and connected tools all take space.
- Reasoning. Models that "think" before answering spend tokens on that thinking.
- The reply itself. The answer the model is writing shares the same window.
Models also have a separate limit on output: the longest answer they can write in one go. It is usually much smaller than the window. A model with a 1-million-token window may still stop an answer at 64,000 or 128,000 tokens.
Why your chat forgets
There are four common reasons.
1. The conversation no longer fits. Each turn adds tokens. When the total approaches the limit, the app must make room. Chat apps often drop the oldest messages first; Anthropic notes that interfaces such as claude.ai can manage the window on a rolling "first in, first out" basis. Some apps summarize older turns instead, which keeps the gist but loses detail. Either way, the exact wording of something you said early on may be gone.
2. Long inputs are used unevenly. A model can technically see the whole window and still overlook a detail in the middle. The study Lost in the Middle (Liu et al., 2023) found that models performed best when the relevant information was at the start or end of the input and worse when it sat in the middle. Anthropic's docs call the general effect "context rot": as the token count grows, accuracy and recall degrade.
3. A new chat starts empty. Each conversation has its own context. Some assistants have a separate memory feature that saves facts about you between chats, but that is a different system with its own rules, not the context window.
4. The window is smaller than you think. The model's advertised maximum and the window your app actually uses can differ. Apps and plans set their own limits, and local runners often pick small defaults to save memory. As of October 2026, Ollama defaults to a 4K window on machines with less than 24 GB of VRAM (docs). That is about 3,000 words, so a local chat can forget after a few pages.
How big context windows are in October 2026
Context windows have grown fast, and the numbers change with every model release. These are examples from the vendors' own documentation, checked on October 19, 2026.
Context windows of current flagship models (checked October 19, 2026)
| Model | Context window | Max output per reply | Source |
|---|---|---|---|
| Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 | 1M tokens | 128K tokens | Anthropic |
| GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna | 1.05M tokens | 128K tokens | OpenAI |
| Gemini 4 Argon | 1,048,576 tokens | 65,536 by default | |
| Local models in Ollama | 4K to 256K by default, set by VRAM | Depends on the model | Ollama |
These are API limits for developers. The chat apps built on the same models may use a smaller window depending on your plan, and they do not always say so. Older and smaller models commonly have windows of 128K or 200K tokens.
What does a million tokens hold? At Anthropic's estimate of about 555,000 words, it is a few long novels' worth of text. A large window does not mean the model treats every page equally, as the previous section explains.
How to keep a long chat on track
You cannot change a model's window, but you can decide what goes into it.
- Put the essentials up front. State the goal, the audience and the hard constraints in your first message, so they are clear from the start.
- Paste only what matters. Instead of a whole 80-page report, paste the two sections you are asking about. Fewer tokens also means the model is less likely to miss the important part.
- Restate key facts when you change topic. A one-line reminder such as "Remember: the budget is $5,000 and the launch is in March" costs a few tokens and prevents drift.
- Ask for a summary, then start fresh. When a chat gets long, ask the model to summarize decisions and open questions, then paste that summary into a new chat.
- Use files or projects for reference material. Many assistants let you attach documents to a project and retrieve the relevant parts when needed. That is retrieval-augmented generation; see what RAG is.
- Count before you paste. Check how many tokens a long document takes with the Token Counter so you know how much room is left for the reply.
Why tokens also decide what you pay
AI APIs bill per token, usually priced per million, and output tokens typically cost more than input tokens. Because the whole conversation is resent with each turn, a long chat costs more per message as it grows. Our guide on how to count tokens and estimate your API bill walks through the math.
In a chat app with a subscription you do not pay per token, but limits still apply: long conversations and large files use up your plan's message or usage allowance faster.
These cover the steps above: three long-context assistants, a local runner where you set the window yourself, and a notebook that answers from your files:
FAQ
What is a context window in simple terms?
It is the model's working memory: the maximum amount of text, counted in tokens, that it can consider when it writes a reply. Your messages, its answers, files and instructions all have to fit.
How many words is 1,000 tokens?
For typical English text, roughly 700 to 800 words with OpenAI's tokenizers. Other tokenizers differ: Anthropic's current one fits about 555 words into 1,000 tokens. Code, numbers and many other languages use more tokens per word.
Why does ChatGPT or Claude forget what I said earlier?
Usually because the conversation outgrew the context window and older messages were dropped or summarized, or because a detail in the middle of a very long chat was overlooked. A new chat also starts with an empty context.
Does a bigger context window make answers better?
Not automatically. It lets you include more material, but accuracy tends to drop as the input grows. A focused prompt with only the relevant text often works better than a huge one.
Is the context window the same as memory?
No. The context window is what the model sees in the current conversation. Memory features in some assistants store facts between conversations, and the app can add them to the context of a new chat.
Do images and files count toward the context window?
Yes. Images, PDFs and other attachments are converted into tokens and share the same window as your text.
