Why AI Gets Worse in Long Chats

The first ten minutes are great. The AI understands what you want, the answers are sharp, and you think you have found a proper working partner.

Then, somewhere around message thirty, it starts to slip. It forgets a rule you set at the beginning. It hands you back something you already rejected. You correct it, it apologises, and it makes the same mistake again two replies later.

This is not you losing patience, and it is not the model having a bad day. Long conversations really do get worse, it happens to every model on the market, and researchers have now measured how much.

What happens to AI answers in a long conversation

Researchers at Microsoft and Salesforce ran an experiment that is easy to describe. They took tasks a model can normally handle, like writing a bit of code or answering a question from several documents, and asked for them two ways.

In the first version, the whole request arrived in one complete message. In the second, the exact same information was dripped out over several turns, the way a real person actually talks: a vague opening ask, then a detail, then a correction, then another detail.

Same model, same information, same final goal. The only thing that changed was the delivery.

Across more than 200,000 simulated conversations and six kinds of task, every top model they tested did significantly worse in the drip-fed version. The average drop was 39%. The work was posted in May 2025 and presented as an oral paper at ICLR 2026, one of the field’s main research conferences, in April.

The more interesting finding is how it fails. The models did not become less capable. They became less reliable, which is a different and more annoying problem. Ask the same model the same multi-turn question ten times and you get a much wider spread of quality than you would in a single message.

The reason is simple enough to picture. When your first message is vague, the model does not wait for clarity. It quietly fills in the gaps with a guess, commits to that guess, and starts producing an answer built on it. If the guess was wrong, everything after it inherits the error. The researchers found that once a model heads down the wrong path early in a conversation, it tends not to find its way back, even when you correct it directly.

Why a bigger AI memory does not fix long chats

The obvious objection is that models now have enormous memories. Some read a million words at once. Surely a fifty-message chat is nothing.

That is true about capacity and false about quality, and the gap between those two things is the whole story. If you want the full picture of how that memory works, we covered what an AI context window actually is separately. The short version: it is the amount of text the model can hold in front of it while it answers.

Holding text is not the same as using it well. A research team at Chroma tested 18 leading models in July 2025 and found the same pattern in all of them. Accuracy fell as the input got longer, and it fell well before anyone got near the advertised limit. A model that can technically accept a million words can already be noticeably worse at fifty thousand.

Two details from that work stand out because they are so counterintuitive:

• One irrelevant passage is enough to hurt. Adding a single distracting chunk of text, not a hundred, measurably lowered accuracy.
• Neatly ordered text can be worse than a shuffled pile. Models scored higher when the surrounding material was jumbled than when it was logically arranged.

There is also an older, well-replicated finding that explains a lot of everyday frustration. A Stanford-led study back in 2023 showed that models are best at spotting information at the very start or the very end of a long input, and much worse at finding it in the middle. Your careful instruction from message eight is now buried in the middle of the pile. That is the worst possible place for it.

Put those together and the picture makes sense. Your long chat is not a tidy record the AI consults. It is a growing heap of text where your important line competes with forty messages of small talk, dead ends and corrections you have already moved past.

The AI labs know, and they built a workaround

This is not a fringe complaint. It is documented by the companies themselves.

Anthropic’s developer documentation describes a feature called compaction, which automatically summarises the older parts of a conversation once it gets long. The stated reason is blunt: “as a conversation grows, response quality degrades”, so the old material gets replaced with a short summary.

Read that again, because it is the useful bit. The official fix for a long conversation is to throw most of it away and keep a summary. You can do exactly the same thing by hand, for free, in any chat app, and you do not need to wait for a feature to do it for you.

What to do when a chat starts drifting

Here is the practical translation. Match the symptom to what is actually going on.

And five habits that prevent most of it in the first place:

1. Front-load the whole request. The single strongest finding in this research is that one complete message beats the same information delivered in pieces. Spend an extra thirty seconds writing the full ask.
2. Restart instead of arguing. When a chat goes wrong, opening a new one costs you nothing. Paste in only the good parts.
3. Write your own summary at the end of a long session. Ask the AI for a short brief of decisions made so far, check it, then open a fresh chat and paste it in as your first message.
4. Keep separate topics in separate chats. One thread for the work project, another for the holiday plan. Mixing them puts distractors in both.
5. Switch model when one gets stuck. A different model has no idea your last conversation went badly. It comes to the problem clean, and it will often disagree with the first one in useful ways, which we looked at in why AI models give you different answers.

That last one is the cheapest trick of the lot, and the one people skip because they only have one AI app open.

Where this advice falls short

A few honest caveats, because the fix is not universal.

Starting over has a real cost. If you have spent an hour teaching an AI the tone of your newsletter, throwing that away hurts, and a summary never captures everything. In that case, keep a short reusable brief in a note on your computer and paste it into every new chat. You are keeping the useful part and dropping the clutter.

The research also tested models against simulated users following a script, not the messy back-and-forth of a real person who can spot a wrong turn and stop it. A sharp human who catches the mistake in message three will do better than these numbers suggest. That is an argument for reading the early replies carefully, when a correction still costs almost nothing.

And some jobs genuinely need a long single thread, like working through a large document you cannot summarise without losing the point. A model built for long documents, such as Claude Opus 4.7, handles that better than most, but it does not remove the problem. For those jobs, keep the thread and restate your core instruction every few messages so it stays near the end of the pile, where the model looks hardest.

The short version

Long AI conversations do not fail because the machine gets tired. They fail because the model made a quiet assumption early on, built everything on top of it, and then buried your best instruction in the middle of a pile of text it does not read evenly.

Once you know that, the habits are obvious. Say the whole thing at once. Start fresh when it drifts. Keep your brief somewhere you can paste it again. And when one model is stuck in a loop, stop arguing with it and hand the same question to a different one.