Sometimes the biggest performance gains don’t come from a new model. They come from flipping the right switches. OpenAI’s latest technical write-up is a striking example of that: two API configuration changes tripled GPT-5.6‘s scores on the ARC-AGI-3 benchmark — one of the most demanding tests of abstract reasoning in AI evaluation today. No new weights. No architecture overhaul. Just better use of what was already there. That’s either a story about engineering efficiency or a quiet admission that most people have been running these models wrong. Probably both.
What Is ARC-AGI-3 and Why Should You Care?
The ARC-AGI benchmark series, originally created by François Chollet at Google and later stewarded through the ARC Prize Foundation, is specifically designed to resist the kind of memorization and pattern-matching that lets large language models ace most standard tests. ARC-AGI tasks require fluid reasoning — the ability to spot an abstract rule from a small number of visual examples and apply it to a new case. It’s closer to an IQ test than a knowledge exam.
ARC-AGI-3 is the latest and hardest iteration. Most frontier models have historically struggled with it, which is precisely why it’s become a credible signal for genuine reasoning capability rather than benchmark overfitting. When OpenAI says GPT-5.6 tripled its score, they’re not talking about a trivia leaderboard. They’re talking about one of the harder proxies for general intelligence currently in use.
So the claim is significant. And the mechanism behind it is worth unpacking carefully.
The Two Settings: Reasoning Retention and Conversation Compaction
OpenAI’s engineers identified two specific API behaviors that, when enabled together, produced the dramatic score increase. Here’s what each one does:
1. Retaining Reasoning Across Turns
By default, GPT-5.6‘s internal reasoning — the chain-of-thought the model builds before producing a final answer — gets discarded between turns in a multi-turn conversation. That means every time the model responds to a follow-up prompt, it starts reasoning from scratch using only the visible conversation history.
When reasoning retention is enabled, the model carries forward its internal reasoning state. It doesn’t have to rediscover what it already worked out. For tasks like ARC-AGI-3, where solving a puzzle often requires iterative hypothesis testing across multiple exchanges, this is enormous. The model can build on its prior reasoning rather than reconstructing it cold each time.
Think of it like the difference between a mathematician who can reference their scratch paper versus one who has to redo every calculation from memory with each new question.
2. Conversation Compaction
The second setting is about context efficiency. Long conversations accumulate tokens fast, and when a context window fills up, either old content gets dropped or performance degrades. Conversation compaction compresses earlier parts of the conversation into a dense summary, freeing up active context space for the reasoning that matters right now.
For benchmark tasks that unfold over many turns — which ARC-AGI-3 tasks often do when a model is working through solutions step by step — this keeps the model’s effective memory clean and usable rather than cluttered with stale tokens.
Together, the two settings address the same core problem from different angles: making sure the model’s reasoning doesn’t get lost or crowded out as a task progresses.
The Numbers
OpenAI reports that with both settings disabled, GPT-5.6 posted baseline ARC-AGI-3 scores that were respectable but unremarkable for a frontier model. With both settings enabled, scores tripled. They also noted improvements in cost efficiency — compaction in particular reduces redundant token processing, which cuts API costs on long reasoning tasks. That’s a meaningful secondary benefit for anyone running these models at scale.
- Reasoning retention preserves the model’s internal chain-of-thought between conversation turns
- Conversation compaction compresses older context to free up active window space
- Combined effect: tripled ARC-AGI-3 scores, improved token efficiency
- No model update required — this applies to the existing GPT-5.6 API
- Cost impact: lower per-task API costs on extended reasoning sessions
What This Actually Means for Developers
Here’s the thing: this announcement is as much a documentation failure as it is a technical success. If two API settings this impactful were available all along, why weren’t they on by default? Why weren’t they more prominently documented? OpenAI is essentially telling developers that they’ve been leaving significant performance on the table because the right configuration wasn’t obvious.
That’s a UX problem dressed up as a benchmark win. To OpenAI’s credit, they’re surfacing it clearly now. But developers who’ve been benchmarking GPT-5.6 against competitors without these settings enabled have been making potentially flawed comparisons. That matters.
For anyone building agents or multi-turn reasoning applications — coding assistants, research tools, complex planning systems — these settings should almost certainly be enabled. The retained reasoning setting in particular changes the calculus for agentic use cases where a model needs to maintain a coherent problem-solving thread across many exchanges. We’ve written about how AI coding agents are now doing real science, and that kind of sustained, multi-step reasoning is exactly where retained context pays off most.
There’s also a competitive angle here. Anthropic’s Claude models have long been noted for their extended context handling, and Google’s Gemini family has made long-context performance a flagship feature. OpenAI catching up via smart API configuration rather than raw context window size is an interesting strategic choice. It suggests that model capability and model usability are increasingly separate problems — and that the latter might be just as important for real-world performance.
Implications for Benchmark Interpretation
This development should make everyone a little more careful about how they read AI benchmark results going forward. If a model’s score can triple based on two configuration flags, then benchmark comparisons that don’t control for those settings are essentially comparing different things. The research community and the AI press — this publication included — need to be more explicit about evaluation conditions when reporting numbers.
It also raises a question that’s been lurking around reasoning benchmarks for a while: are we measuring the model, or are we measuring the deployment setup? The answer, increasingly, is both. And that means the gap between a model’s theoretical ceiling and its practical performance in production is largely an engineering problem, not a research one.
OpenAI has been pushing hard on the cost-performance angle lately — something we covered in depth when GPT-5.6 launched with a focus on intelligence per dollar. This announcement fits that narrative tightly. Better scores at lower token cost is exactly the value proposition they want developers to internalize.
Key Takeaways for Developers and Researchers
- Enable reasoning retention for any multi-turn task that involves iterative problem-solving or hypothesis testing
- Enable conversation compaction for long sessions to maintain context quality and reduce token costs
- Re-run your benchmarks if you’ve been evaluating GPT-5.6 without these settings — your baselines may be inaccurate
- Treat configuration as a first-class variable when comparing models across providers
- Watch for similar settings in other providers’ APIs — this kind of optimization is likely not unique to OpenAI
Are These Settings Available to All API Tiers?
OpenAI’s write-up doesn’t carve out specific tier restrictions, suggesting both settings are available through the standard GPT-5.6 API. Developers with existing API access should be able to enable them immediately through the API parameters documented in OpenAI’s platform documentation. That said, compaction behavior may interact differently depending on how you’re managing conversation history on the client side, so testing in your specific pipeline is worthwhile before rolling it out to production.
Does This Change How GPT-5.6 Compares to Claude or Gemini?
Potentially, yes. If competitors’ benchmark numbers were generated under default conditions and GPT-5.6’s new numbers reflect optimized settings, you’re not comparing like with like. The honest answer is that we don’t yet have fully controlled head-to-head comparisons with these settings factored in. Anthropic’s Claude Opus 5 and Google’s Gemini models both have their own context and reasoning optimizations — whether those are defaults or opt-in is a question worth asking their respective teams.
Is ARC-AGI-3 a Reliable Measure of Real-World Intelligence?
More reliable than most, with caveats. ARC-AGI tasks are specifically designed to avoid the training-data contamination that inflates performance on knowledge benchmarks. But tripling a score via API settings does raise questions about whether the benchmark is capturing model intelligence or model infrastructure. Chollet and the ARC Prize Foundation have consistently updated the benchmark to stay ahead of these issues — expect further iterations as the tooling around frontier models gets more sophisticated.
If there’s a broader takeaway here, it’s that the frontier of AI performance is no longer just about model weights — it’s about the full stack of how those models are deployed, configured, and maintained across a conversation. OpenAI just made that point very clearly with three times the score and a two-checkbox solution. I wouldn’t be surprised if we see other providers publishing similar optimization guides in the coming months, because right now GPT-5.6 just made a strong case that their model was being systematically underrated — and that’s a competitive message that lands hard.
