01
Who does the rewrite?
Self and external rewriting were tested. A fine-tuned specialist or cheaper model is a next experiment; a classifier could select blocks or decide when to compact.
REALM @ EMNLP 2026
Why Pay for Long Reasoning When Rewrite Do Trick?
Rewrite a model’s earlier reasoning into compact state. Then see how much reasoning the same model generates next.
Useful analysis can be expensive to express. This study asks how much of that expression can change without retraining the target or telling it to be brief.
The intervention
Original history
Need to compare A and B.Budget is 10. A costs 12.12 is greater than 10.So A exceeds the budget.A must be rejected.Need B’s remaining constraints.
Compact history
Budget: 10.Reject A: cost 12.Check B’s constraints.
Same target
Same weights
Illustration. Only earlier assistant reasoning changes at the controlled fork.
01 / Measured
Shorter history. Shorter continuations.
151/172
paired continuations used less reasoning
6/6
tested routes had a negative median change
20–50%
route-level median reductions
New reasoning, after the rewrite
95% CI−80%−60%−40%−20%0+20%
Accepted histories, equal-horizon token comparisons. Acceptance rates differ across routes.
See the paired effects from the paper
Twenty comparisons grew longer and one was unchanged. The intervention does not shorten every continuation.
02 / Retained coding-agent runs
Count the rewrite, too.
Six of eight configurations cost less with rewriting included. Only the two MiniMax configurations reduced rewrite-inclusive model-request time.
64.0% of cache-distinct reasoning-history words were removed across the treatment runs. These are history words, not billed input tokens.
Target / compactorTask scoreSpend change
Spend includes captured rewriting charges. Nemotron target costs use a common-tariff estimate. All assigned tasks remain, including unsolved and model-limit outcomes.
What about task quality?
Controlled exact success was 165/174 with original history and 166/174 with compact history, with changes in both directions across routes. This does not establish quality noninferiority. Deployment scores above also vary by model and compressor.
03 / Design space
One simple compactor.
Five knobs to turn.
02
What can it see?
The deployment compactor sees a reasoning block and its action name. More prior context might improve retention, at an additional input cost.
03
How much at once?
Rewrite each block, a whole work phase, or an evolving state record. Tool calls and observations stay separate from the reasoning being compacted.
04
What should survive?
The paper aims at information preservation. Selective forgetting could be more aggressive, with a different risk: discarding something the agent later needs.
05
Hide work behind tools.
Example: rewrite 2s, tool 3s
After tool arguments are fixed, rewrite an immutable pre-result snapshot while the tool runs. Resume only after both finish. This scheduler is proposed, not benchmarked.
04 / Proposed training
Make the short style
the default.
Use a frozen copy of the model with compact history as the teacher. Train the student under its original history to match that continuation behavior.
Compact history H꜀Budget 10 · reject A · check B
→Teacher θFrozen · same model
Original history HOriginal reasoning and messages
→Student φTrainable
KL targets from teacher↳ update φ only
Match the next-token distribution
TeacherStudentIllustrative probabilities
KL(teacher ∥ student)0.000
The KL objective and gradient flow
L = ∑ₜ ∑ᵥ pθ(v | H꜀, y<ₜ) log [pθ(v | H꜀, y<ₜ) / pφ(v | H, y<ₜ)]
The teacher sees compact reasoning; the student sees the original context. Compare their distributions along a shared continuation prefix and backpropagate into φ only. The bars interpolate a toy student distribution to explain the objective; they are not a training run or a measured gain.
Training on the rewritten traces directly is another option, using supervised token loss. A weaker compactor can generate text for the teacher to read; it does not need to share the teacher/student tokenizer.
This proposal needs quality-filtered targets and evaluation for capability retention. Related work: Self-Distillation Enables Continual Learning.
The broader bet
The capability may already be there.
The tested models generated shorter continuations from changed histories without retraining. I suspect compact reasoning is much more widespread, perhaps universal, in current open-weight reasoning models. We should incentivize future generations to use it while keeping their problem-solving ability.
