acceptodds
Under review as a conference paper at ICLR 2027

Compress What You See, Not What You Say: Anchored Context Distillation for Latent-Observation Software Engineering Agents

Abstract

Tool observations dominate the context of software-engineering agents, making long interaction histories costly to maintain. Existing context compression methods can discard information needed by later actions, while adapting agents to soft-token representations can compromise their original behavior. To reduce context while preserving action-critical information and agent behavior, we combine Latent Observations, Hard Actions (LOHA), a context layout that separates compressed history from text needed for exact reference, with Anchored Context Distillation (ACD), a training method that enables latent reading while constraining behavioral drift. LOHA compresses older tool observations into soft tokens while retaining the agent's own turns and the last K observations in text, providing compact access to historical information and exact access to recent content. To enable the agent to use this representation, ACD distills the base model's full-text predictions into the latent view while anchoring its behavior on plain-text inputs to the same base model. On SWE-bench Verified, K=3 reduces context per call by 43% for Qwen3-4B and 57% for SWE-Master-4B-RL, with resolve rates of 12.1% and 21.8% versus 14.5% and 27.5% for their uncompressed bases. A single-run recency sweep reaches 14.4% and 23.0% at K=8, with larger windows generally favoring task performance over compression. Under a 32K-token limit, Qwen3 with K=3 resolves 21.1% of a 199-instance subset versus 11.1% for the same adapted agent using full text. In concurrent single-GPU serving, it achieves 1.9 times that full-text agent's instance throughput.

Primary area:
foundation or frontier models, including LLMs
Keywords:
software engineering agents, context compression, soft tokens, latent context, knowledge distillation, tool use, large language models, SWE-bench

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6ygGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper compresses older tool observations of SWE agents into soft tokens (LOHA). Agent turns and the last K observations are kept as raw text. Training (ACD) uses a low-rank patch with two KL terms: one distills the base model's full-text predictions into the compressed view, and one anchors the model's full-text behavior to the original model. On SWE-bench Verified with two 4B models, K=3 cuts context per call by 43–57%. Resolve rate drops from 14.5% to 12.1% (Qwen3-4B) and from 27.5% to 21.8% (SWE-Master-4B). Additional results cover a 32K context limit, a K sweep, and serving throughput.

Critical strengths
  1. Clear design principle. "Compress what you see, not what you say" is well motivated. Keeping recent observations as raw text is justified by the finding that exact recitation at 16× compression is only about 15%.

  2. Honest and detailed cost accounting. The paper reports encoder compute, prefix-cache reuse, call counts, and throughput. Negative results are not hidden.

Critical weaknesses
  1. The method gives no net benefit over trivial baselines. On Qwen3, simple observation masking matches LOHA's resolve rate (12.2% vs. 12.1%) with less context (20.1K vs. 24.8K tokens per call) and higher throughput (64.7 vs. 63.4 instances/GPU-hour). Pruning baselines reach a higher resolve rate on both models. Total compute goes up rather than down (2.10 + 1.64 PF vs. 1.82 PF for the base; about 2.12 PF even with encoder caching), because the compressed agent makes more calls (60.9 vs. 37.9).

  2. ACD does not preserve behavior, which is its stated goal. After ACD training, the model already loses performance with no compression at all (14.5% → 13.2% and 27.5% → 24.3% at K=∞), so the anchor does not prevent behavioral drift. The main positive results (the 32K limit and the 1.9× throughput) are measured mostly against this degraded model. Against the original base, the gains are small (63.4 vs. 58.1 instances/GPU-hour) or come from a single run on a 199-task subset.

Questions for authors
  1. Is there any setting where LOHA beats both the uncompressed base and masking on resolve rate and total compute at the same time?

  2. Does the 32K result (42 vs. 31 tasks against the base) hold up across multiple seeds?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 6ygG: Tight-Budget Advantage, Base-Relative Results, and Behavior Preservation

Thank you very much for your careful review, for recognizing the design principle and the transparent cost accounting, and for your clear and direct questions. We address them in turn; section and table numbers refer to the submission.

Q1 (a setting where LOHA beats both the base and masking on resolve rate and compute). We appreciate this pointed question. The closest is a tight context budget: under a 32K limit, LOHA resolves the most of all arms (34.3 per run vs. 30.2 for the base and 27.7/30.5 for the two masking variants; six seeds) while using less compute than either masking variant (decoder 0.74 vs. 0.90 and 0.95 PF per trajectory). Of these, only the base is cheaper (0.21 PF), because 42% of its trajectories are cut off at the cap. At the full window, two points change the trade-off:

  • The 2.12 PF you cite is the submission's replay estimate with each observation encoded once (Section 3.4; Appendix B.5), whereas Table 1's 1.64 PF counts encoder input per request, without reuse. Instrumenting the encoder forward shows that the serving engine already performs this reuse, skipping regions inside the prefix-cache hit and caching encoder outputs by content: the encoder costs 0.008 PF per trajectory, and LOHA's total is 2.11 PF against 1.82 PF for the base.
  • LOHA reduces context per call by 43–57% relative to the base.

Q2 (32K across seeds). Thank you for this question. All five arms now have six seeds (s1–s6) on the same 199 instances. LOHA has the highest mean (Q1): +2.1 points over the base (95% CI [0.1, 4.3]) and +3.4 over base + masking ([1.2, 5.7]). It resolves at least as many instances as the base in every seed and hits the 32K cap less than half as often (34.7 vs. 83.7 trajectories per run, fewer in all six seeds). Per seed, LOHA resolves 42, 33, 32, 35, 33, and 31 against 31, 25, 32, 31, 33, and 29 for the base.

W1 (comparison with simple baselines). As Section 3.1 notes, the pruning baselines that resolve more keep larger contexts (29–49K tokens per call, against 22–25K for LOHA), and Table 1 shows that masking with the same window makes as many calls as LOHA (60.6 vs. 60.9), so the extra calls come with removing older observation text rather than with latent encoding. We would also like to respectfully clarify that all reduction baselines, including masking, ran on the adapted model, so they isolate the context layout. Over 499 instances with 2–3 seeds per arm:

Same adapted modelK=3 minus maskingK=3 minus plain text
SWE-Master-4B-RL+4.2 [+2.0, +6.4]−2.5 [−4.9, −0.2]
Qwen3-4B−0.1 [−1.9, +1.8]−1.1 [−3.2, +1.1]

On the stronger agent, keeping the history in latent form is significantly better than deleting it, so the compressed observations are read and used. Masking on the untrained base is a different comparison, since the weights change too, but it is the relevant zero-training alternative. Under a 32K limit, LOHA resolves 3.4 points more than it (34.3 vs. 27.7, six seeds); on SWE-Master at the full window the two are comparable (54/53 vs. 54/50 on 199 instances). LOHA's distinguishing property is that older observations are retained in compressed form rather than discarded, within one model that also runs on plain text.

W2 (does ACD preserve behavior). Thank you for this question.

  • Adaptation itself has no significant cost on Qwen: pooled over seeds, the adapted full-text agent trails its base by a non-significant 1.3 points (95% CI [−3.1, +0.6]), and per-seed McNemar tests (Appendix B.1) do not separate the two on either family (Qwen p=0.19, 1.00; SWE-Master p=0.13, 0.12).
  • The anchor does limit drift, as designed. New: on full-text agent trajectories, the adapted Qwen model has a KL of 0.014 nats per token from the base (97.7% top-1 agreement); removing the anchor raises this by 70%. Task success shows no significant change, because the base-teacher latent objective keeps drift in a safe range; the agent collapses only when the full-text reference is external (drift 0.089–0.235).
  • We attribute the larger SWE-Master gap to the adaptation budget, not the layout: SWE-Master-4B-RL was post-trained with full-parameter SFT and RL, whereas ACD adapts it through a rank-64 patch (<4% of decoder parameters) on CoderForge trajectories it did not generate. Consistent with this, more than half of the gap (−3.1 [−5.7, −0.6] of −5.6 points) is already present in full text, where the layout plays no role, so higher-rank or on-policy adaptation may narrow it.
  • The 32K gain is measured against the original base, not the adapted model: LOHA resolves 2.1 points more than the untrained base (95% CI [0.1, 4.3]; Q2). Relative to the base, as you suggest, throughput rises by 7% through the 90th completion.

We hope these results clarify what LOHA + ACD provides and how its costs compare with the alternatives. Thank you again for your time and feedback; we would be happy to provide any further details.

Thank you again for your review. As the discussion period closes on November 18, we wanted to check whether our response addresses your questions. The revised PDF reports measured encoder work (total 2.11 vs. 1.82 PF for the base; Section 3.4), the six-seed 32K results against the original base (Section 3.2; Appendix B.2, Table 5), pooled comparisons of the adapted agent with its base (Appendix B.1, Table 4), and the anchor separation with measured drift (Appendix B.4, Table 8). If these resolve your concerns, we would be grateful if you would consider updating your assessment.

Reviewer_6ygGOpenReview ↗

Official Comment

Thanks for the new experiments. I maintain my rating.

  • No setting beats both the base and masking on resolve rate and compute. At the full window LOHA still resolves fewer tasks (12.1 vs. 14.5; 21.8 vs. 27.5) at higher compute (2.11 vs. 1.82 PF), and resolved tasks per GPU-hour stay below both at every reported point.
  • The 32K gain is marginal: +2.1 points, and the seed-level interval [-0.3, 4.5] covers zero.
  • Removing the anchor does not change resolve rate (p=0.69, 0.54), so its value for task success is not shown.
Reviewer_qfYDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper combines a mixed context layout with behavior-preserving adaptation for coding agents. LOHA compresses older tool observations into soft tokens while keeping assistant turns and the most recent K observations in text. ACD teaches the adapted decoder to match its base model on compressed inputs while anchoring predictions on ordinary text. Experiments on SWE-bench Verified with Qwen3-4B and SWE-Master-4B-RL measure task success, context usage, and serving behavior. Compression reduces context size but lowers full-window resolve rates; its clearest task benefit appears under a 32K context cap.

Critical strengths
  1. The design targets a real distinction in tool use: understanding an old observation and reproducing an exact string for an edit are different requirements. Retaining recent observations and actions in text is a simple way to accommodate that distinction. The recency sweep and literal read-back diagnostics support the claim that a better reconstruction score alone is not enough to preserve agent behavior.

  2. The evaluation separates representation savings from actual rollout and serving costs, and includes both the original base model and the adapted model in full text. Under the 32K cap, LOHA resolves 42/199 tasks versus 22 for adapted full text and 26 for masking, with paired tests reported. The paper also reports the negative full-window results, encoder overhead, and cache behavior. I lean toward acceptance for this useful design and empirical analysis, with the practical benefit limited to the regimes actually demonstrated.

Critical weaknesses
  1. The benefit over deploying the original agent remains limited. At full windows, resolve rates fall from 14.5% to 12.1% and from 27.5% to 21.8%, while estimated total compute increases after including the encoder. The 1.9× serving improvement is relative to the slower adapted full-text model; the original base reaches 58.1 instances/GPU-hour versus 63.4 for LOHA, and masking reaches 64.7. At 32K, the direct comparison against the original base is 42 versus 31 resolved tasks with p=0.052 in one run. These are useful results, but context compression and completions per hour do not establish an overall improvement in successfully resolved tasks per unit cost.

  2. The role of the anchor is not fully isolated. The main teacher comparisons change the teacher for both objectives; the separated teacher/anchor experiment is a 20-task diagnostic, selected from tasks solved in an earlier base run, and its reference differs in cached-logit width. A same-initialization control with the anchor removed, keeping the latent objective and data fixed, is missing. The evidence supports self-anchoring over the tested external references, but is weaker for how much the anchor itself prevents behavioral drift.

Questions for authors
  1. With the same data, initialization, cached targets, and latent loss, what changes when the full-text anchor is removed? Can the teacher/anchor comparison be repeated on the random 199-task subset with a common cached-logit width?

  2. On the same serving workload, what are successfully resolved tasks per GPU-hour and total cost for LOHA, the original base, and masking, including the long-trajectory tail? This would distinguish throughput gains from changes in task success.

  3. Does the 32K advantage over the original base persist across evaluation seeds? The current paired comparison uses one run on 199 instances, so its stability would determine how confidently the tight-context result supports deployment.

2 comments
AuthorsOpenReview ↗

Response to Reviewer qfYD: Anchor Isolated on the Random Subset, Tail-Inclusive Throughput, and Six Seeds at 32K

Thank you for the thoughtful review, your positive view of the design and analysis, and the precise questions. We ran all three requested analyses.

Q1 (isolating the anchor, random subset, common width). Thank you for this excellent suggestion. As you note, the submission separates teacher and anchor only on the 20-task diagnostic (Table 3b; Appendix B.4), where 30B/self resolved 12 and 9 tasks against 11 and 11 for self/self. The new students share data (the first 3,514 trajectories, as in Table 3b), initialization, rank, and latent loss, change only the reference, and are evaluated on Table 3b's random 199-instance subset. Drift is the exact full-vocabulary KL(p_base ‖ p_student), teacher-forced on 24 full-text training-corpus trajectories (155K assistant tokens).

Distill. teacher / anchor (cached width)Resolved (full text / K=3)Non-empty patchesDrift KL
self / self (top-16, Table 3b)28 / 291400.014
self / none, λ=0 (top-16)31 / 251270.024
self / self (top-64)33 / 291450.014
30B / self (top-64)31 / 321430.021
30B / 30B (top-16, Table 3b)15 / 14680.089
SWE-Master / SWE-Master (Table 3b)0 / 0220.235
  • With the anchor on the base, a stronger teacher does no detectable harm: at a common width, 30B/self vs. self/self gives McNemar p=0.83, 0.68, as in the diagnostic.
  • The anchor's reference decides collapse: with a 30B teacher, anchoring to the base instead lifts resolves from 15/14 to 31/32 (width alone: self/self top-64 vs. top-16, p=0.30, 1.0). External full-text references raise drift 6.5–17× over self/self, and resolve rates fall with it.
  • Removing the anchor raises drift, the quantity it targets, by about 70%, with no significant change in resolves (p=0.69, 0.54): the base-teacher latent objective already pulls the shared patch toward the base, keeping drift where all base-referenced students perform alike (25–33).

In short, taking the base as reference in both terms prevents collapse, and the anchor, the only direct constraint on the full-text view, cuts its drift by 42%.

Q2 (resolved tasks per GPU-hour, including the tail). Thank you for this more informative metric. Appendix B.5 reports throughput on this workload (one GPU, 100 instances, 16 concurrent) without the final tail, plus trajectory wall-clock percentiles. We now report the rate at the k-th completion (k over the time from the start of the run), including the tail:

inst./GPU-h at 90th / 95th / 99th / 100thest. resolved/GPU-h, same pointsPF/traj.
Base58.5 / 59.8 / 58.7 / 57.38.5 / 8.7 / 8.5 / 8.31.82
LOHA, K=362.6 / 61.6 / 49.6 / 34.47.6 / 7.5 / 6.0 / 4.22.11
Masking (same model)65.0 / 64.7 / 65.1 / 64.87.9 / 7.9 / 8.0 / 7.91.69
Same model, full text34.9 / 33.2 / 32.9 / 30.54.6 / 4.4 / 4.4 / 4.04.24

Resolved/GPU-h multiplies throughput by the 499-instance resolve rate, since one run on 100 instances gives a noisy rate.

  • Through the 95th completion, LOHA completes more instances per GPU-hour than the base (+7% at the 90th, +3% at the 95th).
  • The last two points reflect LOHA's last three trajectories, all failures: one hit the 200-iteration cap (47 min); two looped, repeatedly generating up to the 8,192-token output cap (85 min, and 112 min until the rollout timeout). The base's slowest took 34 min. These reflect agent behavior rather than serving overhead, yet they count fully in a fixed batch. We kept the protocol identical across arms.

The PF column uses encoder work measured at the encoder forward (0.008 PF for LOHA).

Q3 (32K across seeds). Appendix B.2 reports the single 32K run; all five arms now have six seeds (s1–s6) on the same 199 instances (overflow: trajectories stopped at the 32K cap):

s1s2s3s4s5s6mean ± sdoverflow/run
Base31253231332930.2 ± 2.983.7
Base + masking26292631282627.7 ± 2.149.0
LOHA, K=342333235333134.3 ± 4.034.7
Same model, full text22302532263328.0 ± 4.355.0
Masking (same model)26303132323230.5 ± 2.425.7
  • LOHA has the highest mean of all five arms: LOHA minus base is +2.1 points (95% CI [0.1, 4.3]), averaging six seeds per instance and bootstrapping over instances; a seed-level paired t-interval gives [−0.3, 4.5]. Against masking on the base it is +3.4 ([1.2, 5.7]).
  • LOHA resolves at least as many instances as the base in every seed, and ranks first or tied first in five of the six.
  • LOHA also overflows less than the base and than full text on the same weights in all six seeds.

Thank you again; we believe the new results strengthen the design and analysis you valued, and would be glad to provide more details.

Thank you again for your detailed review. As the discussion period closes on November 18, we wanted to check whether the new analyses address your questions. The revised PDF includes the teacher/anchor separation on the random subset at a common cached width with measured drift (Appendix B.4, Table 8), tail-inclusive throughput (Appendix B.5), and the six-seed 32K results (Section 3.2; Appendix B.2, Table 5). If they resolve your concerns, we would be grateful if you would consider updating your assessment.

Reviewer_n76pOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LOHA and Anchored Context Distillation (ACD) for compressing software-engineering agents’ interaction histories. LOHA replaces older tool observations with soft tokens while retaining assistant turns and recent observations as text to support precise tool calls and code edits. ACD trains the agent to use compressed observations by matching the original model’s full-text predictions while constraining behavioral drift on ordinary text inputs. Experiments on SWE-bench Verified examine task performance, context usage, and serving efficiency, showing a trade-off between compression and task success, with benefits most apparent under constrained context windows. Ablations investigate the recent-observation window and distillation choices.

Critical strengths
  • Practical motivation. The method recognizes that coding agents need both compact historical context and exact source text for editing. Retaining recent observations provides a straightforward way to accommodate these requirements.

  • Transparent evaluation of trade-offs. The paper reports performance degradation alongside context savings and distinguishes encoder costs, cache effects, and serving throughput. This helps clarify where the approach offers practical benefits.

Critical weaknesses
  1. Substantial performance degradation. Under full context windows, LOHA + ACD reduces task success on both backbones, with a particularly pronounced drop on SWE-Master. Even the adapted model using full-text inputs underperforms the original model, indicating that adaptation itself compromises the agent’s capabilities despite the behavioral anchoring objective.

  2. Limited practical gains over simpler baselines. On Qwen3, the method achieves essentially the same resolve rate as observation masking, while pruning baselines retain higher task performance on both backbones. Moreover, shorter contexts do not translate into lower total computation: including encoder costs, estimated compute per trajectory exceeds that of the original models. Qwen3 also requires approximately 61% more model calls, substantially offsetting the per-call savings. Given the additional training and deployment complexity, the practical advantage remains limited, particularly outside constrained-context settings.

  3. Compatibility with inference optimizations needs clarification. Converting older observations from text to soft tokens modifies previously processed prefixes and reduces prefix-cache reuse. The paper should clarify compatibility with optimized attention implementations such as FlashAttention and quantify any additional serving overhead. Although soft-token inputs do not inherently preclude FlashAttention, the interaction with standard inference optimizations remains insufficiently characterized.

Questions for authors

See Weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer n76p: Standard Attention Stack, Measured Compute Within 16% of the Base, and Adaptation Cost

Thank you for your careful review and for recognizing the practical motivation and the transparent cost reporting. Section and table numbers refer to the submission.

W3 (compatibility with optimized attention). The submission quantifies part of it: Table 1 reports the prefix-cache share (92.5% for LOHA, 96.9% for the base, 91.5% for masking), and Appendix B.5 reports per-call decoder latency on the serving workload (median 1.71 s for LOHA, 2.04 s for the base, 1.44 s for masking). LOHA is a vLLM (V1) multimodal model plugin:

  • Soft tokens fill placeholder positions as multimodal embeddings. The decoder runs the standard FLASH_ATTN backend (FlashAttention-2) with paged KV, chunked prefill, and CUDA graphs; nothing in the attention path is LOHA-specific.
  • Prefix caching works unchanged: block hashes include each memory region's content hash, and encoder outputs are cached by content.
  • New measurements on the same workload:
    • Rendering and region extraction take a median of 19 ms per call, against ~1.6 s of generation.
    • As you note, advancing the window invalidates part of the prefix. LOHA recomputes 1,133 tokens per call, against about 480 for the base and 1,255 for masking, which also edits its prefix. The extra ~650 tokens are about 3% of its 20.8K-token context, and its per-call latency stays below the base's. (Plain text on the same weights recomputes 7,743, as its contexts overflow the KV cache.)

The prefix-cache cost is therefore bounded and similar to that of any recency-window method.

W2 (practical gains and total compute).

  • Compute. Table 1 counts encoder input per request, without reuse (1.64 PF); Section 3.4 and Appendix B.5 estimate by replay that encoding each observation once lowers total LOHA compute to 2.12 PF. New: instrumenting the encoder forward (100 Qwen K=3 trajectories) shows that the serving engine already realizes this reuse, skipping regions inside the prefix-cache hit and caching encoder outputs by content; encoder work is 9.3K tokens per trajectory, 0.5% of the request-level count:
Qwen3-4B, 262Kdecoder PFencoder PFtotal PF
Base1.82—1.82
LOHA, K=32.100.008 (no-reuse count: 1.64)2.11
Masking (same model)1.69—1.69

Total compute is within 16% of the base, while context per call falls by 43% (Qwen) and 57% (SWE-Master).

  • Calls. Section 3.1 (Trajectory cost) discusses the extra calls on Qwen; same-weights masking with the same window makes as many (60.6 vs. 60.9; Table 1), so they come with removing older observation text, not with latent encoding. On SWE-Master, calls are similar (81.5 vs. 85.0) and decoder input per trajectory falls from 4.40M to 1.81M.
  • Throughput. 62.6 vs. 58.5 instances per GPU-hour for the base (+7%) through the 90th of 100 completions, with lower per-call latency (W3).
  • Tight context budgets. Under a 32K limit, LOHA has the highest resolve rate of five arms over six seeds (+2.1 points over the base, 95% CI [0.1, 4.3]; +3.4 over masking on the base) and overflows less than half as often (34.7 vs. 83.7 trajectories per run).
  • Baselines. As Section 3.1 notes, the pruning baselines keep larger contexts (29–39K tokens per call on Qwen, 34–49K on SWE-Master, against LOHA's 24.8K and 22.2K). We also clarify that the reduction methods in Tables 1–2 ran on the adapted model; under this control, a latent history beats deletion on SWE-Master by +4.2 points (95% CI [+2.0, +6.4]).

W1 (degradation from adaptation).

  • Adaptation itself has no significant cost on Qwen: pooled over seeds, the adapted full-text agent trails its base by a non-significant 1.3 points (95% CI [−3.1, +0.6]), and per-seed McNemar tests (Appendix B.1) do not separate the two on either family (Qwen p=0.19, 1.00; SWE-Master p=0.13, 0.12).
  • This is what the anchor is for: keeping full-text behavior aligned with the original. New: on full-text trajectories, the adapted Qwen model has a KL of 0.014 nats per token from the base (97.7% top-1 agreement); removing the anchor raises drift by 70%, and external references raise it 6.5–17× and collapse the agent.
  • We attribute the larger SWE-Master gap to the adaptation budget, not the layout: SWE-Master-4B-RL was post-trained with full-parameter SFT and RL, whereas ACD adapts it through a rank-64 patch (<4% of decoder parameters) on CoderForge trajectories it did not generate. Consistent with this, more than half of the gap (−3.1 [−5.7, −0.6] of −5.6 points) is already present in full text, where the layout plays no role, so higher-rank or on-policy adaptation may narrow it.
  • When context is scarce, the adapted agent outperforms the original: under a 32K limit, LOHA resolves 2.1 points more than the untrained base (95% CI [0.1, 4.3]; W2).

We hope the serving characterization and measured encoder cost address your soundness concerns, and would be grateful if you would take them into account.

Thank you for the additional experiments and clarifications. I still have a few concerns regarding the practical value of the proposed method.

W1. Performance degradation

I understand the authors' explanation that the performance drop, particularly on SWE-Master, may be related to the limited adaptation capacity of the rank-64 patch and the mismatch between the training trajectories and the original RL-trained agent. However, this also raises concerns about whether the current adaptation approach is suitable for LOHA. Context compression is only meaningful if comparable task performance can be maintained; otherwise, reducing context length alone offers limited practical value. The additional results on KL divergence and token-level agreement are helpful, but they do not fully address the gap in downstream task performance, especially on SWE-Master.

W2. Practical efficiency

The additional encoder measurements are helpful, and I appreciate the clarification regarding encoder caching. However, the reported total computation is still higher than that of both the original model and observation masking on Qwen3. This is particularly concerning given that LOHA (K=3) also incurs a noticeable performance drop on both backbones, making the overall efficiency-performance trade-off less convincing.

I also think the comparison with pruning baselines remains somewhat incomplete. While these methods retain larger contexts, they achieve higher resolve rates on both backbones. It would be useful to understand whether LOHA still offers an advantage under comparable context budgets, rather than comparing methods with substantially different context lengths.

NEW: W4. Comparison with related context management approaches

I also noticed that the paper cites AgentFold [1] and Context-Folding [2] in the related work section, but does not clearly discuss how LOHA differs from these approaches. In particular, Context-Folding also evaluates on SWE-Bench Verified and reports promising results in maintaining task performance with a much smaller active context. So, what advantages latent observation compression offers over actively folding or summarizing historical interactions, and whether these approaches could provide better efficiency-performance trade-offs under similar context budgets?

[1] AgentFold: Long-Horizon Web Agents with Proactive Context Folding. [ICLR]

[2] Scaling Long-Horizon Agent via Context Folding. [ICML]

Thank you for the follow-up and for engaging with the new results.

W1 (performance at the full window). We agree that full-window parity is the ideal, but context compression is usually assessed as an efficiency–effectiveness trade-off, and full-window losses are common even at far milder compression than LOHA's 16×:

  • On SWE-Bench Verified, observation masking costs Gemini 2.5 Flash (thinking) 4.0 points and LLM summarization 9.0, yet Lindenbauer et al. (2025) call context management "an economic necessity" because it halves cost. With the compact Qwen3-Coder-Next (3B active), every pruner loses resolve rate at only 8–14% fewer input tokens (LongCodeZip −10.6 points, SWE-Pruner −4.2, SWE-Pruner Pro −1.2; Wang et al., 2026). Context-Folding trails full-context ReAct by 6.0 points before task RL (Sun et al., 2025), and soft-token prompt compression retains 62–73% of QA ability (Li et al., 2024).
  • In our tables, no reduction method reaches the SWE-Master base, including pruners that keep 34–49K tokens per call (21.8–24.0% vs. 27.5%). They share LOHA's adapted weights, and LongCodeZip (1.01× compression) matches the adapted full-text agent (24.0% vs. 24.3%), so most of the gap comes from adapting an RL-trained policy. On the same weights, the layout costs 1.1 points on Qwen (95% CI [−3.2, +1.1]) and 2.5 on SWE-Master ([−4.9, −0.2]) while more than halving context per call.
  • The value of compression lies in what the saved context buys: less memory and latency per call (W2) and longer horizons within a fixed window. When the window binds, LOHA outperforms the original agent: under a 32K limit it resolves 2.1 points more than the untrained base (95% CI [0.1, 4.3]; at least as many in each of six seeds) and 3.4 more than masking on the base.

W2 (efficiency trade-off).

  • At an equal budget. The 32K experiment holds the context budget fixed for all arms. LOHA has the highest resolve rate and uses less decoder compute than either masking variant (0.74 vs. 0.90 and 0.95 PF per trajectory); only the base is cheaper (0.21 PF), because 42% of its trajectories are cut off at the cap.
  • Against pruners. On SWE-Master, Self-Prune resolves exactly as many tasks as LOHA (109) with 52% more context per call (33.8K vs. 22.2K) and 8% more compute (4.70 vs. 4.36 PF); SWE-Pruner and LongCodeZip resolve 11 more at 1.6–2.2× LOHA's context and about 1.8× its compute (7.8 PF). On Qwen, the pruners compress mildly (representation ratio 1.0–1.5× vs. 2.2× for LOHA) and resolve at the level of the same-weights full-text agent (67–71 vs. 66.0 ± 2.8).
  • Against masking. On SWE-Master, LOHA costs 3% more compute than masking on the same weights (4.36 vs. 4.22 PF) and resolves 4.2 points more ([2.0, 6.4]). On Qwen, the extra decoder compute over the base comes from additional calls that same-weights masking also makes (60.6 vs. 60.9).
  • Serving. In concurrent serving, KV memory and latency often bind before FLOPs: at their mean context lengths, the KV cache holds about 20 LOHA trajectories against 15 for the base, and LOHA has lower per-call latency (1.71 vs. 2.04 s) and 7% higher throughput.

W4 (AgentFold and Context-Folding). Thank you for pointing to these. They act on a different axis: folding replaces completed spans of the trajectory with a summary the agent writes, whereas LOHA changes how each raw observation is represented and keeps it.

  • What is retained. After a fold, the raw observations are gone, and only what the agent chose to write survives; a misjudged fold cannot be undone. LOHA keeps every older observation readable at 16× compression; with the weights fixed, keeping them rather than deleting them gains 4.2 points on SWE-Master.
  • What it takes. Context-Folding reaches its result through end-to-end RL of a 36B model on 740 SWE tasks. GRPO also lifts full-context ReAct to 57.4 (folding: 56.4), so their FoldGRPO agent (58.0) is 0.6 points above an RL-trained full-context agent, while making 1.7× as many tool calls (96.5 vs. 55.4). AgentFold fine-tunes a 30B model on trajectories from a dedicated generation pipeline with rejection sampling, as prompting alone does not produce reliable folds, and is evaluated on web search. LOHA needs no rollouts, rewards, or new trajectories: ACD distills offline from cached teacher distributions through a rank-64 patch, and LOHA generates no summary tokens.
  • Same budget. At a 32K active context, both improve on the uncompressed agent by a similar relative margin: folding before RL 49.2 vs. 43.6 for 32K ReAct (+13%), and LOHA 34.3 vs. 30.2 (+14%) with a 4B model. The models and harnesses differ, so we read this as consistent evidence rather than a ranking.
  • Complementary. Folded summaries are agent-written text, which LOHA keeps verbatim; a folding agent could hold its working-context observations in latent form.

We hope this clarifies where LOHA offers value; we would be glad to provide further details.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Anchor Isolation, Measured Encoder Cost, Six-Seed 32K Results, and Summary of Revisions

We sincerely thank all reviewers for their careful reading and constructive feedback, which led us to run new experiments, measure serving costs directly, and upload a revised PDF. The main findings are below; each reviewer thread has the details, and Section/Table numbers refer to the revised PDF.

1. Isolating the anchor (qfYD, 6ygG). New students share data, initialization, rank, and latent loss and change only the reference; they are evaluated on the random 199-instance subset at a common cached width, and drift is the exact full-vocabulary KL from the base on full-text trajectories (Appendix B.4, Table 8).

  • With the agent itself as both references, drift is 0.014 nats per token (97.7% top-1 agreement); with the anchor on the agent, a stronger 30B teacher does no detectable harm.
  • The anchor's reference decides collapse: with the 30B teacher, anchoring to the 30B gives 15/14 resolved tasks and anchoring to the base gives 31/32. External full-text references raise drift 6.5–17×.
  • Removing the anchor raises drift by about 70% (0.014 to 0.024), the quantity it targets, without a significant change in resolve rate, as the base-teacher latent objective already pulls the shared patch toward the base.

2. Compute and serving (n76p, qfYD, 6ygG).

  • Encoder work, now measured. The submission counted every compressed region in every request (1.64 PF per Qwen trajectory), the cost without reuse. Instrumenting the encoder forward shows that the serving engine runs the encoder only for regions outside the prefix-cache hit and caches its outputs by content: 9.3K tokens and 0.008 PF per trajectory. Total LOHA compute is 2.11 PF against 1.82 PF for the base, in line with the submission's 2.12 PF replay estimate (Tables 1–2, Section 3.4, Appendix B.5).
  • Serving path. LOHA runs as a vLLM multimodal plugin on standard FlashAttention-2 with paged KV, chunked prefill, CUDA graphs, and prefix caching. Advancing the window recomputes 1,133 tokens per call against about 480 for the base, and per-call latency stays below the base's. Through the 90th of 100 completions, LOHA completes 7% more instances per GPU-hour than the base; Appendix B.5 also reports the rate including the tail.

3. Is the latent history used? (n76p, 6ygG). The reduction baselines in Tables 1–2 run on the adapted model, which Table 1 now states. With the weights fixed, keeping older observations latent instead of deleting them raises SWE-Master's resolve rate by 4.2 points (95% CI [2.0, 6.4]) and is on par on Qwen; adaptation alone costs a non-significant 1.3 points on Qwen (Appendix B.1, Table 4). Masking on the untrained base, a zero-training reference, is comparable to LOHA on SWE-Master at the full window and, on Qwen under a 32K limit, 3.4 points below it (CI [1.2, 5.7]).

4. 32K with six seeds (qfYD, 6ygG). All five arms now have six seeds. LOHA has the highest mean (34.3 resolved; base 30.2): +2.1 points over the base (95% CI [0.1, 4.3]) and +3.4 over masking on the base. It resolves at least as many instances as the base in every seed and hits the 32K cap less than half as often (34.7 vs. 83.7 trajectories per run) (Section 3.2, Figure 3, Appendix B.2, Table 5).

Comparisons with pruning baselines at an equal budget and with context folding are in the reply to n76p.

Summary of revisions. The changes are limited to corrections and the analyses requested in the reviews; the method, the resolve rates in the main tables, and the structure of the paper are unchanged.

  • Corrections. Encoder work measured at the encoder call (Tables 1–2, Sections 2.4 and 3.4, Appendix B.5), replacing the per-request count and the statement that old observations are re-encoded at every call; six-seed 32K results instead of one run (abstract, introduction, Section 3.2, Figure 3a–b); the note that the reduction baselines run on the adapted model (Table 1).
  • New in the appendix. Pooled paired comparisons and masking on the untrained base (B.1, Table 4); the six-seed 32K table (B.2, Table 5); teacher/anchor separation with measured drift (B.4, Table 8); the serving path, prefix recomputation, and tail-inclusive throughput (B.5).

Thank you; we would be glad to provide further details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.