acceptodds
Under review as a conference paper at ICLR 2027

Does Compaction Really Matter?

Abstract

Long horizon agentic tasks are now pushing LLM agents past their context window. To continue the task, the agent scaffold has to free up the context window, a process known as context compaction. Compaction is generally lossy by design: agents can silently lose task constraints, instructions, or their reasoning behind earlier decisions. Although compaction is a standard component of coding agent scaffolds, it has not been studied systematically with frontier models on realistic, long horizon agentic tasks. Previous work evaluate smaller open weight models on toy or synthetic trajectories. We fix the agent scaffold (OpenCode), implement five compaction strategies inside it and evaluate four frontier models on three long horizon benchmarks (PaperBench, ProgramBench, PostTrainBench). For the three 1M context models, the compaction threshold does not measurably affect the task performance, whereas GPT-5.3-Codex, the model with the smallest context window, performs worse at lower thresholds on PaperBench and less clearly, on PostTrainBench. No compaction method outperforms default self-summarization, and methods that discard more content cost some models up to 20 points while leaving other models the same. Compaction can, however, silent remove task constraints: on ProgramBench, the first compaction summary already drops about one third of the task's anticheating rules on average, and this rises monotonically with the number of compactions. On real user-agent sessions from SWE-chat, instructions about how the agent should work and what it should not do are dropped 5-7 times more than specifications of what to build. We argue that current benchmarks are compaction-friendly: i.e., the objective is fixed and the agent's progress stays on disk, so most of what compaction discards can be recovered, except constraints that don't exist anywhere else.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, agent, scaffold, compaction, long-horizon, tool-use

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3SxdOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper systematically investigates the effects of context compaction on long-horizon LLM agents, focusing on downstream task performance and the preservation of user-specified constraints. Using OpenCode as a fixed agent harness, the authors evaluate 4 frontier models across 3 long-horizon benchmarks, comparing different compaction thresholds and five compaction strategies. The results show that compaction thresholds have limited effects on task performance for models with 1M-token context windows, while no alternative strategy consistently outperforms default self-summarization. However, analyses of ProgramBench and real user-agent sessions from SWE-chat reveal that compaction can progressively discard task constraints, particularly instructions about how the agent should work and what it should avoid. The authors further suggest that existing benchmarks may underestimate such risks because task objectives remain fixed and much of the agent's progress can be recovered from persistent files.

Critical strengths
  1. Meaningful and practically relevant research topic. Context compaction is widely used in long-horizon LLM agents, yet its effects on downstream task performance and instruction preservation remain insufficiently understood. This paper addresses an important practical question for designing and evaluating agent systems, particularly as agents increasingly execute complex tasks requiring multiple rounds of compaction.

  2. Relatively systematic experimental design. The paper evaluates multiple frontier models, compaction thresholds, and compaction strategies under a fixed agent harness, reducing potential confounding factors introduced by different agent frameworks. The experiments also span three distinct long-horizon task settings, providing a broader perspective on the effects of compaction across models and workloads.

  3. Insights beyond downstream task performance. Rather than relying solely on final benchmark scores, the paper additionally examines the preservation of task constraints during compaction. The analyses of ProgramBench and real user-agent sessions from SWE-chat reveal that process-related instructions and scope restrictions are more likely to be discarded than feature requirements. This distinction provides a useful perspective on the limitations of existing benchmarks, where stable task performance does not necessarily imply faithful preservation of user instructions.

Critical weaknesses

1. Limited Model Coverage and Compaction Threshold Range. The experiments do not fully explore how compaction affects models with different context window sizes. For the three models with 1M-token context windows, compaction thresholds are limited to 160K–380K tokens, leaving the effects of compaction near the context window limit unexplored. Meanwhile, GPT-5.3-Codex is the only model evaluated with a context window smaller than 1M tokens, making it unclear whether the observed sensitivity to lower compaction thresholds generalizes to other models. These limitations weaken the generalizability of the findings regarding the relationship between compaction thresholds, context window sizes, and downstream task performance.

2. Relatively Small Task Coverage. Although the experiments span three long-horizon benchmarks, the evaluation includes only nine tasks or task configurations in total (3 from PaperBench, 2 from ProgramBench, and 4 from PostTrainBench). Such a limited sample may not adequately capture task-level variability, potentially weakening the reliability and generalizability of the findings. Expanding the evaluation to more tasks, ideally the full benchmark sets where feasible, would provide stronger evidence for the reported conclusions.

3. Limited Analysis of the Downstream Impact of Constraint Erosion. While the paper provides evidence that compaction can progressively discard user constraints and reports some observed instruction violations, the practical consequences of such information loss remain insufficiently explored. In particular, the extent to which constraint erosion directly contributes to subsequent instruction violations or degraded task outcomes remains unclear. A more systematic analysis of agent behavior after compaction could help establish the practical significance of this finding.

Questions for authors
  1. Have the authors measured whether the selected tasks would exceed the 1M-token context window without compaction? For tasks that can be completed within this limit, have the authors evaluated a no-compaction baseline? Such a comparison would help clarify whether compaction is necessary or beneficial for these tasks.

  2. This paper suggests that existing benchmarks with fixed objectives and persistent files may be relatively insensitive to compaction, while the SWE-chat analysis highlights greater risks of constraint loss in real user-agent interactions. Have the authors considered evaluating how compaction affects downstream task performance in multi-turn interactive settings, where user instructions may evolve over time? It would also be valuable to investigate whether different compaction strategies exhibit different effectiveness in such settings.

2 comments
AuthorsOpenReview ↗

Response 1/2

We thank the reviewer for the careful reading and the constructive suggestions. The general response lists what changed in the revised PDF.

Weakness 1 - Model coverage and threshold range.

We agree that the submission did not explain how we chose the thresholds. The variable we meant to vary is the number of compactions per run, because each compaction is an opportunity to lose information. And the threshold is how we control it. We chose thresholds that produce a wide range of compaction counts: for the GPT models at a threshold of 160K we have between 15 to 28 compactions per run on average, and at 380K we have fewer than 2 on average (App. 6.8, Fig. 8, Table 2). Section 4.1 now explains this.

On the effect of the context window size, our existing results show that the fraction of the window does not explain the GPT-5.3-Codex sensitivity. GPT-5.3-Codex is at 160K is 40% of its window, and the 1M models at 380K are 38% of theirs. At this matched fraction, Codex scores 0.38 on PaperBench while Claude Opus 4.6 and GPT-5.4 score about between 0.6 and 0.65. The 1M models are also flat at 16% of their window, a lower fraction than any Codex condition.

Conversely, our range already reaches 95% of the Codex window (380K), and there Codex matches the 1M models. Its degradation appears only at low thresholds, where it compacts most often: Fig. 5 shows Codex declining from about 8 compactions per run onward, while GPT-5.4 does not degrade even at 16 or more. This points to a model-specific fragility to repeated compaction rather than an effect of window size. We agree that a single model with a smaller window cannot show whether this generalizes, and we now state explicitly in the paper that the Codex result may be specific to that model.

For the 1M models, thresholds near 90-95% of the window would rarely trigger compaction on these benchmarks: even at 380K, Opus 4.6 compacts only 0.71 times per run on average and GPT-5.4 1.96 times (Table 2). Such thresholds would therefore mostly measure runs without compaction (see our answer to Question 1)

Weakness 2 - Task coverage.

The reviewer's count of nine was correct for the submission. The revision adds one PostTrainBench target (BFCL), for 11 task configurations in total: 3 PaperBench tasks, 2 ProgramBench tasks and 6 PostTrainBench configurations (2 base models × 3 targets). The constraint-erosion analysis also uses the lua task, and the reward-hacking analysis uses lua and luajit.

We could not afford more, because the cost multiplies. Each task is run under 10 conditions (six thresholds with the default strategy, and four alternative strategies at 160K), for three models with two seeds each. That is about 60 runs per task before reruns of failed submissions. Each run takes 6 to 10 hours and costs 30-50 USD on average in API credits, and around 30-40 USD in GPU time.One more task therefore costs roughly 3,600 USD to 5,400 USD. We chose to cover several frontier models, thresholds and strategies on realistic long-horizon tasks rather than many tasks with fewer conditions.

No strategy beats self-summarization on any of the three benchmarks (Fig. 3). The 1M models are insensitive to the threshold on PaperBench and ProgramBench, while on PostTrainBench the error bars are too wide to conclude either way (Section 4.1). We state our claims as conditional on this task set in the Limitations paragraph.

AuthorsOpenReview ↗

Response 2/2

Weakness 3 - Downstream impact of constraint erosion.

We added two analyses of our existing runs.

First, a rule-aware re-scoring of our existing runs (App. 6.11, Fig. 11). A run that violates a rule gets a score of 0, and we compare this with the raw score. We can do this on ProgramBench and PostTrainBench. When we bin runs by number of compactions, the pooled raw score stays within 0.07 of the task mean in every bin. The rule-aware score is 0.03 below the task mean with no compaction, 0.14 below at 4 to 7 compactions, and 0.28 below at 32 or more. This shows that the standard score does not register rule violations: runs that violate rules score about as well as the ones that don’t. But on its own, it does not show that compaction causes the violations.

Second, we tested whether rules that a summary drops are broken more often afterwards (App. 6.13). On ProgramBench, a rule that the summary dropped was broken before the next compaction in 23 of 3,940 cases (0.6%). A rule that it kept was broken in 3 of 1,040 cases (0.3%). On SWE-chat, 7 of 24 dropped constraints that came up again were violated (29%), against 26 of 137 preserved ones (19%). Dropped rules are thus violated about twice as often in both settings. The counts are small and neither difference is statistically significant (p = 0.33 and p = 0.28, Fisher exact test), so we do not claim a causal link. The same appendix shows when violations start (Table 4). For Claude Opus 4.6, 10 of 12 violating runs break a rule before any compaction. For the GPT models, almost all first violations come after a compaction, and most follow a summary that had dropped the rule. We corrected the statement in Section 4.2 accordingly. We see the causal question as the main open question raised by our results.

Q1: Would the tasks exceed 1M tokens, and is there a no-compaction baseline?

We didn’t run a separate no-compaction baseline, but Fig. 5 shows accuracy as a function of the number of compactions per run, including runs with no compaction. For the 1M models, runs with no compaction score about the same as runs with one or more compactions. On the benchmarks and models we study, compaction therefore doesn’t seem to affect performance, which supports our argument that current benchmarks are compaction-friendly. Regarding tasks exceeding 1M tokens, we did not measure this directly. At 380K, Claude Opus 4.6 averages 0.71 compactions per run (Table 2), so most of its runs compact at most once and would fit in the 1M window. GPT-5.4 averages 1.96, so its typical run is close to the limit.

Q2: Multi-turn settings with evolving instructions.

We agree, and this is the main conclusion of our paper: such benchmarks are currently missing, and they are where compaction should matter most. We could not create and run a new interactive benchmark within the discussion period, and we list it as future work (Section 5).

Our SWE-chat analysis is the closest setting we could study. It shows that process instructions and scope exclusions given by real users are dropped about 6 times more often than feature requirements.

Whether compaction strategies differ in such settings remains open, but even on our fixed-objective benchmarks, strategies that discard more (removing reasoning blocks, summarizing with a smaller model) already cost some models up to 20 points on PaperBench (Fig. 3), so we expect larger differences when instructions exist only in the context.

Reviewer_ov7GOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper compares five compaction strategies and different compaction thresholds in OpenCode. It evaluates task performance on three benchmarks and studies constraint erosion in ProgramBench and SWE-chat.

Critical strengths

The paper is clearly written, and the figures are easy to follow.

Critical weaknesses
  1. The experiments use one scaffold and few tasks: three from PaperBench and two from ProgramBench. This limits the generality of the conclusions about compaction thresholds and strategies.

  2. Missing comparisons at the same fraction of each model’s context window. The evaluation uses the same absolute compaction thresholds across models with different context windows. The 160K–380K range covers 40–95% of GPT-5.3-Codex’s window, but only 16–38% of the 1M context windows. The authors should add threshold comparisons at the same fractions of each model’s context window, including near 90–95%, and compare compaction strategies at these settings. This would clarify whether the findings depend on the absolute threshold or its fraction of the context window.

Questions for authors

see above

1 comment

We thank the reviewer for the careful reading and the insightful comments. The general response lists what changed in the revised PDF.

Weakness 1: one scaffold and few tasks

Task count: The submission evaluated 3 PaperBench tasks, 2 ProgramBench tasks and 4 PostTrainBench configurations (2 base models and 2 target benchmarks). The revision adds BFCL as a third PostTrainBench target. This gives 6 PostTrainBench configurations and 11 task configurations in total. The three benchmarks test different capabilities: paper replication, software reimplementation and ML post-training. The constraint erosion and reward hacking analyses also use two additional tasks for reward hacking. We exclude both from the accuracy results because they are reward hacked too often (App. 6.10).

Why not more tasks: The limit is cost. Each task is run under 10 conditions: six thresholds with the default strategy, and four alternative strategies at 160K. With three models and two seeds, this is about 60 runs per task, before reruns of failed submissions (App. 6.7). Each run takes 6 to 10 hours and costs 30-50 USD on average in API, and around $30-40 for GPU time. One more task therefore costs roughly 3,600 USD to 5,400 USD. We chose to cover several frontier models, thresholds and strategies on long-horizon tasks, instead of more tasks under fewer conditions, because prior work already studies compaction on shorter and synthetic tasks (Section 2.2). How far the conclusions hold: No strategy beats self summarization beyond the confidence intervals on any of the three benchmarks (Fig. 3). The two 1M models are insensitive to the threshold on PaperBench and ProgramBench. On PostTrainBench the confidence intervals are too wide to conclude anything about the threshold, and the revision now says it (Section 4.1).

One scaffold: Fixing the scaffold was deliberate. It separates the effect of the compaction method from scaffold-specific factors such as tool formats, retry logic and system prompts. The cost is generality, which we state in Limitations. The SWE-chat sessions give limited evidence from a second scaffold: they were produced with Claude Code and other models, and they show the same pattern of constraint loss (Section 4.2). They say nothing about thresholds or strategies under other scaffolds.

Weakness 2: comparisons at the same fraction of each model's context window

We agree that the submission did not explain how we chose the thresholds, and that an absolute threshold means different things for a 400K and a 1M window.

Why absolute thresholds. The variable we meant to vary is the number of compactions per run, because each compaction is an opportunity to lose information. The threshold is how we control it. At 160K, GPT-5.4 averages 15 compactions per run and GPT-5.3-Codex 28. At 380K, the averages are between 0.7 and 2.0 for all three models (App. 6.8, Table 2). Section 4.1 now says this.

Matched fractions. GPT-5.3-Codex at 160K is 40% of its window, and the 1M models at 380K are 38% of theirs. At this matched fraction, Codex scores 0.38 on PaperBench while Opus 4.6 and GPT-5.4 score about 0.60–0.65. The 1M models are also flat at 16% of their window, a lower fraction than any Codex condition. The absolute threshold does not explain the result either: at the same 160K threshold, GPT-5.4 compacts 15 times per run and loses no accuracy. So neither the fraction of the window nor the absolute threshold explains the Codex sensitivity, which appears to be specific to that model.

Near 90 to 95% of the window: For GPT-5.3-Codex, our range already reaches 95% of its window (380K). There Codex matches the 1M models. Its degradation appears only at low thresholds, where it compacts most often: Codex declines from about 8 compactions per run onward, while GPT-5.4 does not degrade even at 16 or more (App. 6.5, Fig. 5). For the 1M models, a threshold at 90 to 95% of the window would rarely trigger compaction on these benchmarks. At 380K, Opus 4.6 already averages only 0.71 compactions per run and GPT-5.4 1.96 (Table 2). Runs with zero or one compaction score no differently from the others (Fig. 5).

Strategies near the limit: Strategies can only differ when compaction triggers. We compared them at 160K, where every model compacts several times per run: 4.9 times for Opus 4.6, 15.1 for GPT-5.4 and 27.7 for Codex (Table 2). At 95% of the Codex window the average is 0.76 compactions per run, and at 90 to 95% of a 1M window it would be lower still. A comparison at those settings would mostly compare runs in which no strategy was applied.

What we could not test. We could not run new experiments during the discussion period, so we cannot add near-limit conditions for the 1M models. They would also need tasks several times longer than ours for compaction to trigger. Section 4.1 states that our range does not test compaction near the scaffold default for the 1M models.

Reviewer_12faOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work studies context compaction in a fixed scaffold with four frontier models on PaperBench, ProgramBench and PostTrainBench, varying the compaction threshold and method. The findings:

  • the threshold barely matters for 1M-context models;
  • no method beats self-summarization;
  • some methods hurt some models;
  • summaries drop task rules, and the loss grows with more compactions.
Critical strengths
  • Addresses a practical question for agent builders, on realistic long-horizon tasks with frontier models.
Critical weaknesses
  • No task counts rule-following toward the score. Such a task is needed to show that scores drop under compaction while standard benchmarks do not.
  • Constraint erosion is measured as a rule missing from the summary, judged by an LLM with no human check. Reward hacking starts before the first compaction, so erosion is not linked to violations. The real-session analysis is small.
  • The Complexity Trap [1], which compares summarization and observation masking on SWE-bench Verified with several models, is not discussed.
Questions for authors
  • Can you add a task where rules or changing user instructions are part of the score, to test the compaction-friendly claim directly?
  • Can you add more runs or tasks, and test thresholds close to the context limit?
  • Do dropped rules lead to more violations after compaction in any setting you ran?
  • How well does the LLM erosion judge agree with human labels?
2 comments
AuthorsOpenReview ↗

Response 1/2

We thank the reviewer for the careful reading and the concrete questions. The general response lists what changed in the revised PDF.

Question 1 & Weakness 1 - Rule-following as part of the score

We agree that a benchmark with changing user instructions would test the claim most directly. The conclusion of our paper is that such benchmarks are currently missing, and we could not build and run one within the discussion period.

Instead of adding such a new task, we added a “rule-aware” re-scoring of our existing runs (new App 6.11, Fig. 11). A run that violates a rule gets a “rule-aware” score of 0, and we compare it with the “raw” score. This is possible on the two benchmarks where violations are detected. On PostTrainBench the benchmark’s judge flags training on test data or swapping the model. On ProgramBench we use our reward-hack judge (App. 6.4.2). PaperBench has no violation judge, so we leave it out.

We bin runs by their number of compactions. The pooled raw score stays within 0.07 of the task mean in every bin. The rule-aware score is 0.03 below the task mean with no compaction, 0.14 below at 4 to 7 compactions, and 0.28 below at 32 or more. We want to be precise about what this shows. It shows that the standard score does not register rule violations: runs that break the rules score about as well as runs that do not. A benchmark that scored compliance would rank these runs differently. It does not by itself show that compaction causes the violations, and the appendix lists the caveats.

Question 2 - Adding more runs, tasks and thresholds.

More runs or tasks: Each task is run under 10 conditions: six thresholds with the default compaction strategy, and four alternative strategies at 160K. With three models and two seeds, this is about 60 runs per task, before reruns of failed runs (App. 6.7). Each run takes 6 to 10 hours and costs 30-50 USD on average in API costs, and around 30-40 USD for GPU costs. One more task therefore costs roughly 3,600 USD to 5,400 USD. The revision adds one PostTrainBench target (BFCL), for 11 task configurations in total. We could not afford more, and we state our claims as conditional on this task set (Limitations).

How we chose the thresholds: The submission did not explain this well. The variable we meant to vary is the number of compactions per run, because each compaction is an opportunity to lose information. The threshold is how we control it. We chose thresholds that produce a wide range of compaction counts: for the GPT models at a threshold of 160K we have between 15 to 28 compactions per run on average, and at 380K, the averages are between 0.7 and 2.0 (App. 6.8, Table 2). Section 4.1 now makes this clearer.

Thresholds close to the limit: For GPT-5.3-Codex, our range already reaches 95% of its window (380K). There, Codex matches the 1M models. Its degradation appears only at low thresholds, where it compacts most often (App. 6.5, Fig. 5). For the 1M models, a threshold at 90 to 95% of the window would rarely trigger compaction on these benchmarks. At 380K, Claude Opus 4.6 already averages only 0.71 compactions per run and GPT-5.4 1.96 (Table 2). Runs with zero or one compaction score no differently from the others (Fig. 5). Section 4.1 notes that our range does not test compaction near the scaffold default for the 1M models.

Question 3 & Weakness 2 - Do dropped rules lead to more violations?

We tested whether rules that a summary drops are violated more often afterwards in two settings (new App. 6.13). The first is ProgramBench, that defines three anti-cheating rules, and the second is real SWE-chat sessions, where the rules are constraints given by the user. In both settings, dropped rules are violated around twice as often (in ProgramBench, for example, a violation followed 0.6% of rules that the summary had dropped and 0.3% of rules it had kept). But violations are very rare in our results (26 in ProgramBench, 33 in SWE-chat), so the difference could be due to chance. So we don’t claim that dropped rules cause violations more often. We provide more details on the methodology and describe the limitations of our results in App 6.13.

Question 4 & Weakness 2 - Human validation of the judges.

One author labeled 110 compaction summaries for whether they restate the three rules. The human - LLM judge agreement is high (Weighted Cohen’s kappa on the number of rules dropped per summary is 0.87), which means our erosion judge largely agrees with human judgment. We updated App 6.4.3 with detailed methodology and results on how we compute this agreement score.

SWE-chat: We agree it is small (44 sessions, 269 constraints). We report Wilson 95% intervals (Fig. 4b), and a second judge from another model family agrees on 88.8% of constraints (kappa 0.68) and shows the same pattern across categories (App. 6.12.2).

AuthorsOpenReview ↗

Response 2/2

Weakness 3 - The complexity trap.

Thank you for the pointer; we now discuss The Complexity Trap (Lindenbauer et al., 2025) in Section 2.2. It compares LLM summarization with observation masking in SWE-agent on SWE-bench Verified, across five model configurations. It finds that masking matches, and sometimes slightly exceeds, the solve rate of summarization, at about half the cost of the unmanaged agent.

Our results agree where the two studies overlap. Our tool-output pruning condition clears old tool outputs and is used together with summarization. It stays close to the self-summarization baseline, and no more elaborate strategy beats that baseline (Fig. 3).

Our study differs in three ways. First, our runs last 6 to 10 hours, and one model averages 28 compactions per run at the lowest threshold, so later summaries are written from earlier summaries. Second, we vary when compaction triggers. Third, we measure what the summaries lose, which a solve rate does not capture. We do not report cost, which is their main metric.

Comments on the paper

1 comment

We thank all the three reviewers for their careful reading and responses. We have uploaded a revised PDF, and we list below what changed, as some numbers differ from the submission.

We could not launch new agent runs during the discussion period: each run takes 6 to 10 hours and costs 30-50 USD on average in API costs, and around 30-40 USD for GPU costs.

Changes since the submission

  1. Tasks: PostTrainBench now includes BFCL as a third target benchmark. This gives 6 PostTrainBench configurations (previously 4) and 11 task configurations in total.
  2. Gemini 3.1 Pro Preview: Gemini averages at most one compaction per run, even at the 160K threshold (Table 2). Its results are therefore not informative. The threshold and strategy claims now rest on Claude Opus 4.6 and GPT-5.4 ("the two 1M models"). Gemini's PaperBench results remain in App. 6.6.
  3. New analyses: Accuracy as a function of the number of compactions in a run (App. 6.5, Fig. 5), and the number of compactions at each threshold (App. 6.8, Fig. 8, Table 2).
  4. PostTrainBench: We no longer claim a threshold effect for GPT-5.3-Codex on PostTrainBench. Its confidence intervals overlap at every threshold (Section 4.1).
  5. Constraint erosion on ProgramBench: The three rules are now described as the task prompt states them. The judge now returns a binary preserved or dropped verdict, and we validated it against human labels (App. 6.4.3). Rules dropped per compaction moved from 2.62 to 2.73 for GPT-5.3-Codex, from 1.87 to 1.76 for GPT-5.4, and from 0.68 to 0.51 for Claude Opus 4.6. The ordering of the models and the conclusions are unchanged.
  6. SWE-chat: We revised the category taxonomy and re-assigned the constraints. Overall preservation is unchanged at 81%. Drop rates are now 6.2% for feature requirements (was 6.4%), 35.9% for process instructions (was 46.2%) and 36.8% for scope exclusions (was 30.2%). The headline ratio is now "about 6 times" instead of "5 to 7 times".
  7. Added in response to the reviews: A rule aware re-scoring of existing runs (App. 6.11); the rationale for the threshold range (Section 4.1); a discussion of The Complexity Trap (Section 2.2); and additions to Limitations and Future Work. The individual responses give the details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.