acceptodds
Under review as a conference paper at ICLR 2027

ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement

14 views

Abstract

Recent work extends Recursive Self-Improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve their execution mechanisms from experience. However, achieving and demonstrating generalizable harness RSI remains challenging. First, existing approaches often evolve harnesses directly on evaluation benchmarks or subsets drawn from them, making it difficult to distinguish reusable harness improvements from benchmark-specific adaptation. Second, updates derived from individual trajectories can entangle systematic harness deficiencies with instance-specific reasoning and solution details, leading to task-specific modifications that transfer poorly to unseen tasks. Third, even when recurring behavioral deficiencies are identified, localizing them to the responsible components within a monolithic harness remains difficult. Whole-harness optimization can therefore entangle unrelated mechanisms and produce changes that are difficult to attribute and validate. We propose ModularRSI, a benchmark-disjoint, contrastive, and modular framework for generalizable harness evolution. ModularRSI contrasts successful and failed trajectories for the same task and aggregates evidence across tasks to identify recurring behavioral deficiencies. It further decomposes the evolvable harness into five functional modules: Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection. Each module is evolved independently within a restricted modification scope, after which validated improvements are integrated into a unified harness. To evaluate generalization beyond evolution experience, we additionally curate 2,000 executable evolution tasks from external data sources that are disjoint from downstream evaluation benchmarks. Experiments on TerminalBench 2.0 and SWE-Bench Verified show consistent improvements on unseen in-domain and cross-domain tasks, with the evolved harness also transferring across different foundation models.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Harness RSI, Agentic Agnet, Terminal Bench, Swe bench

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_QqXLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the generalization problem of harness evolution for LLM agents.

It proposes ModularRSI, a method that decomposes harness evolution into agent loop, tool use, observation management, context management, and task completion detection. It evolves each module independently and the independent evolved modules are integrated to construct the final harness.

Experiments evolve the harness on self-curated TerminalBench-like and SWE-like tasks and evaluate it on TerminalBench (TB) 2.0 and SWE-Bench-Verified, with DeepSeek-V4-Flash-Preview as the backbone. The evolved harness improves over the unevolved baseline and outperforms existing harness evolution methods under matched rollout budgets.

Critical strengths
  • the generalization problem of harness evolution is important, and the proposed method is well-motivated.
  • The ablations (Table 6) are comprehensive.
Critical weaknesses
  • The method is complex and involves multiple rounds of judgment and verification. I suggest adding a cost analysis that accounts for these steps. Is this modular decomposition still necessary for stronger models? It would be helpful to evaluate stronger models and check whether the modular approach still performs best.
  • The paper lacks details on the self-curated SWE-related and TB-related tasks. It is unclear whether these tasks are of high quality and to what extent they differ from TerminalBench 2.0 and SWE-Bench-Verified.
  • The comparison with existing harness evolution methods is limited. The paper would be stronger if Meta-Harness and AHE were evaluated on the full set of settings in Table 1.
Questions for authors
  • Why is the method called ModularRSI? A name like ModularHarnessEvolution seems more accurate.
  • Were the evolution conducted with a single random seed?
4 comments
AuthorsOpenReview ↗

Response (1/N)

We thank the reviewer for the constructive comments and questions. Our point-by-point responses are provided below. We will incorporate these clarifications and additional experimental results into the revised paper.

Q1. Cost analysis and evaluation using a strong model The LLM token consumption of ModularRSI primarily comes from six stages: Trajectory Collection, Contrastive Trajectory Analysis, Harness Modification, Diff Review, Function Library Management, and Cross-Module Integration.

We compare the total token consumption of ModularRSI with that of Meta-Harness and AHE, and provide a breakdown of ModularRSI's token costs below. All values are reported in millions of tokens (M).

Method / StageToken Cost (M)
Meta-Harness2,664
AHE1,489
ModularRSI (Total)1,218
├─ Trajectory Collection909
├─ Contrastive Trajectory Analysis191
├─ Harness Modification95
├─ Diff Review12
├─ Function Library Management9
└─ Cross-Module Integration2

ModularRSI consumes fewer tokens than both comparison methods. Trajectory Collection accounts for the majority of its computational cost, while the additional stages for trajectory analysis, harness modification, validation, and integration contribute a comparatively smaller overhead.

Q2. Is this modular decomposition still necessary for stronger models?

To investigate whether modular decomposition remains beneficial with a stronger foundation model, we use Claude Opus 4.6 as the backbone model for harness evolution and compare ModularRSI with joint all-module evolution. The results on Terminal-Bench 2.0 (TB2.0) are shown below.

MethodTB2.0 Accuracy (%)
No-Evolution Baseline66.67
Joint All-Module Evolution65.16
ModularRSI69.66

ModularRSI improves accuracy by 2.99 percentage points over the non-evolution baseline and by 4.50 percentage points over joint all-module evolution. In contrast, jointly evolving all modules reduces performance below the non-evolution baseline. These results suggest that independent module evolution remains beneficial even with a stronger foundation model, as it helps avoid the performance degradation observed when all modules are optimized jointly.

Q3. Details of the self-curated SWE-related and TB-related tasks

Please refer to our response to Q1 from Reviewer a7Es, where we provide further details on the construction and characteristics of the self-curated evolution tasks.

Q4. Comparison with existing harness evolution methods

To further address the reviewer's concern, we conducted a more comprehensive comparison with Meta-Harness and AHE under the experimental settings used in Table 1. We evaluate both in-domain and out-of-domain transfer on SWE-Bench Verified and TB2.0. The results are presented below (all metrics are percentages).

MethodEvolution SetSWE Eval. SettingSWE AccSWE Pass@3SWE Pass^3TB2.0 Eval. SettingTB2.0 AccTB2.0 Pass@3TB2.0 Pass^3
No-Evolution BaselineNo Evolution—73.4083.2062.80—47.5758.4330.34
ModularRSITB-RelatedOut-of-Domain75.8084.6066.20In-Domain52.4365.1735.96
ModularRSISWE-RelatedIn-Domain76.4085.4066.80Out-of-Domain49.4460.6734.20
Meta-HarnessTB-RelatedOut-of-Domain74.8084.4065.80In-Domain49.9461.8031.40
Meta-HarnessSWE-RelatedIn-Domain75.6085.0065.80Out-of-Domain48.3159.5530.34
AHETB-RelatedOut-of-Domain74.2084.8065.80In-Domain49.8164.0433.70
AHESWE-RelatedIn-Domain75.2084.6065.60Out-of-Domain48.4359.5532.34

ModularRSI consistently achieves the highest accuracy and Pass³ across all four benchmark/evolution-set combinations, outperforming Meta-Harness and AHE in both in-domain and out-of-domain settings. It also achieves the highest Pass@3 in three out of four settings, with the only exception being SWE-Bench Verified under TB-related evolution, where AHE achieves a marginally higher Pass@3 (84.80% vs. 84.60%). Overall, these results provide further evidence of the effectiveness of ModularRSI and its ability to generalize harness improvements across different task domains.

AuthorsOpenReview ↗

Response (2/N)

Q5: Why is the method called ModularRSI?

We use the term Recursive Self-Improvement (RSI) because our framework iteratively analyzes the agent's own execution trajectories, identifies deficiencies in its harness, and modifies the harness to improve subsequent executions. The updated harness is then used to generate new trajectories, forming a recursive feedback loop in which the agent continuously improves its own execution mechanisms.

Unlike parametric self-improvement, which updates the underlying model weights, our approach performs harness-level self-improvement without modifying the foundation model. The term Modular further emphasizes our core methodological contribution: decomposing the harness into functional modules and evolving them independently before cross-module integration. Therefore, we believe ModularRSI accurately reflects both the recursive self-improvement process and the modular evolution strategy underlying our framework.

Q6: Were the evolution experiments conducted with a single random seed?

The randomness in ModularRSI primarily arises from trajectory generation during evolution. Specifically, for each module, we generate three independent execution trajectories per task without enforcing a fixed or shared random seed. This stochastic sampling allows the evolution process to observe diverse execution behaviors and identify recurring harness deficiencies, rather than relying on a single trajectory. As a result, harness modifications are guided by patterns observed across independent rollouts, which helps reduce sensitivity to individual execution outcomes.

Due to the computational cost, we conduct one complete evolution run per experimental setting rather than repeating the entire evolution process with multiple random seeds. We will clarify this distinction in the revised paper.

AuthorsOpenReview ↗

Follow-up on Rebuttal Responses and Remaining Concerns

Dear Reviewer QqXL,

Thank you very much for your time and effort in reviewing our paper and for your valuable feedback.

As the rebuttal deadline is approaching, we would greatly appreciate it if you could let us know whether our responses have adequately addressed your concerns. If any questions or concerns remain unresolved, please feel free to let us know at your earliest convenience so that we can provide further clarification before the deadline.

Thank you again for your thoughtful comments and engagement in the review process.

Thank you so much for the detailed response. I’m leaning toward keeping my score.

Reviewer_a7EsOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ModularRSI, a framework for studying generalizable harness RSI. The authors construct an evolution dataset containing 1,000 terminal tasks and 1,000 SWE tasks across nine categories. They then decompose the harness into five modules—Agent Loop, Observation Management, Tool Use, Context Management, and Task Completion Detection—and evolve them independently. Experiments on TerminalBench and SWE-Bench, using multiple LLMs, show cross-domain performance improvements that provide some support for the paper’s motivation.

Critical strengths
  1. The paper is clearly written.
  2. he framework is clearly presented, and the experimental setup is relatively clean.
  3. The experimental results are clear and show consistent improvements across multiple models.
Critical weaknesses
  1. Could the authors provide more details on how the benchmark-disjoint evolution pool is constructed? How does its distribution differ from those of the evaluation benchmarks, such as Terminal-Bench and SWE-Bench?

  2. Why do the Terminal-Bench experiments only use version 2.0, without evaluating newer versions or related benchmarks, such as Terminal-Bench 4.0 and Terminal-Bench-Science?

  3. Have the authors analyzed where the improvements on Terminal-Bench come from? Is there a case-by-case analysis of the improvements on the evaluation tasks, beyond the evolution-task examples in Appendix K?

  4. Can the authors further clarify the benefits of evolving the modules independently before merging them, particularly the contribution of the final integration stage?

  5. Have the authors evaluated benchmarks with a larger domain gap? SWE-Bench and Terminal-Bench both involve CLI-based agent tasks, so the current experiments may provide limited evidence of broader cross-domain generalization.

  6. Many recent works already analyze trajectories to identify weaknesses and evolve agent harnesses. What is the main novelty of this paper?

Questions for authors

See Weaknesses.

6 comments
AuthorsOpenReview ↗

Response (1/N)

We thank the reviewer for the detailed comments and constructive questions. Below, we clarify our methodology and provide additional experimental evidence addressing the concerns.

Q1. How was the benchmark-disjoint evolution pool constructed, and how does its distribution differ from those of Terminal-Bench and SWE-bench?

A. Construction of the benchmark-disjoint evolution pool

Section 4 of our paper describes the construction of our self-curated evolution dataset, including data-source collection, conversion into Harbor-compatible tasks, and quality control through LLM-based filtering, op and no-op filtering, and manual review. We provide further details below.

Each Harbor-compatible task contains four components: query, solution.sh, test.py, and environment. The environment is specified by a Dockerfile that initializes an executable environment. The query describes the task requirements, completion criteria, locations of necessary data, and relevant tools. The solution.sh provides an executable reference solution, while test.py contains test cases and evaluation scripts for verifying whether the task requirements have been met.

Annotators independently identify sources relevant to the high-level task categories of Terminal-Bench and SWE-bench, including GitHub, Kaggle, Linux kernel documentation, and other relevant resources. We follow two construction procedures depending on the source type:

  • Sources containing existing problems. For problem collections and competition tasks, annotators convert the original problems into Harbor-compatible tasks. For example, for a cryptography competition problem involving RSA cryptanalysis, the original problem description is converted into a query, an executable reference solution is implemented in solution.sh, and the required data and dependencies are packaged into a Docker environment. The annotators then construct test.py using the test cases provided by the original source.
  • Sources without explicitly defined problems. For sources such as GitHub repositories or Kaggle code modules, annotators inspect README files, shell scripts, and execution scripts (e.g., run.py) to understand the code's core functionality. Based on this understanding and the relevant high-level task category, they formulate a query grounded in the source code's actual functionality. They then construct the Docker environment, executable reference solution, and corresponding tests.

All constructed tasks subsequently undergo the quality-control procedures described above to ensure correctness and executability.

B. Distributional differences from the evaluation benchmarks

As stated in Section 4, we use only the high-level categories of Terminal-Bench and SWE-bench to guide the identification of relevant data sources, rather than reusing evaluation instances. We apply semantic similarity filtering to exclude instances similar to those in the evaluation benchmarks and conduct manual inspections to verify that the evolution tasks do not overlap with benchmark instances in their specific evaluation objectives.

To further characterize the differences, we analyzed the fine-grained capabilities assessed by the tasks in each dataset.

TB 2.0. The fine-grained capability categories accounting for more than 10% of its tasks are:

  • Algorithm Design, Rule and Constraint Implementation (22%)
  • File Parsing, Data Extraction, and Format Conversion (18%)
  • Dependency Installation, Environment Configuration, and Version Compatibility (12%)
  • Machine Learning Model Training, Inference, and Applications (10%)

In contrast, the categories accounting for more than 10% of our TB-related evolution dataset are:

  • Cryptography and Digital Forensics (19%)
  • Vulnerability Analysis (17%)
  • Performance, Resource, and Scalability Optimization (12%)
  • Numerical Computing, Statistical Modeling, and Mathematical Optimization (11%)
  • Source Code Compilation, Linking, and Building (10%)

These distributions reveal substantial differences in the fine-grained capabilities emphasized by the two datasets.

SWE-bench Verified. Our SWE-related evolution dataset uses repositories entirely different from those in SWE-bench Verified. The fine-grained task categories accounting for more than 10% of SWE-bench Verified are:

  • Logic and Conditional Errors (21%)
  • Input Validation and Boundary Handling (17%)
  • API Interfaces and Compatibility (12%)

In contrast, the categories accounting for more than 10% of our SWE-related evolution dataset are:

  • State Management and Caching (18%)
  • Attribute and Symbol Errors (13%)
  • File and Resource I/O (11%)

Together, these comparisons illustrate the distributional differences between our benchmark-disjoint evolution data and the downstream evaluation benchmarks.

AuthorsOpenReview ↗

Response (2/N)

Q2. Why were the original Terminal-Bench experiments limited to version 2.0, rather than including newer benchmarks such as TB-Science and TB-4.0?

We thank the reviewer for this valuable suggestion. Our original experiments focused on Terminal-Bench 2.0 and SWE-bench Verified, two representative benchmarks for long-horizon terminal and coding agents. TB-Science and TB-4.0 were released shortly before the ICLR submission deadline, leaving limited time for comprehensive evaluation and analysis. Nevertheless, we agree that these newer benchmarks can provide additional evidence of generalization.

To address this concern, we conducted additional experiments on TB-Science and TB-4.0. Using DeepSeek-V4-Flash-0731 as the backbone model, we evaluated the harness evolved by ModularRSI with 120 TB-related evolution tasks. The results are shown below.

MethodTB-Science (Acc, %)TB-4.0 (Acc, %)
No-Evolution Baseline1.427.93
ModularRSI4.2911.64
Improvement (percentage points)+2.87+3.71

ModularRSI improves accuracy by 2.87 percentage points on TB-Science and 3.71 percentage points on TB-4.0. Although absolute accuracy on TB-Science remains low, the improvements on both benchmarks provide additional evidence that the evolved harness can transfer to previously unseen, challenging terminal tasks.

Q3. Can the authors provide case-by-case analyses of improvements on evaluation tasks, beyond the evolution-task examples in Appendix K?

To address this concern, we provide examples of Terminal-Bench tasks for which the agent's performance improves after harness evolution. The table summarizes the relevant harness mechanisms and the outcomes across three evaluation runs per task.

Task NameModule TypeHarness ImprovementBaselineAfter RSI
openssl-selfsigned-certAgent LoopIntroduces a general iterative verification loop that derives completion criteria from task requirements, checks progress against those criteria, and uses unmet criteria to guide subsequent actions and revisions.2/33/3
adaptive-rejection-samplerTool UseExplicitly requires self-testing of edge cases, including invalid inputs and inputs containing NaN, to address coverage gaps caused by testing only valid inputs.2/33/3
build-cython-extAgent LoopAdds post-build functional verification under the task-specified dependency constraints, prompting the agent to test compiled extensions using representative examples and repository tests.2/33/3
build-pmarsAgent LoopPrompts the agent to verify the acceptance criteria explicitly specified in the task requirements before treating the task as complete.0/32/3

These cases illustrate how ModularRSI identifies and addresses recurring deficiencies in the agent harness, improving performance on downstream tasks.

AuthorsOpenReview ↗

Response (3/N)

Q4. What are the benefits of evolving modules independently, and what is the contribution of the final integration stage?

We thank the reviewer for this question. Independent module evolution and final integration serve complementary purposes: independent evolution localizes mechanism-level improvements, while final integration coordinates interactions among the evolved modules. The motivations and evidence for this design are discussed in Sections 3.1, 3.6, and 6.4, as well as Appendix J.4. We clarify the roles of these stages below.

Benefits of independent evolution. In practice, we observed that LLM-based diagnoses of harness deficiencies can contain hallucinations. Moreover, analyzing and modifying an entire harness places substantial demands on the model's code understanding, reasoning, and context-handling capabilities. When several modules are modified simultaneously based on potentially inaccurate diagnoses, it becomes harder to attribute performance regressions to particular changes or identify conflicts across modules.

We therefore evolve each module independently from the same initial harness, without sharing intermediate updates, and restrict modifications to the corresponding module's functional scope. This limits the complexity of diagnosis and modification, improves mechanism-level credit assignment, and reduces interference with unrelated mechanisms.

Table 4 compares independent evolution followed by integration against joint module evolution and non-modular whole-harness evolution. With the same number of rollouts per task and an aligned total evolution budget (5 × 3 + 1 = 16 epochs), ModularRSI achieves 52.43% accuracy, compared with 44.19% for joint module evolution and 46.44% for non-modular evolution. These results support the benefits of independently optimizing modules before combining them.

Contribution of the final integration stage. As described in Section 3.6, independently evolved modules may introduce redundant mechanisms or behavioral conflicts when combined. We therefore perform an additional cross-module integration epoch to remove redundancies, resolve interaction inconsistencies, and refine coordination, without introducing new capabilities or benchmark-specific adaptations (Appendix J.4).

To isolate the contribution of this stage, we conducted the following additional ablation on Terminal-Bench 2.0:

MethodTB2.0 (Acc, %)
No-Evolution Baseline47.57
ModularRSI w/o Final Integration49.44
ModularRSI52.43

Without final integration, accuracy decreases from 52.43% to 49.44%, a drop of 2.99 percentage points. This result demonstrates that the final integration stage provides gains beyond simply combining independently evolved modules.

Q5. Have the authors evaluated benchmarks with a larger domain gap?

As stated in our Abstract and Introduction, ModularRSI is designed primarily for CLI agent harnesses that support long-horizon coding and terminal tasks. Our principal evaluation therefore investigates generalization across different types of long-horizon CLI tasks.

We selected Terminal-Bench 2.0 and SWE-bench Verified as our main benchmarks because they evaluate complex terminal problem solving and real-world software engineering, respectively. Although both involve CLI interactions, their task types and execution workflows differ substantially. Improvements on both benchmarks support generalization across distinct CLI task domains. Evaluation beyond CLI-based agents is outside the primary scope of the paper.

Nevertheless, to examine transfer across a larger domain gap, we conducted additional experiments on five Harbor-supported benchmarks spanning more diverse task domains. Using DeepSeek-V4-Flash-0731, we evaluated the evolved harness with the following results:

MethodHotpotQASearchQAAppWorldBFCL Multi-TurnOfficeQA Pro
No-Evolution Baseline59.0076.4365.4870.0027.91
ModularRSI63.3378.6469.0575.0041.86

ModularRSI improves performance across all five benchmarks. Despite the differences in their task domains, these results provide further evidence that the learned harness improvements can transfer beyond the original evolution domains.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Response (1/N)

We thank the reviewer for their constructive comments and questions. We provide our responses below and will incorporate these clarifications and additional results into the revised manuscript.

Q1: Evaluation scale and single-primary-model evolution leave some generalization claims under-tested

We thank the reviewer for this valuable suggestion.

Due to computational cost constraints, conducting additional large-scale evolution experiments with other models is challenging. The total cost of our experiments using DS-V4-Flash-Preview, including debugging, has already reached approximately $50,000.

Nevertheless, as the reviewer noted, we have also conducted evolution experiments using other models, including Qwen3.8-Max and DS-V4-Flash-0731, on our evolution dataset. Due to space limitations, we presented these results in Appendix D, as summarized in Table 1 below.

Table1 : Generalization across different evolution models to run ModularRSI on TB 2.0.

Evol ModelMethodAcc ↑Pass@3 ↑Pass³ ↑
DS-V4-Flash-PreviewNo-Evol Baseline47.5758.4330.34
ModularRSI52.4365.1735.96
DS-V4-Flash-0731No-Evol Baseline61.7973.0350.56
ModularRSI67.4278.6556.18
Qwen3.8-MaxNo-Evol Baseline61.0573.0349.44
ModularRSI64.7975.2855.06

The experimental results in Table 1 demonstrate that our method can effectively perform harness evolution across different models.

Regarding the relatively limited performance improvement on TB2.0 achieved by the harness evolved on SWE-related data, this is One possible explanation is that the SWE-related tasks are largely concentrated on repository-level bug fixing and software engineering problems. With the same amount of evolution data, the diversity of harness improvements that can be learned from these trajectories is therefore limited.

To further investigate this issue, we conducted additional ModularRSI evolution experiments using a larger amount of SWE-related evolution data and evaluated the resulting harnesses on TB2.0. The results are presented in the following table:

Table2

Size of Evolution Data060120180240
Performance on TB2.047.5748.3149.4451.6853.93

As shown in Table 2, increasing the amount of SWE-related evolution data leads to a substantial improvement in ModularRSI's performance on TB2.0, providing further evidence of its cross-task generalization capability.

Q2: how is a deficiency assigned to a specific module when a trajectory's failure could plausibly involve several?

To ensure that the modifications are well-justified, we use an LLM to identify and discard trajectories whose failures may involve multiple modules. We manually inspected all failed trajectories from one evolution epoch and found that only 2.3% involved such ambiguous cases, all of which were successfully identified and discarded by the LLM.

Q3: how often the attributed module agrees with an independent/human judgment, or how often a modification targeted at the attributed module actually resolves the deficiency?

To assess whether the module modifications effectively address the identified deficiencies, we conducted a manual evaluation. The results show that 88.7% of the modifications successfully resolve the corresponding deficiencies or improve efficiency. Another 5.5% of the modifications are non-essential and do not meaningfully contribute to resolving the deficiencies, but they do not negatively affect the harness. Only approximately 5.8% of the modifications were judged to be inappropriate or insufficiently justified.

Overall, these results demonstrate that, although our method relies on LLM-based judgments, it effectively reduces the risk of introducing harmful modifications to the harness to a very low level.

Q4: what fraction of proposed modifications are rolled back at each gate (program check / diff review / execution validation), and is there evidence that the retained modifications are genuinely generic (e.g., they also help on held-out evolution tasks not used in that batch)?

We collected statistics over the three epochs of TB-related evolution, during which a total of 38 modification attempts were made. Among these, 1 modification (2.63%) was rejected by Program Check, 6 modifications (15.79%) were rejected by Diff Review, and 3 modifications (7.89%) were rejected by Execution Validation.

We further conducted a manual evaluation of the retained modifications and found that 92.86% were reasonable and well-justified. The remaining 7.14% did not improve the harness's performance, but they also did not negatively affect its normal functionality.

AuthorsOpenReview ↗

Response (2/N)

Q5: the relative rankings and absolute gains remain consistent across models of differing capability, or does evolution with weaker models become noisier/less beneficial?

As shown in the appendix D, ModularRSI consistently improves performance across different models, with gains ranging from 3.74 to 5.63 percentage points. The magnitude of these improvements also depends on the model's ability to analyze and modify the harness. For example, although Qwen3.8-Max achieves benchmark performance comparable to that of DS-V4-Flash-0731, it is less capable of identifying harness deficiencies and implementing effective modifications, resulting in a relatively smaller performance improvement.

AuthorsOpenReview ↗

Follow-up on Rebuttal Responses and Remaining Concerns

Dear Reviewer YQF5,

Thank you very much for your positive assessment of our work and for your thoughtful and constructive feedback.

As the rebuttal deadline is approaching, we would greatly appreciate any further feedback you may have. If any concerns remain unresolved, please feel free to let us know at your earliest convenience so that we can provide further clarification.

Thank you again for your time and valuable feedback.

Dear reviewer, please respond to the author's comment to further facilitate the discussion as the author-reviewer discussion period ends at November 18 at 11:59pm AoE

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.