acceptodds
Under review as a conference paper at ICLR 2027

From Literature to Reasoning: Generalizable Reward Models for AI Optimization

Abstract

AI4AI systems rely on reward models (RMs) to efficiently evaluate candidate system modifications. However, scenario-specific RM training often improves performance on seen tasks while degrading out-of-distribution (OOD) generalization. To address this issue, we propose LitRM, a literature-grounded reward modeling framework that transforms scientific knowledge into context-specific reasoning supervision. LitRM incorporates structured optimization knowledge through continual pretraining and retrieves mechanism-level evidence to guide a teacher model in reasoning over the current system state (BASE) and candidate modification (DIFF). The resulting reasoning guides evidence extraction, while joint training with teacher- and self-generated reasoning enables the student model to evaluate candidates without external retrieval at inference time. Experiments on multiple benchmarks show that LitRM consistently improves OOD generalization over current methods, yielding a percentage-point gain in average OOD AUC while maintaining strong performance on seen tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Reward Modeling, Out-of-Distribution Generalization, Literature Grounding, Chain-of-Thought Reasoning, AI4AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_R8RVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the limited generalization of reward models used to evaluate proposed AI system modifications: training on familiar optimization scenarios can improve in-distribution performance while degrading performance on unseen scenarios. To address this, LitRM incorporates scientific literature through continual pretraining and teacher reasoning supervision, then uses student-generated reasoning to guide evidence reading and reward prediction. Experiments across three benchmarks report improved OOD AUC while maintaining strong in-distribution performance, without literature retrieval at inference time.

Critical strengths
  • The paper investigates an important generalization problem in AI optimization reward modeling. Using scientific literature to enrich reasoning supervision is a promising direction for addressing this problem
  • The reported results show substantial OOD AUC gains over the evaluated baselines across three benchmarks while maintaining strong Seen performance. Additional backbone and distribution-shift experiments broaden the empirical support for these gains.
Critical weaknesses
  • My main concern is the lack of evaluation in an actual AI optimization workflow. The reported AUC and within-BASE ranking results measure reward-model performance, but do not establish whether using LitRM to guide experiments improves the final system or reduces experimentation costs. GenRM (Zhang et al., 2025, Sec. 4) and CLoud (Ankner et al., 2024, Sec. 3.1 and Fig. 4) evaluate downstream performance through Best-of-N selection. Could the authors integrate LitRM into a propose–select–execute workflow and compare final system performance and total computational cost against baseline-guided and unguided search under matched budgets?
  • The rationale for the staged training pipeline could be clearer. Why use separate literature pretraining, teacher warm-up, student rollout, and joint training stages? Although the progressive ablation shows gains from adding components, it does not explain why this particular training arrangement is preferable to a simpler alternative.
  • The case studies in Appendix C are limited and do not systematically explain the model's selection behavior. Which modifications does LitRM evaluate better than the baselines, and where does it fail? Does its preference change appropriately with the BASE configuration? Analyzing these patterns and testing whether removing or replacing reasoning changes rankings would better support the claim of learning transferable optimization mechanisms.
  • The figures need clearer labeling. For example, Figure 1 lacks a legend identifying which methods, model sizes, and benchmarks the points represent. Although Appendix B.1 provides additional details, these mappings should be clear from the figure and caption themselves.
Questions for authors

Seek weakeness section.

2 comments
AuthorsOpenReview ↗

Response [1/2]

Thank you for your careful review and constructive suggestions. We have conducted additional experiments and analyses to address your concerns and further clarify our methodology and findings. Below, we provide point-by-point responses to your comments.


W1. Downstream optimization utility and efficiency.

We have added two execution-grounded evaluations using RecOptBench-OOD for candidate selection and a separate set of 1,020 real-world online execution records for score-based filtering to assess LitRM's practical utility in AI optimization, with results included in the revised manuscript (Sec. 5.1 and Appendix B.11).

  1. Candidate Selection. To evaluate downstream optimization utility, we compare LitRM with baseline reward models and random selection using the same candidate pools for each BASE, and assess the selected modifications based on their actual execution outcomes. As shown in Table I, LitRM improves Success@1 from 13.85% to 24.18%, increases Realized Gain from −2.31 to +3.52 pp, and reduces executions to first success by approximately 30% compared with random selection. These results demonstrate LitRM's effectiveness in prioritizing promising modifications and improving optimization efficiency.

Table I. Downstream candidate selection performance on RecOptBench-OOD. Success@1: success rate (≥10% relative gain); Realized Gain: actual performance change; Spearman: within-BASE rank correlation; Executions: attempts to first success.

MethodSuccess@1 ↑Realized Gain (pp) ↑Spearman ↑Executions ↓
Random Selection13.85%−2.31−0.0242.88
GenRM-Direct14.02%−1.980.0152.83
CLoud13.91%−2.77−0.0192.87
Ours24.18%+3.520.3742.02
  1. Score-based Filtering. We further examine whether LitRM's reward scores can guide execution decisions by filtering out low-scoring candidates before costly validation. As shown in Table II, increasing the score threshold from 0.10 to 0.50 raises the success rate among retained candidates from 19.35% to 52.17%, while reducing the number of candidates requiring execution. This demonstrates a practical trade-off between execution cost and successful-candidate coverage

Table II. Candidate filtering based on LitRM scores and actual execution outcomes. Success Rate: proportion of retained candidates achieving ≥10% relative gain.

Score ThresholdRetained CandidatesRetention RateSuccessful CandidatesSuccess Rate ↑
All1,020100.00%17817.45%
≥0.1088987.16%17219.35%
≥0.2062461.18%15524.84%
≥0.3028828.24%10737.15%
≥0.40929.02%4447.83%
≥0.50232.25%1252.17%

W2. Rationale for the staged training pipeline.

LitRM's training stages serve complementary purposes: Literature CPT incorporates optimization knowledge, Teacher Warm-up initializes structured reasoning, Student Rollout captures student-generated reasoning, and Joint Training optimizes reasoning generation and reward prediction. Beyond the progressive ablation in the original manuscript, we further compare LitRM with simplified training pipelines, as shown in Table III.

Direct joint training without warm-up and rollout substantially degrades performance, while replacing student-generated rationales with teacher rationales reduces OOD AUC from 66.86% to 55.54%. This is consistent with our design rationale: Teacher Warm-up first establishes structured reasoning capabilities, while Student Rollout exposes reward learning to the model's own, potentially imperfect reasoning, mitigating the training–inference mismatch associated with relying solely on teacher rationales. Joint Training then combines teacher-supervised reasoning generation with reward learning from student-generated traces.

Table III. Comparison with simplified training pipelines on TraceML (AUC, %) and training cost (GPU·h).

Training StrategySeen AUC ↑OOD AUC ↑Training Cost (GPU·h) ↓
w/o Literature CPT82.1363.5217.23
w/o Teacher Warm-up & Student Rollout49.5951.3116.29
w/o Student Rollout (Teacher-only Joint Training)51.3155.5419.31
Full LitRM82.7966.8625.49

AuthorsOpenReview ↗

Response [2/2]

W3. Selection behavior and reasoning analysis.

We have added modification-category analysis, reasoning intervention experiments, and a BASE-dependent case study to better understand LitRM's prediction and ranking behavior. The results are included in the revised manuscript (Sec. 6 and Tables 23-25).

  1. Modification Category Analysis. We evaluate LitRM across different types of system modifications, as shown in Table IV. LitRM consistently outperforms GenRM-Direct across all categories, with the largest improvement in model architecture modifications (+18.63 pp) and a smaller gain in optimization strategies (+9.89 pp).

Table IV. OOD AUC (%) across modification categories on RecOptBench.

Modification CategorySamples / PositivesGenRM-DirectOurs
Model Architecture2,635 / 36850.8269.45
Training Engineering2,097 / 29552.6566.65
Loss Design1,255 / 32548.2864.58
Optimization Strategy935 / 6351.4161.30
Data Processing645 / 15350.5563.85
  1. Reasoning Intervention. To examine whether reasoning influences LitRM's predictions and candidate preferences, we compare original student reasoning with empty and length-matched shuffled reasoning. As shown in Table V, removing or shuffling reasoning reduces OOD AUC by 9.77–16.04 pp and reverses 25.31%–29.89% of candidate-pair rankings. These results indicate that LitRM's predictions and rankings depend on the reasoning content rather than merely its presence.

Table V. Effect of reasoning interventions on RecOptBench-OOD. Ranking Flip Rate measures the proportion of candidate pairs whose relative ordering changes compared with the original reasoning.

Reasoning ConditionOOD AUC (%) ↑Δ AUC (pp)Ranking Flip Rate
Original Student CoT70.56——
Empty CoT60.79−9.7725.31%
Length-matched Shuffled CoT54.52−16.0429.89%
GenRM-Direct (Baseline)51.40——
  1. BASE-dependent Case Study. We further include a case study in Appendix C examining similar training-duration modifications under different BASE configurations. For a Keras 1D CNN, increasing training from 3 to 5 epochs yields a 10.08% improvement, with LitRM assigning a score of 0.717. In contrast, increasing training from 5 to 10 epochs for an EfficientNet-based classifier results in a 2.11% degradation, with a lower score of 0.382. The corresponding reasoning reflects different considerations about underfitting and potential training limitations, illustrating LitRM's context-sensitive evaluation of candidate modifications.

W4. Figure clarity and labelling.

Thank you for this suggestion. We have revised Figure 1 by adding clearer labels and a legend to improve readability.

Reviewer_edRhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LitRM predicts whether a proposed system modification will improve execution performance. It combines literature pretraining, literature-assisted teacher rationales, reward learning on student rationales, and reasoning-guided evidence reading. The trained student needs no literature retrieval at inference and improves OOD prediction across three benchmarks.

Critical strengths
  1. The OOD gains are substantial across public and industrial benchmarks, rather than coming from one favorable split. Repository, temporal, and compositional evaluations make the generalization question concrete.
  2. Appendix B.7 checks ranking among candidates sharing the same BASE. This is more relevant to optimization than pooled AUC alone, and the eligible-group coverage is reported clearly.
Critical weaknesses
  1. The literature-free variant controls the architecture, but removing literature also removes substantial pretraining data. Without equal-token generic pretraining, the ablations cannot establish how much comes from optimization-specific knowledge rather than additional training.
  2. All student rationales are generated from one frozen snapshot and never refreshed, while joint training changes the shared backbone. This only partly addresses the stated training/inference reasoning mismatch. No comparison tests whether updating the rationale cache matters.
Questions for authors
  1. Does optimization literature outperform generic scientific text under the same pretraining-token and teacher-generation budgets?
  2. How does periodic regeneration of student rationales affect OOD AUC relative to the fixed cache?
1 comment

Thank you for your positive assessment and constructive feedback. We have conducted additional experiments to address your concerns and further strengthen the manuscript. Below, we provide point-by-point responses to your comments.


W1 & Q1. Domain knowledge vs. additional pretraining data.

Thank you for this suggestion. We have added an equal-token CPT ablation to distinguish the contribution of optimization-specific knowledge from additional pretraining data. All settings share the same literature-grounded teacher reasoning and generation budget, while both CPT variants use 116.1M effective training tokens, with other training configurations unchanged. The results have been included in the revised manuscript (Table 6 and Appendix A.6).

Compared with equal-token generic scientific pretraining, optimization literature improves OOD AUC by 2.96 pp on TraceML and 3.39 pp on RecOptBench, while generic pretraining provides only marginal gains over no CPT. These results indicate that LitRM's generalization benefits stem not merely from additional pretraining tokens, but also from optimization-specific knowledge.

Table I. Effect of CPT corpus on Seen and OOD AUC (%). Both CPT variants use the same token budget and teacher supervision.

CPT CorpusCPT TokensTraceML SeenTraceML OODRecOptBench SeenRecOptBench OOD
No CPT082.1363.5284.1566.90
Generic Scientific Text116.1M81.9863.9084.8167.17
Optimization Literature (Ours)116.1M82.7966.8686.1770.56

W2 & Q2. Fixed cache v.s. periodic rationale regeneration.

We have added experiments comparing the fixed student rationale cache with periodic regeneration at different refresh frequencies, evaluating both OOD performance and training cost. The results are also included in the revised manuscript (Sec. 5.4 and Table 20).

Periodic regeneration yields modest OOD AUC improvements of 0.23–0.35 pp, but increases training cost by 23.66%–72.26%. These results indicate that while refreshing rationales can provide small performance gains, the fixed-cache strategy offers a favorable trade-off between generalization performance and training efficiency.

Table II. Effect of rationale regeneration on OOD AUC and training cost on TraceML.

Rationale StrategyRefresh CountOOD AUC (%) ↑Training Cost (GPU·h) ↓Cost Increase
Fixed Cache (Ours)066.8625.49—
Mid-training Refresh167.0931.52+23.66%
Periodic Refresh367.2143.91+72.26%
Reviewer_sHRiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents LitRM, a literature-grounded reward model for predicting whether candidate machine-learning solutions or modifications achieve at least a threshold of 10% relative performance improvement. It combines literature-based continual pretraining, teacher–student reasoning supervision, and reasoning-guided evidence reading. Experiments on three benchmarks claim the improved out-of-distribution prediction performance.

Critical strengths
  1. The paper addresses a practically relevant problem: evaluating candidate ML modifications before costly execution.
  2. The experiments cover multiple benchmarks, evaluator backbones, and distribution shifts, with additional ablation and sensitivity analyses.
Critical weaknesses
  1. Missing direct comparisons with strong LLM evaluators.

    Since GPT is already used to generate teacher reasoning, a natural alternative baseline is to use a strong GPT model directly, with and without the same literature retrieval. The current comparisons mainly concern different training recipes for small models and do not establish LitRM's quality, cost, or latency advantages over this alternative.

    A direct comparison would clarify the practical motivation for training a separate evaluator.

  2. The demonstrated generalization is narrower than the potential application scope.

    The reported experiments evaluate held-out competitions, repositories, or modification categories, but do not show whether a fixed LitRM trained on one benchmark transfers directly to the other benchmarks. They also do not explicitly hold out families of the models being evaluated. Consequently, it remains unclear how broadly a single trained evaluator can be reused. Changing the evaluator's own Qwen or Gemma backbone does not establish transfer across the architectures of the systems it evaluates.

  3. The binary target is insufficiently justified and leaves a gap between prediction quality and optimization utility.

    The paper defines success as at least a 10% relative improvement, without sufficiently justifying this common threshold across heterogeneous tasks and metrics or testing alternative thresholds.

    Classification discards the improvement magnitude: a small improvement and a severe degradation can both be negative, while a marginal threshold crossing and a large improvement can both be positive.

    The objective also does not directly supervise the ordering of candidates by their actual gains.

    It would be great if an additional experiment would use a fixed LitRM on unseen-domain tasks to rank multiple candidates for each BASE, then evaluate the selected candidates using their actual execution outcomes. Useful measures include realized improvement, the gap to the best available candidate, and final performance under a fixed computation budget.

    If the method can reliably identify high-gain candidates while balancing performance improvement against training cost, it would be highly promising. Regression or ranking baselines would also help assess the consequences of the binary formulation.

  4. Applicability to large-scale LLM optimization remains unestablished.

    The empirical evidence primarily concerns competition-style ML development and industrial recommendation-system optimization. These are valuable settings, but it is unclear whether the approach transfers to large-scale LLM pretraining or post-training decisions, such as training strategies, data mixtures, or compute allocation. Reliable candidate evaluation in these settings could substantially increase the method's impact and potentially provide a useful component for recursive self-improvement (RSI).

Questions for authors

As Above

2 comments
AuthorsOpenReview ↗

Response [1/2]

Thank you for your thoughtful and constructive feedback. We have carefully addressed your concerns through additional experiments, analyses, and clarifications in the revised manuscript. Below, we provide point-by-point responses to your comments.


W1. Comparisons with strong LLM evaluators.

We have added comparisons with five strong LLM evaluators, both with and without literature retrieval, as summarized below. Further details are provided in the revised manuscript (Sec. 5.1 and Table 18).

These results show that LitRM achieves substantially higher Seen AUC (82.79%) and competitive OOD AUC (66.86%), closely approaching the best literature-augmented LLM (67.17%). Notably, despite its much smaller size (1.7B parameters), LitRM achieves comparable OOD performance without requiring external LLM calls or literature retrieval at inference time, offering a more efficient and cost-effective solution for repeated candidate evaluation in AI4AI workflows.

Table I. Comparison with strong LLM evaluators on TraceML. External LLM times are API latencies, while LitRM reports GPU compute time.

ModelSeen AUC ↑OOD AUC ↑Tokens / SampleTime / Sample
DeepSeek v4 (284B)66.6865.121.01k10.28 s
+ Literature68.5366.7011.02k18.79 s
Qwen3.8-Max (2.4T)64.5664.471.02k18.09 s
+ Literature67.0964.7711.04k40.72 s
GLM-5.3 (753B)66.0762.791.02k13.20 s
+ Literature67.1763.0910.97k23.87 s
Kimi-K2.8 (-)62.5163.991.02k9.76 s
+ Literature65.0465.2110.95k21.19 s
GPT-6.1 Sol (-)69.5666.221.01k10.04 s
+ Literature69.9667.1710.94k17.93 s
ours (1.7B)82.7966.861.19k1.16 s

W2. Generalization across benchmarks and architectures.

We have further evaluated LitRM's transferability through cross-benchmark zero-shot transfer and cross-architecture generalization, with additional results included in the revised manuscript (Sec. 5.2 and Appendix A.2).

  1. Cross-Benchmark Transfer. We evaluate fixed reward models trained on TraceML or RecOptBench directly on the other benchmark, without target-domain fine-tuning as shown in the table. LitRM consistently outperforms GenRM-Direct in both transfer directions, improving OOD AUC by 16.48 pp (TraceML → RecOptBench) and 8.62 pp (RecOptBench → TraceML). These results demonstrate the transferability of a fixed LitRM across distinct AI optimization benchmarks.

Table II. Cross-benchmark zero-shot transfer (AUC, %). Models are evaluated on both benchmarks without target-domain fine-tuning.

Training BenchmarkMethodTraceML (Seen / OOD)RecOptBench (Seen / OOD)
TraceMLGenRM-Direct81.05 / 58.1052.07 / 51.39
TraceMLours82.79 / 66.8668.74 / 67.87
RecOptBenchGenRM-Direct54.12 / 52.7982.43 / 51.40
RecOptBenchours63.90 / 61.4186.17 / 70.56
  1. Cross-Architecture Generalization. We further construct architecture-held-out splits on RecOptBench, excluding the evaluated model architectures (MMoE and Transformer Decoder) from training. LitRM achieves the best performance on both unseen architectures, outperforming the strongest baselines by 6.92 pp and 6.12 pp, respectively. Together with the existing cross-repository, unseen-category, temporal, and compositional evaluations (Table 3), these results provide further evidence of LitRM's generalization across optimization scenarios and evaluated-model architectures.

Table III. Generalization to unseen evaluated-model architectures on RecOptBench (AUC, %). MMoE and Transformer Decoder are held out from training (7,122 training samples).

Evaluation SplitSamplesGenRM-DirectCLoudOurs
Seen76584.6683.4289.97
Unseen MMoE15954.3053.1661.22
Unseen Transformer Decoder67252.7149.6458.83

AuthorsOpenReview ↗

Response [2/2]

W3. Binary target and optimization utility.

We have added experiments on improvement thresholds and downstream candidate selection to further examine the relationship between binary reward prediction and actual optimization outcomes.

  1. Threshold sensitivity. We use a 10% relative improvement threshold as a consistent binary evaluation criterion across tasks. To examine its sensitivity, we further evaluate alternative thresholds of 5% and 15%. As shown in Table IV, LitRM consistently outperforms the baselines across all thresholds on both benchmarks.

Table IV. OOD AUC (%) under different relative improvement thresholds on TraceML and RecOptBench.

BenchmarkThresholdGenRM-DirectCLoudOurs
TraceML5%56.8952.7960.42
TraceML10%58.1054.5166.86
TraceML15%61.7061.8070.55
RecOptBench5%50.9647.8465.02
RecOptBench10%51.4049.7670.56
RecOptBench15%53.7152.2874.24
  1. Candidate Ranking and Optimization Utility. Although trained with binary supervision, LitRM produces continuous reward scores for candidate ranking, as evaluated in Appendix B.7 (Table 17). We further assess candidate selection using actual execution outcomes on RecOptBench-OOD, as shown in Table V. Compared with random selection, LitRM improves Success@1 from 13.85% to 24.18% and average Realized Gain from −2.31 to +3.52, while achieving stronger rank correlation (Spearman: 0.374). These results demonstrate the practical utility of LitRM's reward scores for candidate selection.

Table V. Execution-grounded candidate selection on RecOptBench-OOD. Success@1 measures the proportion of top-ranked candidates achieving at least 10% relative improvement; Realized Gain is the average absolute GAUC improvement (percentage points).

MethodSuccess@1 ↑Realized Gain (ΔGAUC, pp) ↑Spearman ↑
Random Selection13.85%−2.31−0.024
GenRM-Direct14.02%−1.980.015
CLoud13.91%−2.77−0.019
Ours24.18%+3.520.374

W4. Applicability to large-scale LLM optimization.

We appreciate this suggestion. Large-scale LLM optimization, including training strategies, data mixtures, and compute allocation, is a promising extension of LitRM. Although our BASE–DIFF formulation is not architecture-specific, validating its effectiveness in LLM pretraining and post-training requires further investigation beyond the current evaluation scope. We have expanded the Discussion section to clarify this limitation and highlight LLM optimization and broader AI self-improvement as future directions.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Response

Dear Area Chair, Senior Area Chair, and Reviewers,

We sincerely appreciate your time and effort in reviewing our submission. The valuable comments help us improve our submission. We have revised the manuscript and are now submitting the revised pdf along with this summary of the revisions. To facilitate differentiation, we have highlighted key modifications in the updated pdf in blue. The main revisions are summarized below.

  • Generalization and Strong LLM Comparisons: We have added direct comparisons with five strong LLM evaluators, both with and without literature retrieval, as well as cross-benchmark zero-shot transfer and cross-architecture generalization experiments (cf. Secs. 5.1–5.2).
  • Downstream Optimization Utility: We have extended our evaluation beyond prediction AUC to examine threshold sensitivity, candidate ranking, actual performance gains, and execution efficiency using recorded optimization outcomes (cf. Sec. 5 and Appendix B).
  • Training and Reasoning Analysis: We have added equal-token pretraining comparisons, periodic rationale regeneration experiments, simplified training pipeline ablations, modification-category analysis, and reasoning interventions to further validate and explain LitRM's design (cf. Secs. 5.3 and 6).
  • Discussion and Presentation: We have clarified the scope and limitations of LitRM, expanded the discussion of large-scale LLM optimization and future applications, and improved figure labeling and manuscript clarity.

We hope these revisions address the reviewers' concerns and further clarify the contributions and practical value of LitRM. Detailed point-by-point responses are provided below.

Sincerely,

Authors of Submission #20280

AuthorsOpenReview ↗

Thank You for Your Time and Consideration

Dear Area Chair and Reviewers,

Thank you for your time and constructive feedback.

We have submitted our detailed responses and revised manuscript with additional experiments and analyses. As the discussion deadline approaches, we would appreciate any further feedback or questions.

Thank you again for your time and help.

Best regards, Authors of Submission #20280

AuthorsOpenReview ↗

Additional Experiments and Updates During Discussion

Dear Area Chair, Senior Area Chair, and Reviewers,

Thank you for your time and constructive feedback. As the discussion deadline approaches, we would like to briefly highlight our main contributions and the additional experiments conducted in response to the reviewers' concerns.

Summary. We propose LitRM, a literature-grounded reward model for AI optimization. By combining optimization knowledge with teacher–student reasoning, LitRM improves OOD generalization by 15.40 percentage points on average across three benchmarks, while maintaining strong seen-task performance without external retrieval at inference time.

Main updates during discussion:

  1. Stronger baselines and generalization (sHRi W1–W2). We added comparisons with five strong LLM evaluators, cross-benchmark zero-shot transfer, and architecture-held-out experiments, further validating LitRM's generalization capabilities.

  2. Practical optimization utility (sHRi W3; R8RV W1). We extended the evaluation from retrospective candidate selection to an end-to-end AI4AI optimization workflow. Under matched 12h budgets, LitRM achieves +1.59% relative GAUC improvement, compared with +0.66% for GenRM-Direct, and exceeds its 6h performance within 3h.

  3. Training design and controlled comparisons (edRh W1–W2; R8RV W2). We added equal-token pretraining comparisons, rationale regeneration studies, and simplified training pipelines under matched training budgets. At approximately 25.5 GPU·h, LitRM achieves 66.86% OOD AUC, compared with 55.43% for teacher-only joint training and 51.58% for direct joint training.

  4. Reasoning analysis and clarity (R8RV W3–W4). We added modification-category analysis, reasoning interventions, and BASE-dependent case studies, and improved figure labeling and methodological descriptions.

We also appreciate Reviewer R8RV's follow-up questions. The two latest experiments on end-to-end optimization and matched-budget training were conducted specifically to address the remaining concerns, with detailed results provided in our latest response.

We hope these updates help clarify our contributions and address the reviewers' concerns. We would appreciate any further feedback and discussion before the deadline.

Thank you again for your time and consideration.

Best regards, Authors of Submission #20280

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.