acceptodds
Under review as a conference paper at ICLR 2027

Understanding Pre-Scaling Reasoning under False Premises in Code Generation

Abstract

Adaptive test-time scaling increasingly relies on instance-level signals to decide where additional reasoning should be spent. Existing allocation strategies commonly use task difficulty, agreement, or verifier feedback, yet misleading specifications introduce a different source of risk: an otherwise solvable instance can fail because a false premise redirects its solution strategy. We study this specification-conditioned vulnerability in code generation and show that it is both recurrent and selective. Across models, benchmarks, and premise families, false premises induce failures in only a minority of baseline-correct instances, leaving substantial heterogeneity within coarse model, dataset, and task groups. Behavioral analysis further shows that vulnerable and robust-correct instances begin with similar rates of Direct Premise Exploitation, then diverge sharply after false-premise injection, with induced failures disproportionately shifting toward direct reliance on the misleading premise. We introduce Pre-Scaling Reasoning (PSR), a design principle for adaptive reasoning allocation that uses pre-generation vulnerability estimates to decide where additional robustness reasoning should be allocated. Under a fixed 20% selection budget, PSR identifies 48.64% of induced failures, compared with 31.21% for leave-one-model-out task difficulty. When all routing policies use the same fixed prompting-based mitigation, PSR-guided allocation improves Pass@1 by +5.77 pp and exceeds LOMO-guided allocation by +2.92 pp. These results establish specification-conditioned vulnerability as a practical pre-generation signal for allocating limited robustness reasoning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
code generation, large language models, false premises, robustness, behavior analysis, evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aV1jOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how false statements in task descriptions cause code failures. A model may solve a task correctly but fail when a misleading statement is added. PSR uses text features to predict this risk before code generation. It samples tasks and uses an extra prompt to check false assumptions. The experiments compare this selection rule with task difficulty.

Critical strengths
  • The study separates task difficulty from the risk of following a false statement. This helps explain why easy tasks need extra checking.
  • The classifier uses information available before code generation. It does not need to see whether the program succeeds or fails.
  • The rules use the same prompt to address false assumptions. This makes it easier to judge the benefit of choosing different tasks.
  • The study tests false statements across several models and types of assumptions. This gives a broader value of the problem.
Critical weaknesses
  • All inputs in the routing study contain false sortedness claims. Real requests can contain valid guarantees too; the study does not test when distrusting those guarantees adds needless work or changes the intended solution.
  • The predictor learns lexical cues for sortedness, uniqueness, and sign constraints. Task-held-out testing does not show that these cues transfer to new false premises or unfamiliar wording.
  • The intervention changes a prompt, and the budget counts intervention calls. As Appendix S3 states, tokens and compute are not matched, so the result supports prompt routing more directly than efficient reasoning scaling.
  • The organization and formatting of the Related Work section could benefit from further refinement, as the current presentation makes it somewhat challenging for readers to follow the progression of research and recent developments in this field.
Questions for authors

Please respond to the concerns raised in Critical Weaknesses.

1 comment
AuthorsOpenReview ↗

Rebuttal for Reviewer aV1j

We thank Reviewer aV1j for the constructive feedback and for recognizing specification-conditioned vulnerability as distinct from task difficulty. We particularly appreciate the reviewer noting that pre-generation features enable efficient routing, and that a shared mitigation prompt isolates the value of instance selection from mitigation effectiveness. We address the reviewer’s concerns regarding empirical robustness, evaluation rigor, and presentation in detail below.

Comment 1: Valid premises and unnecessary intervention

All inputs in the routing study contain false sortedness claims. Real requests can contain valid guarantees too; the study does not test when distrusting those guarantees adds needless work or changes the intended solution.

Authors’ response:

We evaluated our intervention on tasks with correct premises, yielding two key observations:

  1. No overall performance degradation: In valid-premise scenarios, applying the intervention did not cause any overall drop in Pass@1. The intended correctness is safely preserved.
  2. Model-specific variations: While the aggregate performance remains stable, we observe slight differences in how individual models respond to the intervention prompt.

Table R3: Pass@1 under valid premises.

ModelStandard promptIntervention promptNet change
GPT-4o-mini84.2%86.6%+2.4 pp
DeepSeek-V3.292.6%94.1%+1.5 pp

Comment 2: Transfer across premise wording

The predictor learns lexical cues for sortedness, uniqueness, and sign constraints. Task-held-out testing does not show that these cues transfer to new false premises or unfamiliar wording.

Authors’ response:

To prove our predictor does not overfit to specific lexical cues, we rigorously tested its robustness to unfamiliar wording:

  1. Stable across paraphrases. Across the tested paraphrases, the mean change in Capture@20% relative to the original wording ranges from −0.27 to +1.52 pp across premise families.
  2. Consistent across premise families. The allocation signal remains stable for all three families; paraphrase-level changes are shown in Table R4.

Table R4: Change in Capture@20% under alternative premise wordings, relative to the original (pp).

Premise familyParaphrase 1Paraphrase 2Paraphrase 3Mean change
Sortedness−0.19+1.02−1.63−0.27
Uniqueness−1.67−0.86+5.15+0.87
Non-negativity+1.32+1.92+1.32+1.52

Comment 3: Intervention-call budget versus compute

The intervention changes a prompt, and the budget counts intervention calls. As Appendix S3 states, tokens and compute are not matched, so the result supports prompt routing more directly than efficient reasoning scaling.

Authors’ response:

We provide additional token-level evaluations to address this concern. Under the original fixed 20% intervention-call budget, PSR consumes the fewest tokens (140.5K) while achieving the highest Pass@1 gain (+5.77 pp). More importantly, under an identical aggregate token budget of 149,831 tokens, PSR achieves a +6.08 pp gain, substantially outperforming all alternative allocation policies.

Table R5. Token consumption and Pass@1 gains under fixed-call and matched-token budgets.

Allocation policyTokens (fixed-call)Gain (fixed-call)Gain (matched-token)
Random150.2K+3.47 pp+3.46 pp
Generic Difficulty147.9K+2.34 pp+2.42 pp
LOMO Difficulty154.4K+2.86 pp+2.75 pp
PSR Vulnerability140.5K+5.40 pp+6.08 pp

The fixed-call comparison uses a 20% intervention-call budget. The matched-token comparison uses an identical aggregate budget of 149,831 tokens across all policies.


Comment 4: Related Work organization

The organization and formatting of the Related Work section could benefit from further refinement, as the current presentation makes it somewhat challenging for readers to follow the progression of research and recent developments in this field.

Authors’ response:

We revised the Related Work section to improve readability and logical progression, reorganizing the discussion to connect misleading-premise risks, adaptive inference methods, and our pre-generation vulnerability-routing contribution.

The updated PDF has been revised to reflect these changes.

Reviewer_ejf2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a framework (Pre-Scaling Reasoning, PSR) to mitigate failures induced by false premises in code generation by allocating additional reasoning effort during inference. The paper addresses two questions: 1) when additional reasoning is needed and what distinguishes "vulnerable" and "robust" questions, 2) whether vulnerability can be predicted before generation.

The paper divides problems into four categories based on task difficulty and vulnerability to false premises, highlighting easy but vulnerable problems. Under difficulty-based reasoning allocation, these instances may receive less additional reasoning effort despite being vulnerable to false premises. In contrast, PSR aims to prioritize problems based on their estimated vulnerability rather than task difficulty itself.

Before generation, the PSR predicts the vulnerability before generation with 23 different signals (Table S3) and train the model with logistic regression model. Under 20% selection budget, PSR captures 48.64% of the induced failure. Furthermore, with prompting based mitigation, the PSR achieves a 5.77 percent improvement of pass@1 over no intervention, consistently outperforming baseline methods.

Critical strengths
  1. The paper tackles an important problem, with new perspective and a simple method. While existing methods allocate based on difficulty, this paper introduces false-premise vulnerability as an alternative. The proposed method is simple to implement and practical, and demonstrates improvements under a limited inference budget.
Critical weaknesses
  1. The experiments evaluation and method is limited. While the authors evaluate the method across different models and benchmarks, the experiment focus on limited set of false-premise types (sortedness, uniqueness, and non-negativity). The evaluation should cover more diverse or realistic scenarios as well.

  2. The technical novelty of the method is limited. The vulnerability predictor relies on 23 handcrafted signals and a logistic regression model. Also, the intervention method is based on existing strategy. The main contribution appears to be the vulnerability as a criteria specifically in code generation rather than the method itself.

Questions for authors
  1. In figure 6(c) why does random allocation perform better than LOMO difficulty-based allocation? (2.86% vs 3.47%) Could the authors provide further explanation on this result?

  2. How does this method compare against different inference mitigation techniques? (e.g. [1]) Although, these might have different objectives, it would be helpful to compare with other methods. [1] Davidov et al., Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning, ICML 2026

  3. Could the authors provide additional quantitative analysis of the relationship between difficulty and false-premise vulnerability? Additional analysis of what the vulnerability provides would help the readers.

  4. Could the authors provide additional ablation studies on the 23 features in Table S3? (e.g. ablation study for each group) This would help the user understand whether the predictor relies on specific features or not.

  5. Could the authors provide additional evaluation on different benchmarks specifically targeting code security?

  6. How well does the vulnerability predictor generalize to unseen modesl? Since the vulnerability may vary depending on the model, is the same predictor used for different models in the paper?

7 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer ejf2 - 1/3

We thank the reviewer for the constructive feedback and for recognizing the importance of false-premise vulnerability as an alternative signal for reasoning allocation. We also appreciate the positive assessment of our paper’s clarity and practical value. We address the concerns point by point below.

Comment 1:

The experiments evaluation and method is limited. While the authors evaluate the method across different models and benchmarks, the experiment focus on limited set of false-premise types (sortedness, uniqueness, and non-negativity). The evaluation should cover more diverse or realistic scenarios as well.

Authors’ response:

We extend our evaluation to real-world coding scenarios beyond the three originally studied false-premise categories.

  1. Evaluation on real-world scenarios. We evaluate PSR on DS-1000, a real-world data science code generation benchmark. The selected tasks involve naturally occurring user assumptions or failed attempts, providing a realistic evaluation beyond our original controlled scenarios.
  2. Consistent improvements across models. As shown in Table R6, PSR achieves the highest Pass@1 gains over Random on both models, with +2.07 pp on GPT-4o mini and +3.27 pp on GPT-OSS-120B, outperforming both Generic Difficulty and LOMO Difficulty.

Table R6. Pass@1 changes relative to Random on DS-1000.

Allocation StrategyGPT-4o miniGPT-OSS-120B
PSR Vulnerability+2.07 pp+3.27 pp
Generic Difficulty+0.58 pp−2.70 pp
LOMO Difficulty−0.31 pp+0.73 pp

Comment 2:

The technical novelty of the method is limited. The vulnerability predictor relies on 23 handcrafted signals and a logistic regression model. Also, the intervention method is based on existing strategy. The main contribution appears to be the vulnerability as a criteria specifically in code generation rather than the method itself.

Authors’ response:

We respectfully clarify that we do not claim novelty in the classifier or intervention. Our contribution is to establish specification-conditioned vulnerability as a pre-generation signal for allocating test-time compute.

  1. A controlled allocation design. We keep the intervention fixed and vary only the allocation policy, isolating the value of vulnerability-based selection from the intervention itself.

  2. A predictor without handcrafted features. We replaced the 23 handcrafted features with frozen BGE embeddings of raw specifications (Emb-LR). Under the same 20% budget, it achieves a +7.16 ± 0.22 pp Pass@1 gain, compared with +5.40 ± 0.28 pp for the original PSR and +2.86 pp for LOMO difficulty (Table R1).

Table R1: Pass@1 gains under a fixed 20% intervention-call budget.

Allocation policyInput representationΔPass@1
LOMO Difficulty†Task difficulty+2.86 pp
Original PSR23 handcrafted features+5.40 ± 0.28 pp
Emb-LR (BGE)Frozen specification embeddings+7.16 ± 0.22 pp

† LOMO allocation is independent of cross-validation splits and produces identical results across all five seeds. For PSR and Emb-LR, reported values are mean ± SD across five task-grouped split seeds.


Comment 3:

In figure 6(c) why does random allocation perform better than LOMO difficulty-based allocation? (2.86% vs 3.47%) Could the authors provide further explanation on this result?

Authors’ response:

The key distinction is that task difficulty does not necessarily indicate how much a task can benefit from additional reasoning.

  1. Difficulty does not imply recoverability. LOMO prioritizes tasks that other models fail to solve under the original specification. However, the hardest tasks are not necessarily those that can be recovered through our fixed intervention.
  2. Empirical evidence of limited recoverability. In our exploratory analysis, the target model's pass rate on the highest-difficulty instances increases only from 7.0% to 8.6% after intervention, indicating that these instances remain challenging even with additional reasoning.
  3. Implications for allocation. Under a fixed intervention budget, LOMO may allocate too many calls to difficult but poorly recoverable instances. Random allocation, by sampling across different difficulty levels, can therefore achieve greater overall improvement. In contrast, PSR prioritizes vulnerability to misleading specifications, which provides a more relevant signal for identifying instances that can benefit from intervention.
AuthorsOpenReview ↗

Rebuttal for Reviewer ejf2 - 2/3

Comment 4:

How does this method compare against different inference mitigation techniques? (e.g. [1]) Although, these might have different objectives, it would be helpful to compare with other methods. [1] Davidov et al., Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning, ICML 2026

Authors’ response:

We further compare PSR with a Davidov-style dynamic abstention strategy [1], representing an alternative inference-time mitigation approach.

  1. Comparison setup. We evaluate both strategies using the same expected reward criterion, assigning an abstention reward of 0.5. PSR selectively allocates robustness intervention, whereas the Davidov-style strategy dynamically monitors tasks and allows abstention.
  2. Performance comparison. As shown in Table R7, PSR achieves an expected reward of 82.39%, outperforming the Davidov-style strategy (70.07%) by 12.32 pp. PSR maintains an 82.39% correct delivery rate without abstention, whereas the Davidov-style strategy abstains on 51.41% of tasks.

Table R7. Comparison between PSR and dynamic abstention.

StrategyCorrect Delivery ↑Abstention ↓Expected Reward ↑
PSR Vulnerability82.39%0%82.39%
Davidov-style44.37%51.41%70.07%

Comment 5:

Could the authors provide additional quantitative analysis of the relationship between difficulty and false-premise vulnerability? Additional analysis of what the vulnerability provides would help the readers.

Authors’ response:

  1. Difficulty and vulnerability are positively associated at the group level. The induced-failure rate increases monotonically across the four LOMO difficulty levels, from 12.32% to 26.14%, 28.95%, and 50.00%, respectively.

  2. Vulnerability provides additional discrimination beyond difficulty. Even among instances assigned the lowest LOMO difficulty, PSR captures 54.17%–63.16% of induced failures across three held-out models at a 20% selection budget, compared with approximately 20% for LOMO with random tie-breaking. This demonstrates that PSR can distinguish vulnerable instances even among tasks with the same difficulty level.

These results suggest that difficulty provides a coarse indication of vulnerability, while PSR offers finer-grained instance-level discrimination beyond difficulty.


Comment 6:

Could the authors provide additional ablation studies on the 23 features in Table S3? (e.g. ablation study for each group) This would help the user understand whether the predictor relies on specific features or not.

Authors’ response:

We conduct group-wise ablations of the 23 handcrafted features, retraining the predictor after removing each feature group (Table R8).

  1. Operation cues contribute most. Removing them causes the largest drop in Capture@20 (−3.48 pp), consistent with our behavioral analysis in §4.3 and Figure 4, where false premises interact with specific operations, such as adjacent element comparison.
  2. Instance-level features improve prediction beyond coarse metadata. A metadata-only predictor reduces Capture@20 by 6.61 pp, demonstrating that coarse task-level priors are insufficient to identify vulnerable instances and that instance-level specification features provide additional predictive information.

These results connect the predictor's feature contributions to the underlying mechanism of false-premise vulnerability.

Table R8. Change in Capture@20% relative to the full predictor.

Feature configurationΔ Capture@20%
− Operation cues−3.48 pp
− Input-modality cues−1.46 pp
− Task/premise cues−1.20 pp
− Text statistics−0.79 pp
Metadata only−6.61 pp

AuthorsOpenReview ↗

Rebuttal for Reviewer ejf2 - 3/3

Comment 7:

Could the authors provide additional evaluation on different benchmarks specifically targeting code security?

Authors’ response:

We appreciate the suggestion to evaluate PSR on code-security benchmarks. We clarify the scope of our study as follows.

  1. Distinct evaluation targets. Our study focuses on specification-conditioned failures induced by false premises, examining how misleading assumptions redirect otherwise solvable tasks and how such vulnerability can guide selective reasoning allocation. Code-security benchmarks target security-specific vulnerabilities and require different evaluation criteria.
  2. Scope of the current work. Our additional evaluation on DS-1000 extends the study to more realistic data-science coding scenarios. However, systematic evaluation of security-specific vulnerabilities represents a complementary research direction, which we leave for future work.

Comment 8:

How well does the vulnerability predictor generalize to unseen modesl? Since the vulnerability may vary depending on the model, is the same predictor used for different models in the paper?

Authors’ response:

We evaluate a single shared predictor on three unseen generation models.

  1. Shared predictor. We use one logistic-regression predictor jointly trained on pooled samples from multiple generation models. The same predictor scores instances across different models, without model-specific predictors.
  2. Cross-model generalization. On three models excluded from training, PSR achieves the largest Capture@20 gain over Random on each model, with +28.07 pp on Qwen2.5-Coder-7B, +37.23 pp on DeepSeek-V3.2, and +28.83 pp on GPT-4.1-mini (Table R9). This demonstrates that a shared predictor can transfer across unseen generation models.

Table R9. Capture@20 gains over Random on three unseen models.

Allocation StrategyQwen2.5-Coder-7BDeepSeek-V3.2GPT-4.1-mini
Generic Difficulty+7.46 pp+2.86 pp+11.79 pp
LOMO Difficulty+16.34 pp+4.77 pp+11.28 pp
PSR (Shared Predictor)+28.07 pp+37.23 pp+28.83 pp
PSR margin over best baseline+11.73 pp+32.46 pp+17.04 pp
Reviewer_DQoiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper looks at how false statements in code-generation specs break tasks a model would otherwise solve. PSR predicts this risk before generation using logistic regression on text features and premise family, and the top 20% of instances go to a robustness prompt. The authors report catching 48.64% of failures and a 5.77 pp gain in Pass@1.

Critical strengths
  1. The paper identifies a risk that difficulty-based allocation misses. An easy task can still break because of a false premise.
Critical weaknesses
  1. There is limited novelty on the modeling side. While the risk and its analysis are new, PSR is a logistic regression on length and keyword features with a mitigation that follows He et al. (2026).
  2. The experiments are not convincing. The paper does not explain how the 20 keyword flags were chosen or publish the list. Task-grouped folds only protect the regression weights, so if the list was shaped after seeing failures, 48.64% and +5.77 pp could be optimistic and the evaluation wouldn’t catch it. Sample sizes are also too small. The 3,846 instances come from 421 tasks, and only 148 tasks show any induced failure (Fig. 3b), which is not much for 23 features. No repeated runs are reported, the recovery claim relies on 7 cases from one model on one benchmark, and generalization is untested because no benchmark is held out.
Questions for authors

See weekness

4 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer DQoi

We thank Reviewer DQoi for the thoughtful and constructive review and for recognizing the importance of identifying specification-conditioned vulnerability as a distinct source of failure that difficulty-based allocation may overlook. We particularly appreciate the reviewer’s recognition that even otherwise easy and solvable tasks can fail when misleading premises redirect the model’s solution strategy. We are also grateful for the positive assessment of the paper’s clarity and informativeness. We address the reviewer’s concerns regarding modeling novelty and experimental rigor in detail below.

Comment 1:

There is limited novelty on the modeling side. While the risk and its analysis are new, PSR is a logistic regression on length and keyword features with a mitigation that follows He et al. (2026).

Authors’ response:

We respectfully clarify that we do not claim methodological novelty in model architecture or mitigation. Our core contribution is establishing specification-conditioned vulnerability as an actionable pre-generation signal for allocating test-time compute.

  1. A controlled allocation design. Following established adaptive compute paradigms [1, 2], we freeze the mitigation method [3] and vary only the allocation signal. This design isolates the value of instance selection from mitigation effectiveness, a methodological strength also recognized by Reviewer aV1j.
  2. A predictor without handcrafted features. We replaced the 23 handcrafted features with frozen BGE embeddings of raw specifications (Emb-LR). Under the same 20% budget, Emb-LR achieves a higher Pass@1 gain (+7.16 ± 0.22 pp) than the original PSR (+5.40 ± 0.28 pp) and LOMO difficulty, despite using no handcrafted features.

Table R1: Pass@1 gains under a fixed 20% intervention-call budget.

Allocation policyInput representationΔPass@1
LOMO Difficulty†Task difficulty+2.86 pp
Original PSR23 handcrafted features+5.40 ± 0.28 pp
Emb-LR (BGE)Frozen specification embeddings+7.16 ± 0.22 pp

† LOMO allocation is independent of cross-validation splits and produces identical results across all five seeds. For PSR and Emb-LR, reported values are mean ± SD across five task-grouped split seeds.

Comment 2:

The experiments are not convincing. The paper does not explain how the 20 keyword flags were chosen... Task-grouped folds only protect the regression weights... Sample sizes are also too small... No repeated runs are reported, the recovery claim relies on 7 cases... generalization is untested.

Authors’ response:

We address these concerns through additional analyses of the existing generation outputs:

  1. Feature-free prediction. Motivated by the reviewer’s concern, we evaluated a predictor using frozen BGE embeddings of raw specifications without handcrafted features. As shown in Table R1, it achieves a higher Pass@1 gain (+7.16 ± 0.22 pp) than the original PSR with handcrafted features (+5.40 ± 0.28 pp).
  2. Split sensitivity and statistical robustness. We repeated predictor training and evaluation across five task-grouped split seeds, reporting mean ± SD. We additionally performed 10,000 task-cluster bootstrap resamples over 421 tasks. Emb-LR outperforms LOMO by +4.30 pp (95% CI: [+2.53, +6.06]), with the confidence interval excluding zero. A label-permutation control yields AUROC ≈ 0.5, further supporting the predictive signal.
  3. Broader quantitative recovery. The seven cases in the original paper were used for qualitative behavioral analysis. Across five split seeds, Emb-LR yields 95.37 ± 3.06 expected recoveries and only 3.6 expected regressions under the 20% intervention budget.
  4. Cross-benchmark generalization. We conducted a Leave-One-Benchmark-Out (LOBO) evaluation (train on rest, test directly on held-out). As shown in Table R2, Original PSR outperforms LOMO difficulty on both held-out benchmarks, providing evidence of cross-benchmark generalization.

Table R2. Cross-benchmark generalization under Leave-One-Benchmark-Out (LOBO) evaluation.

Held-out benchmarkLOMO DifficultyOriginal PSR
HumanEval23.11%34.08% (+10.97 pp)
MBPP34.43%44.09% (+9.66 pp)

References

[1] Snell et al., Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Parameters for Reasoning, ICLR 2025.

[2] Damani et al., Learning How Hard to Think: Input-Adaptive Allocation of LM Computation, ICLR 2025.

[3] He et al., Verification Trap: Understanding Test-Time Selection Failures under False Premises in Code Generation, EMNLP 2026.

Dear Reviewer DQoi,

Thank you again for your feedback. We added feature-free prediction, five-seed evaluations, statistical robustness analyses, and cross-benchmark validation to address your concerns.

As the rebuttal period is nearing its end, we would appreciate knowing whether you have any remaining concerns. We would be happy to address them.

Sincerely,

The Authors

Thank you for the detailed response and the additional experiments, which address part of my concerns. However, like Reviewer aV1j, I am still not fully convinced by the experimental evidence and the generality of the setting, so I will keep my rating.

Dear Reviewer DQoi,

Thank you for acknowledging our additional experiments. We have substantially strengthened the experimental evidence through feature-free prediction, five-seed evaluations, task-cluster bootstrap analysis, and leave-one-benchmark-out validation.

We recognize that some concerns about broader generalizability may remain. However, we believe these new results directly address several major concerns regarding the soundness and reliability of our approach.

Given this additional evidence, would you be willing to reconsider your rating? If you find the results convincing, we would greatly appreciate your willingness to champion our paper in the final discussion.

Thank you again for your thoughtful evaluation and engagement.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.