- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes CrossView-PRM, a target-verifier-free reward-construction framework for long-horizon language agents. It compares executed actions with sibling actions from the same decision state under matched budgets. Semantic, model, and temporal views produce soft preferences over these action pairs, which are aggregated using correlation-aware weighting and screened by a source-calibrated gate. Accepted labels are used to train a prefix potential, which is then frozen. The primary reward is obtained by comparing the executed action with its siblings using the trained prefix potential, supplemented by potential-difference shaping. The policy is then updated using GRPO. The method is compared with other PRMs and self-rewarding baselines, with outcome-only GRPO included as a supervision-assisted reference.
Critical strengths
The paper explicitly addresses the concern that agreement among multiple judges does not necessarily reflect true action quality, particularly when their errors are correlated. Correlation-aware aggregation and screening by a source-calibrated gate are reasonable mechanisms to address this concern.
Critical weaknesses
- The paper is poorly written and difficult to follow. It introduces terms such as ordinal sibling advantage, reliability rule, and fail-closed before clearly explaining their meaning in the method. This makes it difficult to understand the overall approach and assess its technical contributions.
- The method relies on a lot of hyperparameters and rules. Although the appendix describes source calibration and hyperparameter search ranges, it remains unclear how sensitive final performance is to these choices and how much recalibration is required for a new target domain. This raises concerns about the practical complexity and generalizability of the approach.
- The computational cost is substantial: CrossView-PRM uses 7.5× relative tokens and 2,500 H100 GPU-hours per seed, compared with 4.5× and 1,850 GPU-hours for Majority-view PRM. Although the paper acknowledges this trade-off, the comparisons include only the policy-update budget, not total computation. It therefore remains unclear whether the performance gains justify the additional cost, or whether simpler baselines could achieve comparable results with the same total compute budget.
Questions for authors
- Since labeled source diagnostics are available, how does the proposed method compare with directly training a reward model on the same source labels and transferring it to the target domain?
- Are the same source calibration data and rules used across all downstream tasks and benchmarks? Which components, if any, require recalibration for a new target domain?
- For the ablation study in Appendix E.6, could the authors clarify what homogeneous views means and how online potential is implemented?
- Does the method’s performance depend on how well the judges complement each other? Do the authors have any suggestions for selecting judges with complementary views?