acceptodds
Under review as a conference paper at ICLR 2027

ECPO: Evidence-Coupled Policy Optimization for Evidence-Certified Candidate Ranking

Abstract

Ranking systems used for decision support should expose evidence that can be audited independently of the model's ranking scores. We study a closed-roster setting in which a policy jointly emits a Top- ranking and candidate-linked, span-grounded evidence bundles. Evidence-Coupled Policy Optimization (ECPO) uses rank-blind bundle permutation, global one-to-one assignment, candidate-level contrastive separation, and adaptive constraints to couple the ranking with its certificate. On MAVEN-ERE and RAMS, ECPO improves verifier-audited traceability, rank-blind recovery, and human comparative support relative to matched controls in the tested provenance settings. The gains persist under held-out evaluators, other 8B/14B backbones, and a HotpotQA answer-ranking interface. Ordinary NDCG and runtime remain explicit trade-offs.

Primary area:
reinforcement learning
Keywords:
ECPO: Evidence-Coupled Policy Optimization for Verifier-Audited Candidate Ranking

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ENhEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper aims to produce a better optimization method for improving the rank-based verification capabilities of LLMs than something done in post-hoc analysis. The author/s motivate this by citing that being able to verify outputs as evidence is necessary for improving an LLM’s reasoning and generation capabilities for decision-support tasks. The paper evaluates its proposed Evidence Coupled Policy Optimization (ECPO) method on only two open-weight models and on two datasets (MAVEN, RAMS) for the entire paper.

Critical strengths

I tried to find an angle where I can appreciate the paper more but the content and writing of the paper itself is very unclear and confusing which makes it hard for readers like me to appreciate its contributions. I guess I appreciate the authors’ efforts towards putting together a proposed optimization method that aims to improve verification consistency of LLMs but the corresponding experiments, writing, results do not support this goal coherently in the paper.

Critical weaknesses

One of the main critical issues with the paper is that it is highly unreadable due to the written content being very AI-sounding and the overall writing of the paper could be further significantly improved in quality and readability. I acknowledge that the authors did mention they use AI for the paper and take responsibility for it, but I still highly encourage the authors to edit out any AI written text in the main paper given that they are very poorly written and are obvious to readers. The paper is also full of uncontextualized technical jargons such as “finite window-local roster” “frozen event extraction” “four-stage event skeleton” “auditable support contract” “source-normalized labels”, etc. which could have been rewritten in very crisp and simple language instead. It’s very hard to grasp what the authors wanted to say and hard to differentiate which were the authors’ main idea vs. what has been AI generated from this work.

The paper has no substantive and in-depth discussion of the results. The content written for Section 6 barely makes any sense to the reader and does not make it clear whether the proposed methods are responsive to the research question. Basic simple questions remain unanswered such as “What method per model works?” “Which model is better at what particular test dataset?” “What metrics actually matter for the task?” “How do the methods tested fare against simple and stronger baselines such as through prompting off-the-shelf LLMs like GPT-Sol?” “What type of models have better performance?” “What error analysis exposes the selected models weaknesses for the task?” These are very simple questions you would expect to be discussed thoroughly for an ICLR paper but these are completely missing and not properly and clearly written for the current submission.

The paper uses two types of models for its main results: Llama-3.1-8B-Instruct and Qwen3-14B. There is also no baseline comparison. These two are also grounds for clear rejection of the paper for lack of technical rigor and does not meet the empirical standards of ICLR. I highly and strongly recommend the authors to read previously accepted papers from ICLR/NeurIPS/ICML to get a level of how rigorous technical experiments are conducted and accepted in these venues. Simply running your methods on two models is simply not enough, much less selecting commercial models.

Questions for authors

N/A

Reviewer_p6dXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces Evidence-Coupled Policy Optimisation (ECPO), a framework for jointly generating candidate rankings and candidate-specific evidence bundles.

Critical strengths
  • Explores an important open problem, i.e., ensuring that a ranking produced by a model is supported by evidence
  • Comprehensive evaluation protocol
Critical weaknesses
  • The use of custom evaluation metrics constructed around the specific problem formulation raises some concerns. Larger gains are demonstrated on these metrics, as opposed to the previously used NDCG.
  • RM-only+Det achieves higher NDCG compared to the proposed ECPO.
  • High computational cost compared to deterministic alternatives, especially considering that RM-only+Det achieves higher NDCG compared to the proposed ECPO. How does this affect scalability and potential real-world adoption?
  • The generalisability of the proposed approach outside the specific setting is unclear.
  • It is unclear whether the improved recovery achieved by the proposed ECPO would lead to improved performance on the downstream task.
  • The paper is difficult to read in some sections.
  • The related work section is very brief, and does not clarify the current state of the art.
  • The choice of underlying LLM is not explained.
Questions for authors
  • How is the better NDCG of RM-only+Det compared to EXPO explained?
  • How dependent is the method on the predefined skeletons and role structures?
  • Why were the new metrics proposed? What is their benefit compared to the existing ones?
  • Is the increased computational overhead of the proposed approach justified?
  • How sensitive is the proposed method to the specific verifier design?
Reviewer_pB14OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ECPO, which aims to enhance the traceability and verifiability of evidence for ranked candidate results. Unlike traditional approaches that focus solely on optimizing ranking scores, ECPO treats the top-K ranked candidates and their corresponding sets of evidence as a joint optimization objective. The method integrates trajectory rewards based on predefined event skeletons, evidence validity verification, rank-agnostic evidence matching, and candidate-level contrastive optimization; it further employs adaptive constraints to improve the consistency between the rankings and the associated evidence.

The authors conducted experiments on the MAVEN-ERE and RAMS datasets, evaluating the method’s effectiveness through various verifiers, model backbones, transfer experiments on HotpotQA, and human evaluations. As reported, ECPO outperforms several baselines in terms of evidence-verification metrics and candidate-retrieval metrics, though it involves certain trade-offs regarding traditional NDCG scores and inference efficiency.

Critical strengths

Jointly optimizing ranking and verifiable evidence holds significant research value. Instead of treating evidence merely as an explanation provided after the ranking process, this approach requires the ranking results to be accompanied by verifiable, relevant candidate evidence. By removing candidate identifiers and ranking cues—and subsequently reconstructing the candidates—this framework enables a more rigorous evaluation of the consistency between the evidence and the ranking results. This represents a key advantage over traditional ranking methods and post-hoc explanation techniques.

Critical weaknesses

There is an inconsistency between the experimental results for the core method and the configurations detailed in the appendix.

The ECPO results reported in Table 1 of the main text match those for the "Slot + Fixed" configuration in Table 27 of the appendix; however, the "RBCC + Dual" configuration—the primary method proposed in the paper—shows different performance figures in Table 27. Furthermore, Table 34 in the appendix indicates that the standard NDCG metric changes during the experiment involving the removal of fixed-output identifiers; in principle, this metric should be determined solely by the candidate ranking and the relevance labels. These issues make it unclear which specific method configuration corresponds to the paper’s primary conclusions, and they complicate the assessment of the proposed key innovations and their associated experimental gains. This is my primary concern regarding the reliability of the paper.

Questions for authors

Please clarify which specific configuration in Appendix Table 27 corresponds to the ECPO results presented in Table 1 and Table 3 of the main text. In particular, why are the ECPO values ​​in the main text identical to those for the "Slot + Fixed" configuration, while the results for "RBCC + Dual" differ? Please provide a comprehensive mapping between the full methodology and the results obtained under the same protocol using a fixed slot-cycle baseline. Additionally, explain why the standard NDCG values ​​in Table 34 change after the removal of identifiers from the fixed outputs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.