- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CCID, a multi-round reranking method for recommendation agents. A frozen OneRec-1.7B model proposes candidates, and its hidden state for each candidate is shared as a vector tied to that candidate's ID. A utility role checks if a candidate fits, a risk role checks if moving it up could push out a correct top item, and an adjudicator updates the ranking over several rounds, stopping early when it is stable. Lessons from past runs are pulled from a fixed memory.
Critical strengths
S1. The controls on message content are well designed. Score-only, pooled, within-user shuffled and cross-user states separate how much information is passed from whether it is attached to the right candidate, and the ordering is the same on all four datasets.
S2. Adaptive stopping is a useful, practical result. Reporting harmful revisions next to corrections is also good practice.
Critical weaknesses
W1. The paper combines utility and risk judgments over multiple rounds to reach a final decision. However, the risk role seems closer to judging whether the current top-ranked item is correct. Simply evaluating how well the current top item fits the user might achieve similar results.
W2. The final recommendation requires multiple rounds of utility and risk assessments, which can be costly. Given the same time and compute budget, it is unclear whether this approach is better than having a single model make a more careful judgment. The method averages 2-3 rounds, compared with one round for the single-round variant.
W3. The paper shows improvements over its own variants, but it is unclear whether the complex multi-agent debate structure is necessary. There is no ablation removing each role individually, nor a comparison with a single model under the same compute budget.
W4. The paper does not directly compare with existing methods under the same evaluation conditions. Since the candidate sets and evaluation protocols differ, it is difficult to judge whether the proposed method offers meaningful improvements over existing approaches.
Questions for authors
Q1. Given the same time and compute budget, does the proposed method outperform a single model making a more careful judgment?
Q2. Computational cost is important for recommender systems deployed in real-world services. Could the authors report the actual per-request latency measured on their experimental server and compare it with OneRec alone and the single-round variant?
2 comments
Thank you for the response. While they address some of my questions, I am keeping my score for the following reasons:
- Necessity of the roles (W1, W3). Gains over simpler alternatives (a second utility head, scalar states, rule fusion) are small and lack variance. Moreover, given its supervision target, the risk role effectively just predicts whether the current leader is correct, leaving the need for a multi-role debate unclear.
- Equal budget (W2, Q1). All baselines split the budget into multiple short evaluations rather than testing a single, more thorough pass (e.g., one longer call or a stronger model), which was the main point of Q1. Gains over the recurrent single model are also modest.
- Cost (Q2). Latency is only compared against variants of similar or higher cost. Relative to OneRec alone or the single-round variant, CCID remains substantially slower for only a moderate quality gain.
- Comparison with existing methods (W4). Only Re2LLM was added under the current protocol. Since OneRec's numbers in Table 6 match Table 2—which supposedly uses a different protocol—it remains unclear why other reference methods could not be compared under the same setup.
Response to Reviewer vo1R
Thank you for focusing on the quality obtained for the additional computation and the cost of real deployment. We respond to W/Q as follows.
Table 1: Final top-1 changes from the single-round starting point
We can observe that CCID gives the larger net correction in every domain.