acceptodds
Under review as a conference paper at ICLR 2027

PolyPRM: Mandate-Conditioned Process Evaluation for Role-Specific Agents

17 views

Abstract

Process reward models (PRMs) typically evaluate intermediate reasoning and actions against a single objective. For role-specific agents, however, the same candidate process may have different values under different decision responsibilities. We introduce RIVA-Bench, a matched benchmark that holds decision contexts and candidate slates fixed while varying task-defined mandates and selector-independent, provenance-tracked utility targets. RIVA-Bench contains 111,805 labeled candidate processes, 12,872 decision states, and 50 roles across eight tasks. We also introduce PolyPRM, a 1.5B mandate-conditioned evaluator that learns across roles while preserving their distinct candidate orderings. PolyPRM achieves the highest direct-selection score on all eight tasks and outperforms frontier LLM judges supplied with the same complete mandate. It improves frozen ReAct and Debate agents over their base selection policies in all 16 evaluated settings and attains the best or tied-best non-oracle outcome in 15. Counterfactual ablations show that its selections follow mandate semantics rather than role names. Joint training further matches or exceeds routed role-specific experts across all six datasets while using one checkpoint and \(1/r\) of their aggregate training data.

Primary area:
datasets and benchmarks
Keywords:
Process Reward Modeling, LLM Agents, Role-Conditioned Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6z84OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces a benchmark and a process reward model for evaluating candidate actions under different role-specific objectives. The model uses the active role’s objectives and constraints to score candidates and guide agent decisions.

Critical strengths

None

Critical weaknesses

1.The paper is difficult to follow, and the presentation needs substantial improvement in clarity.

2.The choice of Qwen2.5-1.5B as the backbone is not sufficiently convincing.

3.The paper contains numerous typo and formatting issues, such as “orderings Fu et al. (2026).”

4.The paper’s main contribution and novelty are unclear.

Questions for authors

See weaknesses.

3 comments

Reply to W1 & W3: Presentation Clarity and Formatting

We address these comments together because both concern the paper’s organization and presentation. The manuscript follows “problem formulation → benchmark construction → evaluation method → experimental validation.” It first explains how the value of the same candidate process varies with role objectives, then introduces RIVA-Bench to characterize these differences under fixed contexts and candidate sets. It subsequently presents mandate-conditioned PolyPRM and evaluates the design through candidate selection, downstream tasks, and ablations. In this organization, the mandate specifies the evaluation objectives, RIVA-Bench provides the benchmark, and PolyPRM implements the conditional scoring function.

We will correct citation-integration issues such as “orderings Fu et al. (2026)” and check spelling, citations, and formatting consistency throughout the manuscript.

The broader concern that the paper is difficult to follow does not yet identify specific sections or reasoning steps. Could the reviewer indicate which passages, concept definitions, or methodological explanations require clarification? Concrete examples would help us understand the concern precisely and make targeted improvements.

Reply to W2: Backbone Choice and Extended Evaluation

We chose Qwen2.5-1.5B to build a lightweight process evaluator that can be called repeatedly during agent decision-making. PolyPRM scores multiple candidates at each decision point, making both evaluation cost and selection performance relevant to its practical use. The 1.5B backbone with LoRA allows us to test whether effective role-conditioned process evaluation can be achieved at modest computational cost.

To further examine this choice, we extended the experiments to Qwen2.5-7B, Llama-1B, and Llama-3B. The models use the same data splits, training recipe, and seed settings, all with LoRA (r = 8). Results across eight task configurations are shown below; each task uses its corresponding metric, with higher values indicating better performance (↑).

Task ↑Qwen2.5-1.5BQwen2.5-7BLlama-1BLlama-3B
Trading0.7740.7810.7770.776
Portfolio0.7870.7880.7940.780
Market Making0.8390.8310.8250.834
Job Negotiation0.7190.7280.7190.723
FinQA0.9290.9290.9420.973
Retail0.8280.8420.8250.872
Support0.7190.7140.8190.811
Negotiation0.8130.8310.8130.841

The benefits of larger backbones are task-dependent, without uniform improvements across tasks. Qwen2.5-7B performs close to its 1.5B counterpart overall. Llama-3B improves on FinQA, Retail, and Negotiation, while performing worse on Portfolio. The benefits of mandate conditioning are retained across all four backbones, supporting its applicability across model sizes and families.

Computational cost also supports the smaller backbone as a default: training each task configuration on a single GPU takes approximately 8–17 hours for Qwen2.5-1.5B, compared with 12–47 hours for Qwen2.5-7B. Thus, Qwen2.5-1.5B offers a reasonable balance between performance and computational cost, and PolyPRM’s effectiveness is not confined to this backbone. We will include these extended results in the revised manuscript.

Reply to W4: Main Contributions and Novelty

The central idea is simple and practically meaningful: the same agent step in the same context can have very different value for different roles, so process evaluation must specify whose objectives it serves. A high score under a common task metric does not automatically imply that an agent’s choices satisfy a particular user’s objectives, constraints, and interests. Our paper addresses this gap.

The paper makes three contributions:

  1. Problem formulation: role-conditioned process evaluation. We explicitly model candidate-process value as Vθ(s,h,τ∣m)V_\theta(s,h,\tau\mid m), where mm represents the role’s mandate. With the state, history, and candidate process held fixed, different mandates can imply different or even opposite rankings.

  2. Benchmark construction: RIVA-Bench. We provide a benchmark covering 8 tasks, 50 roles, and 111,805 candidate targets. By holding decision contexts and candidate sets fixed while varying role objectives and corresponding utilities, it tests whether evaluators can identify role-dependent differences in value.

  3. Method and empirical validation: PolyPRM. We use a simple shared evaluator to score candidate processes conditioned on the mandate, and validate this design through candidate selection, downstream tasks, and role ablations. For example, Table 3 shows that removing role names preserves role-dependent behavior, whereas removing the mandate yields zero MCA on both Retail and Email, supporting the importance of explicit objectives and constraints for role-conditioned evaluation.

As emphasized in the introduction, our contribution lies in formulating this evaluation problem, providing a controlled benchmark that makes role-dependent differences identifiable, and validating a simple, effective approach. The standard model architecture enables an economical implementation; the contribution encompasses the problem formulation, benchmark, and empirical findings.

Dear Reviewer 6z84,

Thank you for your review. We hope our clarifications of the paper’s organization and core contributions, together with the new backbone scaling experiments, address your concerns. If so, we would appreciate your consideration of a higher score. We would also welcome specific feedback on any passages or explanations that remain unclear and are happy to answer further questions.

Best regards,

The Authors

Reviewer_tnVzOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

This work addresses the limitation of traditional Process Reward Models (PRMs) that evaluate intermediate agent actions against a single scalar objective. The authors argue that process value inherently depends on role-specific decision responsibilities (mandates). The work proposes RIVA-Bench, a benchmark with labeled candidate processes across 8 tasks and 50 roles, alongside PolyPRM, a 1.5B parameter mandate-conditioned evaluator based on Qwen2.5-1.5B with LoRA. PolyPRM uses mandate-first process encoding and a role-balanced objective to re-rank candidate steps for frozen agents under active mandates. While the empirical benchmarks are broad, the paper's algorithmic novelty is minimal, and several technical claims lack sufficient methodological support.

Critical strengths
  1. Formulating process reward evaluation as a mandate-conditioned function Vθ(s,h,τ∣m)V_\theta(s, h, \tau | m) correctly highlights the multi-stakeholder and subjective nature of real-world decision-making.
  2. The benchmark's matched counterfactual construction, holding the decision state, history, and candidate slates fixed while varying only the active mandate, which provides a controlled setup to isolate mandate-dependent ranking reversals.
  3. The ablation experiments effectively show that PolyPRM responds to mandate semantics rather than memorizing role titles.
Critical weaknesses
  1. The core technical contribution essentially boils down to prepending a text prompt prefix [MANDATE] to the input context before passing it into a standard cross-encoder scoring head. Packaging standard prompt concatenation with LoRA fine-tuning as a fundamental algorithmic breakthrough feels significantly oversold.
  2. The loss objective combines four separate terms (Lrank,Lord,Lutil,Laux)(\mathcal{L}_\mathrm{rank},\mathcal{L}_\mathrm{ord},\mathcal{L}_\mathrm{util},\mathcal{L}_\mathrm{aux}). However, the appendix reveals that utility regression and auxiliary action prediction yield no consistent positive contribution, cluttering the objective with unnecessary hyperparameter complexity.
  3. The loss-function ablation is conducted on only a single privacy-restricted dataset (Email Response) across just two random seeds. The authors fail to demonstrate whether this complex multi-loss setup is necessary or beneficial for the remaining 7 tasks.
  4. All PolyPRM experiments exclusively use a single 1.5B backbone (Qwen2.5-1.5B). There is zero exploration of model capacity scaling (e.g., 0.5B vs. 7B vs. 14B), nor is there any positional ablation testing "Mandate-First" versus "Mandate-Last" sequence formatting.
  5. Nearly half of the utility targets in RIVA-Bench rely on LLM-assisted process-quality judgments. Relying heavily on synthetic LLM supervision to train an evaluator that claims to outperform LLM judges raises serious concerns regarding circular evaluation and label noise leakage.
Questions for authors
  1. Why were utility regression and auxiliary prediction retained in the final loss formulation if ablations show they contribute no consistent value?
  2. How does PolyPRM's performance scale when varying the backbone parameter size?
  3. Does placing the mandate block at the end of the input context ("Mandate-Last") degrade attention or selection performance compared to "Mandate-First"?
  4. Given that 44.2% of RIVA-Bench utilities depend on LLM quality judgments, how do you guarantee that PolyPRM is not merely overfitting to the systemic biases of the annotator LLMs?
  5. Why was the loss-function ablation limited to the Email Response dataset rather than evaluated across the full task suite?
6 comments

Reply to W1: Contribution and Role-Conditioned Process Evaluation

Thank you for this comment. Our central contribution is a simple, effective, and practically meaningful idea: the value of an agent’s process depends on its decision role in a specific setting, and the same step can have opposite values for different roles.

A high benchmark score does not automatically imply that an agent can serve different decision-makers in real-world settings. A shared evaluation criterion can measure whether a step satisfies a predefined task objective, but practical deployment also requires asking: whom does this step benefit, and does it respect their responsibilities and constraints? As illustrated by the climate-negotiation example in our introduction, a market-oriented role may prioritize cross-border carbon trading, while a transition-oriented role favors ending new fossil-fuel investment. Collapsing these judgments into a single generic quality score can obscure differences that matter for actual decisions.

We explicitly formulate this underexplored problem in agent process evaluation and provide a benchmark and method for studying it:

  • Problem formulation: We define process value as Vθ(s,h,τ∣m)V_{\theta}(s,h,\tau\mid m), making evaluation conditional on the active role’s objectives, authority, and constraints while preserving conflicting orderings across roles.
  • Matched benchmark: RIVA-Bench covers eight tasks, 50 roles, and 111,805 candidate processes. It fixes the state, history, and candidate slate while varying the mandate and its corresponding utilities, enabling controlled evaluation of how decision roles change process value.
  • Simple and effective validation: PolyPRM learns role-conditioned process values through a shared evaluator. The original experiments show that removing role names largely preserves performance, whereas removing mandate content reduces mandate-counterfactual accuracy to zero on Retail and Email. This demonstrates reliance on substantive decision responsibilities. Frozen-agent experiments further evaluate the resulting benefits for candidate selection.

Thus, a simple implementation is compatible with a meaningful contribution: we identify an important evaluation problem, construct a controlled benchmark, and demonstrate the value of role information for process evaluation and agent decisions through a straightforward method. Page 2 of the introduction already states that the contribution extends beyond conditioning alone. We will further emphasize this connection between the problem, benchmark, and empirical evidence in our presentation.

Reply to W2 & W3, Q1 & Q5: Training Objectives and Cross-Task Ablations

Thank you for these comments. They raise two connected questions: why the four-term objective retains auxiliary components without consistent gains, and whether ablations conducted only on Email sufficiently support this design. We address them together because the new cross-task ablations examine both the generality of the original observations and the roles and rationale for retaining individual components.

1. Extending ablations beyond Email.

The original analysis used Email’s two roles with graded utilities because they support both within-state selection and cross-state selective prediction. We have now extended component-wise ablations to Negotiation, FinQA, Retail, and Support, and added auxiliary-supervision ablations on Trading and Portfolio.

2. Ranking learns candidate orderings; ordinal supervision establishes a utility scale.

Ranking directly constrains candidate orderings within the same state and mandate. Ordinal supervision uses eleven utility levels to anchor the prediction scale. These objectives support candidate selection and the use of scores for cross-state comparison, threshold-based decisions, and selective prediction.

Component removedTaskMetricChange after removal
RankingNegotiationPairwise accuracy ↑−0.080
RankingNegotiationMandate-counterfactual accuracy ↑−0.242
RankingFinQAPairwise accuracy ↑−0.027
RankingSupportSelection accuracy ↑−0.103
RankingEmailSelected utility ↑−0.064
OrdinalNegotiationExpected calibration error (ECE) ↓+0.052
OrdinalRetailExpected calibration error (ECE) ↓+0.072
OrdinalSupportExpected calibration error (ECE) ↓+0.126
OrdinalSupportCross-state Spearman correlation ρ ↑−0.073
OrdinalRetailUtility at 50% coverage ↑−0.010
OrdinalEmailUtility at 50% coverage ↑−0.017

↑ indicates higher is better; ↓ indicates lower is better. Changes are calculated as ablated − full. Thus, negative changes in ↑ metrics and positive changes in ↓ metrics indicate degradation after removing the component. Comparisons use a fixed seed within each task.

The evidence extends beyond Email: ranking supports mandate-conditioned orderings, while ordinal supervision supports score calibration and cross-state use. These needs are distinct: removing ordinal supervision improves within-state selection accuracy on Support while worsening calibration and cross-state metrics.

3. Utility regression and auxiliary objectives provide supplementary supervision for specific decision needs.

Utility regression directly supervises continuous utility, complementing ordinal supervision over discrete levels. The auxiliary objectives comprise action prediction and diagnostics, using action labels and task-native outcome and evidence/citation components to provide additional supervision to the shared representation. They were introduced to use these available signals, rather than on the assumption that every component would improve selection accuracy on every task.

The additional experiments identify benefits in the following settings:

Component removedTaskMetricChange after removal
Utility regressionNegotiationThreshold-decision metric at τ = 0.6 ↑−0.013 / −0.020
DiagnosticsNegotiationThreshold-decision metric at τ = 0.7 ↑−0.066 / −0.0055
Action prediction (valid action labels)TradingBrier score ↓+0.070 / +0.062

The same ablated − full convention applies; slash-separated values report two training seeds. The differences in the first two rows are statistically significant in both seeds. Brier score measures squared prediction error, so +0.070 / +0.062 after removing action prediction indicates worse prediction quality in both seeds.

These benefits are concentrated in particular tasks and metrics, and some are small. Nevertheless, threshold-based decisions and score quality are meaningful use cases: applications may need not only to select the highest-scoring candidate but also to decide whether a candidate should be accepted based on its score. Our rationale for retaining these components is their supplementary contribution to such needs, rather than a claim of universal gains across tasks.

4. Adding comprehensive ablations and clarifying applicability.

We will include more comprehensive cross-task ablation tables in the revised paper, separately reporting within-state selection, cross-state ranking, calibration, and threshold-based decisions, together with the available auxiliary labels and training configurations for each task. This will clarify the design purpose, empirical contribution, and applicability of each loss component, addressing the limited coverage of the original Email-only analysis.

Reply to W4 & Q2: Backbone Scaling

Thank you for this question. We have completed additional scaling experiments in two backbone families: Qwen (1.5B → 7B) and Llama (1B → 3B), covering eight task configurations. The comparisons use matched data splits, training recipes, and random seeds, with LoRA (r=8) throughout. Direct-selection results are:

TaskQwen-1.5BQwen-7BLlama-1BLlama-3B
Trading0.7740.7810.7770.776
Portfolio0.7870.7880.7940.780
Market Making0.8390.8310.8250.834
Job Negotiation0.7190.7280.7190.723
FinQA0.9290.9290.9420.973
Retail0.8280.8420.8250.872
Support0.7190.7140.8190.811
Negotiation0.8130.8310.8130.841

Higher is better for every column; metrics retain their task-specific definitions.

Scaling yields task-dependent gains rather than uniform improvements. The two Qwen scales perform similarly overall. Llama-3B shows clearer gains on FinQA, Retail, and Negotiation, alongside a decline on Portfolio. Mandate conditioning remains beneficial across all four backbones, indicating that its contribution extends beyond the original model size and family.

These results support the performance–cost tradeoff motivating our lightweight backbone: single-GPU training takes approximately 8–17 hours per configuration for Qwen-1.5B, compared with 12–47 hours for Qwen-7B. Larger models offer gains on some tasks, while the smaller backbone remains a practical choice for efficient process evaluation. We will include these scaling results in the revised paper.

Reply to W4 & Q3: Mandate Position

When training and evaluation use the same format, we find no consistent performance disadvantage for Mandate-Last relative to Mandate-First. Our original format places the mandate first to establish the role objectives explicitly. The scoring head uses mean pooling over all non-padding tokens rather than relying on the final token. To examine the positional choice, we conducted paired training experiments on Negotiation and Retail, each with two seeds.

The two conditions differ only in the location of the mandate block. Input content, token counts, initialization, batch order, and optimization steps are matched, with no input truncation in either condition. All metrics below are higher-is-better (↑).

Task / Metric ↑Seed A: FirstSeed A: LastSeed B: FirstSeed B: Last
Negotiation: Selected utility0.8060.8260.8190.787
Negotiation: MCA0.4780.4330.5290.495
Retail: Region accuracy0.8280.8360.8280.819
Retail: MCA0.9540.9550.9560.954

For Negotiation, the selected-utility differences have opposite signs across seeds; Retail selection performance is also close between formats. Using paired cluster bootstrap with 5,000 resamples, no metric meets our prespecified criterion for a consistent First advantage: a statistically significant advantage in both seeds. MCA measures whether selections follow the supplied mandate when role objectives conflict. Negotiation shows slightly higher MCA point estimates for First, but both seed-specific confidence intervals include zero; Retail achieves approximately 0.95 MCA with either format. These results support that placing the mandate first is not necessary for effective role-conditioned scoring in these experiments.

Training with Mandate-Last differs from moving the mandate only at inference time. We also evaluated First→Last and Last→First. Under these format mismatches, selected utility falls by 0.09–0.13 on Negotiation, and region accuracy falls by 0.33–0.45 on Retail. This indicates dependence on the input layout learned during training and supports keeping the format consistent between training and deployment.

Reviewer_FAgDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a process reward model conditioned on mandates PolyPRM that evaluates intermediate candidate processes according to the objective, authority and constraints of the active decision role, rather than a single, role-independent notion of process quality. It also constructs a matched benchmark Bench fixing state, history, and candidate slate while varying role mandates, covering 111,805 candidates, 12,872 states, and 50 roles across eight tasks. A shared 1.5B evaluator improves direct selection and frozen-agent decisions.

Critical strengths
  1. Standard PRM work assumes a universal notion of good reasoning/action, but real-world, multi-stakeholder agents involved in negotiation, customer dispute adjudication and financial decision-making require context-dependent evaluation criteria. The core insight is that: Process quality is not intrinsic to a trajectory, but rather conditional on the decision-maker's mandate. This insight is clear and impactful for agent research.

  2. The constructed RIVA-Bench is large and diverse: 111,805 candidate processes, 12,872 decision states, 50 task-specific roles across eight tasks, including finance, customer support, retail disputes, email response, and multi-party negotiation.

  3. Extensive experiments across eight diverse task configurations, including evaluator-free scoring regimes (gold answers, rules, observed decisions, native payoffs), reduce the risk that results reflect only alignment with an LLM judge.

Critical weaknesses
  1. 44.2% of candidate targets include an LLM-assisted, mandate-conditioned quality judgement. This paper only validates the supervision integrity layer on 292 retail dispute units, and the filter-only control actually achieves a slightly higher point estimate than the full integrity layer on the rule-only reference. While the evaluator-free tasks and NegoBench native payoffs are helpful, this paper requires stronger human validation and per-task reporting to show the proportion of supervision that is native versus LLM-derived.

  2. PolyPRM is built on Qwen2.5-1.5B with LoRA. Although lightweight deployment is one of its strengths, the paper does not investigate scaling trends. It is therefore unclear how performance changes with larger backbones or full-parameter fine-tuning.

  3. All test‑set mandates are clean, well‑specified natural‑language objectives written by humans/benchmark designers. Real‑world use‑cases will often have ambiguous, partial, or corrupted mandate text.

Questions for authors
  1. Could you provide more information on the integrity checks beyond the Retail Dispute example? More specifically, what proportion of the hybrid LLM-judged utilities were corrected or abstained, and what human or inter-judge validation was performed to assess reliability?

  2. For the “frontier LLM judge + mandate” condition, what prompt structure and rubric guidance were used?

  3. For Negotiation, can you compare joint training and routed experts on the same 10 roles or the same 32 roles to separate transfer from coverage?

4 comments

Reply to W1 & Q1: Supervision Sources and Reliability

Thank you for these suggestions. We have re-examined the supervision sources and added per-task intervention statistics and human validation.

Correction to supervision provenance. The original 44.2% mistakenly counted the rule-derived quality components in Market Making and FinQA as LLM judgments. The actual proportion of candidate records containing an LLM-derived component is 16,211/111,805 (14.5%), comprising Trading (5,399), Portfolio (3,312), and Retail Dispute (7,500). The remaining five tasks use rules, task-native outcomes, or observed decisions. We have corrected this figure in the main text and highlighted the revision in blue.

Cross-domain supervision and noise handling. Across domains, task-native outcomes do not fully capture reasoning support, constraint adherence, and process quality under a particular mandate. We therefore supplement these outcomes with LLM judgments. To address noise in these judgments, our supervision pipeline incorporates task-native outcome grounding, evidence and mandate-consistency checks, and targeted label interventions (§3.3, Appendices B.3 and C.6):

TaskChecks and interventions
TradingLookahead checks capped the quality scores of 117 records (2.2%); role-conditioned relabeling was also performed.
PortfolioHard-constraint violation checks capped the quality scores of 202 records (6.1%); role-conditioned relabeling was also performed.
Retail integrity experimentCorrections affected 550/1,275 case–role units (43.1%), corresponding to 1,327/6,375 candidate records (20.8%); 16 units (1.3%) were flagged for abstention.

Retail interventions were restricted to training and validation data; the test set was untouched. The integrity layer was evaluated in a dedicated controlled experiment, while the Retail checkpoint reported in the main results used the original labels.

Human validation. We added a blinded audit involving four annotators and 190 candidate pairs. Annotators assessed randomly ordered candidates using the mandate and evidence available at decision time, without seeing the original labels or model scores. Label rankings agreed with human judgments in 88% of the evaluated pairs, with inter-annotator agreement of κ = 0.75. These results provide direct evidence that the hybrid supervision aligns with human mandate-based judgments. Appendix B.3 also reports a replication of the DeepSeek integrity experiment using GPT-4o-mini; both showed reductions in silent labeling errors.

Interpreting the filter-only result. Under an independent rule-only reference, the full integrity layer scored 0.969/0.966 across two training seeds, compared with 0.976/0.976 for filter-only and 0.834/0.842 for the original labels. These results identify the detection and handling of unreliable labels as the main source of improvement. The full layer retained all 5,250 training records, whereas filter-only retained 2,985. Correction provides a way to retain recoverable examples, but the current experiments do not establish an additional downstream performance benefit over filtering alone.

Reply to W2: Backbone Scaling

Thank you for this suggestion. We have completed scaling experiments in two backbone families: Qwen (1.5B → 7B) and Llama (1B → 3B), covering eight task configurations. The comparisons use matched data splits, training recipes, and random seeds, with LoRA (r=8) throughout. Direct-selection performance is reported below:

TaskQwen-1.5BQwen-7BLlama-1BLlama-3B
Trading0.7740.7810.7770.776
Portfolio0.7870.7880.7940.780
Market Making0.8390.8310.8250.834
Job Negotiation0.7190.7280.7190.723
FinQA0.9290.9290.9420.973
Retail0.8280.8420.8250.872
Support0.7190.7140.8190.811
Negotiation0.8130.8310.8130.841

Scaling yields task-dependent gains rather than improvements across all tasks. The two Qwen scales perform similarly overall. Llama-3B shows larger gains on FinQA, Retail, and Negotiation, alongside a decline on Portfolio. Mandate conditioning remains beneficial across all four backbones, indicating that its contribution extends across model sizes and families.

These results support the performance–cost tradeoff motivating our lightweight backbone: single-GPU training takes approximately 8–17 hours per configuration for Qwen-1.5B, compared with 12–47 hours for Qwen-7B. Larger backbones improve some tasks while increasing computational cost.

Full-parameter fine-tuning experiments are underway. Due to computational resource constraints, they may not all finish within the rebuttal period. We will report the results as soon as they are completed and verified, and incorporate them into a subsequent revised version.

Reply to W3: Incomplete and Perturbed Mandates

Thank you for this suggestion. The original paper already examines whether selections follow mandate semantics through role-name removal, mandate removal, and wrong/shuffled-mandate controls (§6.1, Table 3). These experiments show that selections respond to the supplied objectives and constraints. We extend this analysis with experiments on missing information, alternative wording, and distracting text.

Incomplete objectives. The original Negotiation setting already provides partial objectives: each mandate lists at most four positive and four negative objectives, covering an average of 44% of nonzero objectives and 58% of total absolute valuation, while evaluation uses the native payoff defined over all objectives.

To control the degree of incompleteness, we evaluate two checkpoints on 734 states, fixing the states, candidates, and full ground-truth payoff while reducing the objectives visible to the model. We use five nested random masks. Selected utility is reported below:

Objective information visible to the modelCheckpoint 1Checkpoint 2
All objectives and valuations0.8410.856
Approximately 80% of objectives retained0.8210.834
Approximately 53% of objectives retained0.7950.802
Approximately 30% of objectives retained0.7710.768
All objective directions retained, numerical valuations hidden0.7910.816

Performance decreases progressively as information is removed; with approximately 30% of objectives retained, selected utility remains 0.771/0.768. Losses are concentrated in cases where the hidden objectives change the optimal candidate set. When that set remains unchanged, we detect no clear loss, linking degradation to the decision relevance of the missing information.

Alternative wording and distracting text. We add irrelevant constraints and distracting historical text across three tasks, keeping the original objectives, candidates, and evaluation criteria unchanged:

TaskIrrelevant constraints: Δ selected utilityDistracting history: Δ selected utility
Negotiation (two checkpoints)−0.012 / +0.014−0.012 / +0.011
Retail (range across three checkpoints)−0.005 to +0.001−0.006 to −0.001
Support (range across three checkpoints)−0.008 to −0.0050.000 to +0.015

Changes in selected utility remain within ±0.015 under these distractions. For the more substantial transformation from an objective list to prose, we additionally evaluate training with diverse mandate formats. The prose template used at test time is excluded from the training augmentation:

Negotiation modelSelected utility with the original mandate formatUtility change under prose rewriting
Original checkpoint0.835−0.045
Format augmentation, seed10.836−0.018
Format augmentation, seed20.828−0.005

Format augmentation reduces the utility loss under unseen prose wording from 0.045 to 0.018 and 0.005, while retaining similar performance with the original mandate format. These experiments extend the original semantic controls to incomplete information, alternative wording, and distracting text, and demonstrate a practical way to improve stability through format-diverse training.

Reply to Q2: Prompt Structure and Rubric Guidance for LLM Judges

Thank you for this question. The “LLM judge + mandate” condition explicitly uses the role’s objectives and constraints as evaluation criteria, directing the judge to assess how well each candidate serves the active role’s decision objectives. Appendix C.8 and Table 16 describe the prompting and decoding protocols. The prompt structure for candidate-selection tasks is illustrated below:

System:
Evaluate the candidate according to the given role's mandate.
Rate how well it serves that role's objectives, rather than generic merit.

User:
[YOUR ROLE] <role name>
[YOUR OBJECTIVE] <objective>
[YOU CARE ABOUT] <focus>
[YOU WANT TO AVOID] <avoid>
[HARD CONSTRAINTS] <constraints>
<serialized task state and evidence>
<candidate content>
Return ONLY JSON {"quality": <0-100>}.

The core instruction explicitly states: “rate the step by how well it serves THAT mandate, not by generic merit.” The mandate therefore supplies both the evaluation criteria and the scoring instruction, rather than merely identifying a role.

Task typeRubric guidance and output
Candidate-selection tasksIndependently assign a score from 0 to 100 according to the supplied objectives and constraints; select the highest-scoring candidate.
Retail / SupportUse a role-specific rubric and task evidence or policies, perform a brief evidence check, and output good / acceptable / flawed / unacceptable, mapped to 1 / 0.66 / 0.33 / 0.

The blind and mandate conditions share the same state and candidate inputs, differing in the inclusion of the mandate and the corresponding evaluation instructions. Each candidate is scored independently at temperature 0. For selection tasks, ties are resolved by taking the first candidate in a pre-shuffled order; Retail / Support pairwise evaluation assigns half credit to ties. This setup evaluates how effectively a directly prompted LLM selects candidates when explicitly provided with the role’s objectives and constraints.

Reply to Q3: Joint Training versus Routed Experts under Matched Role Coverage

Thank you for this suggestion. In the original Table 4, the joint model was trained on 32 roles, while the routed experts covered 10 roles; both were evaluated on the same 10 roles. To separate cross-role sharing from additional role coverage, we trained new joint models using only those same 10 roles.

The joint model uses 11,694 training records, matching the union of the ten experts’ training data record by record, with no data from the additional 22 roles. Both approaches use the same backbone and training recipe. At test time, the experts receive oracle routing based on the true role. Results on the same 505 test states, measured by role-macro-averaged native utility, are:

ModelTraining rolesSelected utilityImprovement over routed experts [95% CI]
Routed experts100.532—
Joint10, seed 1100.865+0.333 [0.266, 0.400]
Joint10, seed 2100.871+0.338 [0.280, 0.402]

Both joint models outperform the corresponding expert on all ten roles, with minimum per-role gains of 0.150 and 0.183, respectively. After extending expert training to a maximum of 30 epochs and selecting checkpoints by validation performance, the joint models retain gains of +0.334 and +0.444.

Thus, the joint model’s advantage persists when training and evaluation role coverage are matched, supporting the benefit of cross-role sharing within the same role set without relying on training data from the additional 22 roles. The matched quantity is the aggregate training-data union: the joint model learns from these records together, while the experts learn separately from their respective roles.

We chose the reviewer-suggested matched 10-role comparison, for which a complete set of role-specific experts was already available. Each of the remaining 22 roles has only 54–102 training records, so we did not extend the experiment to 32 experts. The matched comparison above directly tests whether the joint model’s advantage depends on additional role coverage.

Dear Reviewer FAgD,

Thank you for your thoughtful review. We hope our new human validation, backbone scaling experiments, mandate robustness analyses, and matched-role comparison of joint training versus routed experts address your concerns. If so, we would appreciate your consideration of a higher score. We are happy to answer any further questions.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.