- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces HICO-Bench, a benchmark of 180 paired coordination scenarios that checks whether two agents reach a shared, feasible plan supported by their dialogue. It also proposes PaCPO, which compares tentative plans across the two versions of each scenario to assign rewards across dialogue turns.
Critical strengths
-
Each scenario has two versions that differ in one private condition and require different valid plans. This design provides a clear test of whether an agent uses the relevant information to adjust its response.
-
PaCPO uses tentative plans to provide feedback at individual dialogue turns while keeping the total reward unchanged. This offers a practical way to train from final outcomes without manually scoring every utterance.
Critical weaknesses
-
PaCPO sees 96 training scenarios, while the group baselines see 48, and its extra plan probes and disclosure checks add training cost. Matching dialogue counts alone does not separate the method's benefit from differences in data coverage and total cost.
-
The ablations do not directly compare PaCPO with using only the intermediate feasibility term in Eq. 6. A control with a comparable reward scale would help show whether paired comparisons add value beyond this simpler signal.
-
PaCPO and GRPO have the same Feasibility score, so their gap in Verified Success depends on the Integrity check. Appendix K reports 90% agreement between two judges, but their pass rates differ by ten percentage points and the effect on method ranking is not reported.
-
HICO scenarios are adapted from SOTOPIA, but the paper does not explain whether HICO training and the SOTOPIA transfer test share source scenarios. This makes it hard to judge how far the transfer goes beyond situations already used in training.
-
The training study uses Qwen3-8B, and the main HICO evaluation also uses Qwen3-8B as the partner. The additional partner settings are useful, but they do not establish whether the HICO gains hold with a different partner model.
Questions for authors
-
Does PaCPO remain ahead when all methods see the same training scenarios?
-
How does PaCPO compare with keeping the intermediate feasibility term while removing the paired term?
-
On the shared outputs in Appendix K, what is each method's Verified Success under each of the two judges? Does the ordering of PPO, GRPO, and PaCPO stay the same?
-
Are the source scenarios used to build HICO training separate from the fourteen SOTOPIA environments used for the transfer test?
-
How much of the HICO gain depends on using Qwen3-8B as the partner?
4 comments
Thank you for the detailed replies and for running the new experiments. I still have a few concerns about the new results.
1. The gain over the control is small
The authors compare the full method with a control that keeps only the within world term. The gap between them is a few tasks out of the full test set, and the same reply reports seed variation that is larger than this gap. No interval is given for this comparison, so the data do not show that the paired term helps.
| Condition | Verified Success |
|---|---|
| Full method, average of two runs | 26.04 |
| Control, average of two runs | 25.21 |
| Gap | 0.83 |
2. One setting, two values
The reply to Reviewer iATJ reports a lower value at zero paired weight in the weight sweep. The reply to me reports a higher value at the same zero weight. Both replies say the other settings are held fixed. The two values differ by more than the paper claims for the paired term itself. The authors should say which run each value comes from, and if the runs differ, they should report both under one setting.
| Setting | Verified Success |
|---|---|
| Zero paired weight, weight sweep | 23.33 |
| Zero paired weight, control comparison | 25.21 |
3. Baselines drop when they get more data
The new run gives every method the same scenarios, and the baselines also get more training dialogues than in the original run reported in paper Table 3. More data should not make a method worse. Several baselines do get worse, and one stays exactly the same. This run therefore does not show a stronger method. It shows baselines that moved down while the proposed method kept its old score.
| Method | Original run | New run |
|---|---|---|
| GRPO | 26.25 | 25.00 |
| Dr.GRPO | 21.25 | 22.50 |
| GiGPO | 24.58 | 22.92 |
| GraphGPO | 24.17 | 24.17 |
| Training dialogues | 384 for every method | 768 for baselines, 384 for PaCPO |
I would be glad to update my score if these points are addressed.
1. Clarifying the β = 0 results and the contribution of paired credit
Thank you for noting the discrepancy. At α = 0.25 and β = 0, the first two additional runs scored 27.08% and 23.33% on Verified Success. Thus, 23.33% is a single-run result and 25.21% is the two-run mean. We will clarify both passages.
We suspected that using different machines might have contributed to the variation. To investigate this and address your concern about training randomness, we repeated the comparison on one machine with four independent training seeds. Both methods used the same plan probes and within-world feasibility term (α = 0.25).
| Run | Full PaCPO | β = 0 control | Difference (pp) |
|---|---|---|---|
| Seed 1 | 24.58% | 23.33% | +1.25 |
| Seed 2 | 23.33% | 23.75% | −0.42 |
| Seed 3 | 25.83% | 22.92% | +2.92 |
| Seed 4 | 25.00% | 21.25% | +3.75 |
| Mean | 24.69% | 22.81% | +1.88 |
Full PaCPO scored higher on three of four seeds, with 237 successes in 960 evaluations versus 219 for the control. The mean gain was 1.88 percentage points, with a 95% bootstrap CI of [+0.42, +3.33] percentage points across training seeds.
We used the six task families from Figure 3 and Section 5.3. Each family comprised 10 test scenarios and 40 tasks per run, for 160 evaluations across four runs.
| Task family | Full PaCPO | β = 0 control | Difference (pp) |
|---|---|---|---|
| Set-based consent | 23.75% | 25.00% | −1.25 |
| Joint scheduling | 8.75% | 8.75% | 0.00 |
| Resource allocation | 21.25% | 14.38% | +6.88 |
| Shared-resource scheduling | 4.38% | 4.38% | 0.00 |
| Authorization | 45.63% | 45.00% | +0.63 |
| Bounded quantities | 44.38% | 39.38% | +5.00 |
| All six families | 24.69% | 22.81% | +1.88 |
Resource allocation and bounded quantities gained most (+6.88 and +5.00 percentage points). In resource allocation, Integrity passes rose from 23 to 34, all of which also passed Feasibility. In one anti-bullying campaign task, the participants had to divide 12 poster panels within respective limits of 3–4 and 3–9. PaCPO agreed on a 4/8 split and passed both checks. The control submitted a feasible 3/9 split but failed Integrity after incorrectly claiming that 8 was outside the 3–9 range and then repeating itself.
Set-based consent had two fewer successes (−1.25 percentage points). The failures involved tracking all requirements and knowing when to end the dialogue. In one task, the organizer requested mime, funny faces, and a group photo, while the other participant ruled out the photo. Both methods submitted the same correct agreement, but PaCPO failed Integrity because its dialogue became repetitive.
Overall, paired credit yielded 18 additional verified successes, mainly in resource allocation and bounded quantities. The consent example highlights an opportunity to improve dialogue closure once agents reach a valid agreement.
2. How did additional training affect the baselines?
Thank you for raising this point. PaCPO in Table 3 already covered all 96 training scenarios, so in our previous response we reused its checkpoint and score and trained the four group-based baselines (G = 4) on the same 96 scenarios. As in response for question 1, we suspected that using different machines might have contributed to the variation. We therefore retrained and evaluated PaCPO under the same settings.
| Method | Coverage-matched Verified Success |
|---|---|
| PaCPO | 27.50% |
| GRPO | 25.00% |
| Dr.GRPO | 22.50% |
| GiGPO | 22.92% |
| GraphGPO | 24.17% |
With training coverage matched, the retrained PaCPO achieved 27.50% Verified Success, outperforming all four baselines by at least 2.50 percentage points (GRPO, 25.00%). PaCPO scored 0.83 percentage points below its Table 3 result (28.33%), narrowing its lead over GRPO from 3.33 to 2.50 percentage points. The ranking of all five methods remained the same.
We hope these additional analyses address your questions. Thank you again for examining our results so carefully!
Thank you for the detailed replies and for the additional runs. I still have a few concerns about the new results.
1. The reported result is above every run reported later
The paper reports one number for the full method. The latest reply reports four new seeds, and none of them reaches that number. Their average is well below it. The paper value therefore looks like the best of several runs rather than a typical one.
| Quantity | Verified Success |
|---|---|
| Paper value | 28.33 |
| Average of four new seeds | 24.69 |
| Best of four new seeds | 25.83 |
2. The spread between runs is larger than the gain claimed
Both methods move across a wide range in the runs reported so far, and the gain in the new study is smaller than that range. An interval built from the spread across the four seeds includes zero, so the new study does not separate the two methods.
| Quantity | Verified Success |
|---|---|
| Lowest full method run | 23.33 |
| Highest full method run | 28.33 |
| Lowest control run | 21.25 |
| Highest control run | 27.08 |
| Gain claimed | 1.88 |
3. The machine explanation does not keep the same size
One reply puts the size of the machine effect below the gain claimed. Another puts it above. If a change of machine can move a result by more than the gain itself, then a single run per method in the main table cannot support the comparison it makes.
| Comparison | First | Second | Difference |
|---|---|---|---|
| Paper value and retrained value | 28.33 | 27.50 | 0.83 |
| Mean of three seeds and mean of four seeds | 26.81 | 24.69 | 2.12 |
Thank you for highlighting the value of paired scenarios and PaCPO’s use of tentative plans for turn-level credit. Your questions about paired credit, source overlap, and dialogue partners helped us clarify when PaCPO’s gains hold.
1. Does PaCPO still lead with the same training scenarios?
We trained PaCPO and all four group-based baselines (G = 4) on the same 96 source scenarios with the same training seed and evaluated all five methods on the same 240 test tasks. PaCPO sampled 384 training dialogues and achieved 28.33% Verified Success. Each baseline sampled 768 training dialogues. Verified Success was 25.00% for GRPO, 22.50% for Dr.GRPO, 22.92% for GiGPO, and 24.17% for GraphGPO. With half as many training dialogues as each baseline, PaCPO outperformed the strongest baseline, GRPO, by 3.33 percentage points.
2. What does the paired term in Eq. (6) add?
We retained the plan probes and within-world feasibility term and set β = 0 to remove paired credit. For both the control and full PaCPO, we tested the default α = 0.25 and an alternative α ≈ 0.3348 calibrated to the reward scale on the training data. With the same development set and tuning budget, both selected α = 0.25. The two values tied for the control, and our prespecified rule selected the smaller one. With only β differing, full PaCPO (β = 0.10) scored higher in both runs. Mean test Verified Success was 26.04% for full PaCPO and 25.21% for the feasibility-only control. These results suggest that cross-world comparisons improve Verified Success beyond what within-world feasibility shaping alone achieves.
3. Does the ranking change with the Integrity judge?
We extended the comparison in Appendix K to all 240 test tasks per method. Qwen3.8-Max and Kimi-K3 evaluated the same dialogue outputs using the same rule-based Feasibility decisions. Each judge assessed whether the dialogue supported the final plan (Integrity). The table reports Verified Success.
Both judges ranked the methods PaCPO > GRPO > PPO > Base. Under Kimi-K3, PaCPO led PPO by 2.50 percentage points and GRPO by 2.08.
4. Do the SOTOPIA transfer environments overlap with HICO?
Matching original SOTOPIA environment IDs revealed one overlap between the 14 transfer environments and HICO’s 96 training sources,
free_stuff. During benchmark construction, we screened 884 source records by source ID, scenario family, and alias. We found no prior project use offree_stuffat the time and selected it as one of the 103 sources for new scenarios. Although we rewrote the training task, we counted it as an overlap because it shares the original environment ID.Excluding
free_stuffleft 13 environments and 65 environment–character pairs. With GPT-4o as the partner, Goal scores (out of 10) were 7.03 for PaCPO, 6.35 for PPO, and 6.82 for GRPO. In self-play, the corresponding scores were 6.71, 6.17, and 6.58.Checking all 180 HICO sources revealed one further overlap with the HICO test set. Excluding both environments left 12 environments and 60 environment–character pairs with no source ID shared with HICO. On this subset, PaCPO scored 7.07 with GPT-4o as its partner, compared with 6.35 for PPO and 6.85 for GRPO. We will report the overlap and both filtered evaluations in the revision.
We also evaluated the method with task-specific training. On MultiWOZ, 81.17% of PaCPO’s recommendations satisfied all user constraints (PPO, 80.52%). On CaSiNo, PaCPO reached agreement in 63.95% of negotiations (PPO, 60.47%).
5. How much does the HICO gain depend on the Qwen3-8B partner?
We evaluated the original PaCPO, PPO, and GRPO checkpoints with three additional partners. All combinations used the same 240 HICO test tasks and the same Qwen3.8-Max evaluation protocol. The table reports Verified Success.
PaCPO scored higher than both baselines with each additional partner. With DeepSeek V4.1 Flash 1017, its margins were 6.67 percentage points over PPO and 4.17 over GRPO.