- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how to allocate counterfactual training data when a structural causal model contains an imperfect learned mechanism. CG-PPO combines known causal mechanisms with a Model of the Other Agent trained through response-equivalence supervision. It uses consecutive factual response correctness to select rollout starting states through CCW and CAPW. Experiments in an invader–defender environment examine reliability transfer after counterfactual branching, policy-learning benefits, interactions with factual updates, and exogenous-noise reuse under model misspecification.
Critical strengths
-
The finding that factual response correctness predicts correctness after counterfactual branching is interesting and directly relevant to allocating synthetic training data. Section 8.2 provides empirical support for this relationship beyond the shared initial state.
-
The experiments include matched comparisons across multiple training seeds and a controlled analysis of factual policy updates. In mixed learning, CAPW improves the mean win rate over the final six evaluations from 57.0% to 63.0% relative to Uniform, providing evidence of practical value in the tested setting.
Critical weaknesses
-
The choice of starting-state selection baselines needs further justification. I am less familiar with this specific literature and would appreciate clarification on whether recent studies provide alternative starting-state selection strategies for counterfactual or model-based rollout generation. Beyond the proposed CCW and CAPW, Uniform is the only evaluated selector baseline. Comparisons with applicable recent methods, or a discussion of why they are unsuitable here, would help assess the added value of the proposed weighting.
-
The contribution of CAPW’s censor-aware treatment is not isolated. Compared with CCW, CAPW changes both censoring treatment and the weighting function, including for uncensored candidates. The comparison therefore does not distinguish the benefit of censor handling from that of replacing capped linear weights with percentile weights based on uncapped durations. Section 8.5 establishes an interaction with factual policy updates, and Section 9 suggests broader branch coverage as a possible explanation, but the specific role of censor awareness in these gains remains unclear.
Questions for authors
-
Are there recent starting-state selection methods applicable to this setting? How do their selection rules relate to CCW/CAPW, and could they provide additional experimental comparisons?
-
What evidence distinguishes the contribution of CAPW’s censor-aware treatment from that of the accompanying changes to its weighting function?
-
Section 9 hypothesizes that factual updates may reduce sensitivity to imperfect counterfactual data, potentially making broader branch coverage useful. Can the authors clarify how this interpretation relates specifically to CAPW’s censor-aware treatment, as opposed to its changed weighting function?