- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies how uniform initialization can create excessively large initial positive regrets in CFR-based algorithms. Initial Regret Reduction rescales the regret vector and corresponding prediction term once, when positive regret first appears at an information set, and then resumes standard updates. The paper establishes an average-regret rate and an interpretation as a one-time OMD step-size adjustment. Experiments cover 16 games, reporting favorable average-strategy results on non-HUNL benchmarks while acknowledging stronger DCFR performance on HUNL.
Critical strengths
-
The modification is simple and addresses a clearly illustrated mechanism. The motivating examples show how initial regret magnitudes can delay differentiation among effective actions. Applying the intervention once per information set preserves the regret vector's direction and avoids substantial recurring overhead.
-
The theoretical treatment and evaluation acknowledge important distinctions. The proofs explicitly account for the scaled regret and prediction terms. The experiments report both average-strategy and last-iterate behavior and include settings where the proposed method is not strongest.
Critical weaknesses
-
The presentation makes the central mechanism unnecessarily difficult to understand. The reader must connect several layers of notation covering regret matching, predictive updates, averaging, and the OMD interpretation. The paper does not provide a sufficiently clear, unified explanation of how these components relate to the one-time intervention. Figure 1 illustrates motivating examples and Figure 2 presents results, but neither provides a mechanism diagram connecting the original regret vector, the interpolation baseline, the scale factor, and the subsequent update. Despite the numerical examples, this substantially increases the effort required to assess the method as a whole.
-
The main average-strategy comparison does not isolate the contribution of IRR. The paper explicitly uses for PCFR+ and for IRR-PCFR+. Although the averaging exponent does not affect the last iterate, it directly changes the average strategy used in the principal comparison. Consequently, the reported improvement combines the intervention with a different averaging scheme. The last-iterate results provide complementary evidence but do not resolve this attribution problem.
-
The benefit of the specific adaptive scaling rule remains insufficiently established. The theoretical guarantees apply to general one-time scaling factors , rather than depending on the particular norm ratio in Equation (3). For a two-action information set with initial regret , the proposed rule reduces to the fixed coefficient . Without a fixed-scaling baseline, the experiments do not establish whether the adaptive construction contributes beyond the simpler intervention of shrinking the initial regrets.
-
Parameter selection and evaluation on the same benchmarks limit the robustness of the empirical conclusions. The authors disclose that and were selected through benchmark sweeps and then frozen. However, the submission provides neither sensitivity results nor evaluation on games excluded from parameter selection. Several competing methods retain their original configurations. The evidence therefore does not adequately distinguish a robust algorithmic improvement from an advantage associated with the selected configuration.
Questions for authors
See weaknesses.
Thank you for the careful reading and constructive feedback. We ran all requested ablations across the 16-game suite (matched γ and fixed versus adaptive one-time scaling) and added the analyses and full trajectory curves in Appendix D of the revised PDF (pages 22 to 27: text on pages 22 to 24; Figure 3 on page 25 for matched γ and Figure 4 on page 26 for fixed versus adaptive scaling).
W1 (Presentation and mechanism diagram). We appreciate the feedback on readability. Because Eq. (3) is an empirical heuristic rather than a principled derivation, we chose not to add a mechanism diagram that would make it look more theoretical than it is, and instead added the empirical scaling ablation in Appendix D.2 (page 23) and Figure 4 (page 26).
W2 (Matched γ). We re-ran all 16 games × {IRR-PCFR+, PCFR+, PDCFR+} × γ ∈ {2, 5, 9} (144 runs; full trajectory curves in Figure 3, page 25, discussed in Appendix D.1, page 23). Because curves eventually stop decreasing, fluctuate near the bottom of the axis, or hit floating-point zero, we count both terminals at or below 1e-13 as a tie (Appendix D, page 22); all reported wins are by at least 10×, so none of the counts depend on this cutoff. At matched γ = 9 against PCFR+, IRR-PCFR+ wins 6 games (from 10.6× on SmallMatrix (1) to 1.8e13× on SmallMatrix (4)), ties 8, and loses the two HUNL subgames (1.44× and 1.56×). Against PDCFR+ at γ = 9, it wins 3 (up to 5.8e6×), ties 10, and loses 3 (Liar's dice (5) 8.6×, Subgame (3) 3.4×, Subgame (4) 2.4×). The large Kuhn gap under unmatched exponents disappears once γ is matched. Also, PDCFR+ is not monotone in γ: at γ = 9 it finishes 7.5e4× worse than PCFR+ on Goofspiel-imp (5).
W3 (Fixed scaling vs adaptive rule). Your two-action reduction to ≈1/2 holds, and the data match your observation on near-symmetric games (matrix games and HUNL), where fixed and adaptive scaling behave similarly or s = 1 is slightly better. Adaptivity helps when initial regret scales differ widely across actions. We tested fixed one-time scales s ∈ {1, 0.5, 0.1, 1e-3, 1e-5} (s = 1 is PCFR+; full trajectory curves in Figure 4, page 26, discussed in Appendix D.2, page 23). Every fixed constant fails by 5 to 13 orders of magnitude on at least one game (s = 1 and 0.5 on SmallMatrix (4); s = 0.1 and 1e-3 on Goofspiel (5); s = 1e-5 on Liar's dice (5)), while the adaptive rule stays within 3.9× of the per-game best constant on all 16 games and beats all five constants on Goofspiel (4) and Goofspiel-imp (5).
W4 (Parameter sweep and benchmark protocol). We swept λ ∈ {0.1, 0.3, 0.5, 0.7, 0.9, 0.99, 0.999, 0.9999, 0.99999} and γ ∈ {2, 5, 9} across all 16 games, minimizing terminal average-strategy exploitability. Competitors are evaluated both under their published defaults (main text) and at matched γ ∈ {2, 5, 9} across all 16 games (Appendix D.1, page 23; Figure 3, page 25).