- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies the stability of the best-response (BR) correspondence along the trajectories of continuous-time FTRL with time-varying learning rates in a specific family of two-player games. For entropic FTRL (Hedge), it establishes a threshold above which one player's BR eventually stabilizes to a unique action and below which it switches infinitely often for almost every initialization. The authors also obtain a similar threshold for Tsallis FTRL and a sufficient stability condition for Euclidean FTRL, in a more restricted setting.
Critical strengths
The main result is interesting and technically nontrivial. I particularly appreciate the sharp threshold for best-response stabilization and the use of energy techniques and volume preservation in its proof.
Critical weaknesses
First, I am not entirely convinced that eventual stabilization of instantaneous best responses is an important property of learning dynamics given that the phenomenon studied occurs near a non-strict Nash equilibrium where payoff differences vanish. Thus, even when best responses change infinitely often, the gains from deviation converge to zero and the players' strategies converge to a Nash equilibrium. This leaves me rather perplexed and I do not think the importance of this question is sufficiently motivated, so I find the game-theoretic significance of these best-response changes somewhat unclear. Note also that the payoff gap results partly address this concern, but the implications remain limited to this very specific setting, which brings me to my second point.
Indeed, I find the results rather too specialized. The main theorem concerns a specific family of games with a very particular payoff structure and, much more importantly, the switching best response relies crucially on the time-varying learning rates. Indeed, in the two-action game, infinitely many best response changes are impossible under constant learning rates since player B's cumulative payoff difference is monotone so player A's instantaneous payoff difference is monotone as well and its best response cannot change more than once. I find this particularly important because autonomous continuous-time FTRL is the standard and most extensively studied model in the literature and stochastic approximation theory shows that these autonomous dynamics arise as the mean-field limit of standard discrete-time learning algorithms, including Hedge with diminishing step sizes (see Mertikopoulos et al., 2024, "A Unified Stochastic Approximation Framework for Learning in Games"). Hence I would have liked a stronger justification for focusing on these particular nonautonomous dynamics, as well as evidence that the phenomenon extends beyond the specific games considered.
Finally, I found the exposition somewhat dense in places, making the main text and the analysis harder to follow than necessary.
Questions for authors
- Could you elaborate on why eventual stabilization of instantaneous best responses is important from a game-theoretic or learning perspective, given that strategies converge to Nash equilibrium and gains from deviation vanish even when best responses keep switching ?
- What motivates your particular formulation of time-varying FTRL, given that autonomous FTRL also arises as the mean-field limit of discrete-time learning algorithms with diminishing step sizes ?
- What is the motivation for studying this specific family of games beyond illustrating the possibility of infinitely many best response changes ? Do you expect the switching to extend to more general games ?
- Do you expect the sharp threshold established in Theorem 3 to extend to the discrete-time Hedge algorithm ? Is there a way to establish this rigorously, beyond the numerical evidence provided ?