- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies when test-time adaptation is beneficial in piecewise-stationary constrained Markov decision processes. It analyzes how detection delay and false alarms affect surrogate dynamic regret, and evaluates detector-coupled adaptation on synthetic, financial, and provisioning tasks.
Critical strengths
The paper separates adaptation timing from update frequency through matched-budget controls. It also reports negative financial results, constraint-violation trade-offs, and the limits of its theoretical guarantees.
Critical weaknesses
- The main theorem applies to a convex last-layer predictor updated by projected mirror descent, whereas the implemented method uses LoRA and AdamW. The policy-value result requires additional sufficiency assumptions, and the cost bound concerns expected per-step cost rather than realized drawdown or cumulative violations.
- The principal positive real-data result in Table 3 compares an exogenous calendar schedule with CUSUM; it does not establish broad superiority of a learned detector. Violations also increase on the traffic and electricity tasks. Although the appendix reports an approximately 1.1% relative regret improvement for the trained policy on concentrated traffic, the practical evidence remains limited.
- The abstract describes the lower bound as “matching” and suggests that the detector-related terms are unavoidable. Theorem 5 establishes a delay lower bound and a generic surrogate-regret lower bound, but not a lower bound matching the false-alarm term. This claim should be narrowed.
Questions for authors
- Can the authors validate the theoretical predictions on a practical task satisfying the theorem’s assumptions, or further characterize the gap between the theorem and the LoRA/AdamW implementation?
- Can an online learned detector be compared with the calendar schedule and CUSUM under matched update budgets, reporting both regret and constraint violations?