acceptodds
Under review as a conference paper at ICLR 2027

Variance-Aware Fine-Grained Gap-Dependent Bounds for Online Reinforcement Learning

Abstract

We study model-free online reinforcement learning (RL) for episodic tabular Markov decision processes, focusing on both gap-dependent regret and policy switching cost. While fine-grained gap-dependent analysis has been established for model-free RL algorithms using Hoeffding-type exploration bonuses, such results for model-free algorithms with variance-based exploration bonuses remain unknown, despite their superior worst-case and coarse-grained gap-dependent guarantees. In this paper, we resolve this open problem by establishing the first fine-grained gap-dependent regret upper bound for UCB-Bernstein+, a refined UCB-Bernstein algorithm, in variance-aware model-free online RL. Moreover, by integrating a stage-wise policy update design into our fine-grained framework and using refined variance-based bonuses, we achieve the best-known gap-dependent local switching cost to date. In addition, our analysis yields improved worst-case guarantees for both regret and local switching cost over the original UCB-Bernstein algorithm. Numerical experiments further demonstrate that UCB-Bernstein+ achieves favorable empirical performance in both regret and local switching cost.

Primary area:
reinforcement learning
Keywords:
Suboptimality Gap, Regret, Policy Switching Cost

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_48TvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies model-free online RL for episodic tabular MDPs. It proposes UCB-Bernstein+, which makes two changes to UCB-Bernstein. First, it adds the stage-wise lazy update of UCB-Advantage (Zhang et al., 2020). Second, it computes the Bernstein bonus from stage-wise samples only and caps it with a Hoeffding term. The main result (Theorem 3.1) is a fine-grained gap-dependent regret bound. Its per-triple term is (Qh⋆+β2)H3/Δh(s,a)(\mathbb{Q}^\star_h + \beta^2)H^3/\Delta_h(s,a). Here Qh⋆\mathbb{Q}^\star_h is the maximum conditional variance at step hh, and β\beta is an analysis parameter. The paper also gives a gap-dependent local switching cost bound (Theorem 3.2). It further gives a worst-case regret of O(H3SATι+H9/2SAι2)O(\sqrt{H^3SAT\iota} + H^{9/2}SA\iota^2) (Theorem 3.3) and a worst-case local switching cost of O(H2SAlog⁡K)O(H^2SA\log K) (Theorem 3.4). The key tool is a pseudo value function that clips VkV^k to [V⋆,V⋆+β][V^\star, V^\star+\beta]. Experiments on random MDPs and GridWorld show lower regret and lower switching cost than the baselines.

Critical strengths

The question is natural. Fine-grained gap-dependent bounds previously existed only for Hoeffding-type model-free methods, and the paper clearly explains the technical obstacle in extending them to variance-based bonuses. The pseudo value function is a clean tool. It does not change the algorithm, it supports both the gap-dependent and the worst-case analyses, and it removes the superlinear S,AS, A term of the original UCB-Bernstein. The proofs are careful and track constants explicitly. The key steps, including the optimism argument in Lemma G.1, the variance decomposition in Eqs. (55)–(60), the recursion in Lemma H.3 and the switching-cost reduction in Appendix K, appear correct. In addition, the paper attains the best known worst-case local switching cost O(H2SAlog⁡K)O(H^2SA\log K) with a Bernstein-based algorithm.

Critical weaknesses

W1. With Qh⋆≤H2\mathbb{Q}^\star_h \le H^2 and β=H\beta = H, the bound has the same HH-dependence as the Hoeffding analysis of Zhang et al. (2026a), as the authors acknowledge. Bernstein bonuses usually help through the law of total variance, but this analysis does not use it. The reason is that Qh⋆\mathbb{Q}^\star_h is a per-step maximum over all (s,a)(s,a). A single high-variance pair at step hh removes any gain at that step. The variance factor is therefore not fine-grained, and the title and abstract overstate the result.

W2. The bound is O(log⁡2T)O(\log^2 T), not O(log⁡T)O(\log T). The last term of Theorem 3.1 is H6SAlog⁡2(SAT)/βH^6SA\log^2(SAT)/\beta. Since β≤H\beta \le H, this term is at least H5SAlog⁡2(SAT)H^5SA\log^2(SAT) for any choice of β\beta, and it dominates the 1/Δ1/\Delta terms as TT grows. Section 5 says the flat Regret/log⁡(K+1)\mathrm{Regret}/\log(K+1) curves are consistent with Theorem 3.1, which is inaccurate. Please state the TT-dependence explicitly and discuss whether O(log⁡T)O(\log T) is achievable.

W3. All synthetic MDPs have transitions drawn uniformly from the simplex, with one instance per scale. These instances are neither low-variance nor gap-controlled, so they say nothing about variance-awareness or gap dependence. Instances with controlled Qh⋆\mathbb{Q}^\star_h and Δmin⁡\Delta_{\min} are needed.

W4. The experiments use ι=1\iota = 1, C1=C3=2C_1 = C_3 = \sqrt{2} and C2=1C_2 = 1. The theory requires C1=8C_1 = 8, C2=119C_2 = 119, C3=2C_3 = 2 and ι=log⁡(2SAT/δ)\iota = \log(2SAT/\delta). The tested algorithm is therefore not the analyzed one. Please state this and report at least one run with the theoretical constants.

W5. There are also a few minor presentation problems: Lemma I.1 is central to Theorem 3.2 but appears only in the last appendix. It should be moved to the main text. Theorem 3.2 is stated after setting β=H\beta = H, which hides its variance-aware form. In the introduction, ∣Z∣|Z| is defined differently for Xu et al. and Zhang et al. within the same display. Theorem 3.1 also requires knowing TT in advance, through δ=1/(8T)\delta = 1/(8T). The introduction says it is open whether UCB-Bernstein admits a gap-dependent bound. The paper analyzes a modified algorithm, so this question remains open.

[1] Haochen Zhang, Zhong Zheng, and Lingzhou Xue. Q-learning with fine-grained gap-dependent regret. In International Conference on Learning Representations (ICLR), 2026.

Questions for authors

Q1. Can Qh⋆\mathbb{Q}^\star_h be replaced by a per-triple variance or by a total-variance term? If not, where does the argument fail?

Q2. How does the coarse-grained version of Theorem 3.1 compare with Zheng et al. (2025b) in HH, SS, AA and log⁡T\log T?

[1] Zhong Zheng, Haochen Zhang, and Lingzhou Xue. Gap-dependent bounds for Q-learning using reference-advantage decomposition. In International Conference on Learning Representations (ICLR), 2025. (Zheng et al., 2025b; cited in Q2)

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part I)

We thank you for your thoughtful comments. Below are our responses.

W1 and Q1: Stepwise maximum variance

The step-wise maximum conditional variance is not essential to our proof. In fact, our original analysis first obtains a bound in terms of the triple-specific conditional variance Vs,a,h(Vh+1⋆),\mathbb{V}_{s,a,h}(V_{h+1}^{\star}), and only subsequently upper bounds it by Qh⋆=max⁡s,aVs,a,h(Vh+1⋆)\mathbb{Q}_h^\star=\max_{s,a}\mathbb{V}_{s,a,h}(V_{h+1}^{\star}) to simplify the presentation.

Motivated by the reviewer’s comment, we have strengthened Theorem 3.1 in the uploaded revision by retaining the triple-specific conditional variances throughout the analysis. In particular, the suboptimal-triple contribution now depends on ∑h=1H∑Δh(s,a)>0(Vs,a,h(Vh+1⋆)+β2)H3ι1Δh(s,a),\sum_{h=1}^H\sum_{\Delta_h(s,a)>0}\frac{\big(\mathbb{V}_{s,a,h}(V_{h+1}^{\star})+\beta^2\big)H^3\iota_1}{\Delta_h(s,a)}, rather than using the step-wise maximum variance for every state-action pair. Similarly, the contribution from optimal state-action pairs is refined from a term involving (Qt⋆+β2)∣Zopt,t∣(\mathbb{Q}_t^\star+\beta^2)|Z_{\mathrm{opt},t}| to one involving ∑(s,a)∈Zopt,t(Vs,a,t(Vt+1⋆)+β2).\sum_{(s,a)\in Z_{\mathrm{opt},t}}\big(\mathbb{V}_{s,a,t}(V_{t+1}^{\star})+\beta^2\big). Thus, a single high-variance state-action pair no longer forces all pairs at the same step to pay the corresponding worst-case variance cost. This directly addresses the instance-adaptivity concern raised by the reviewer and gives an affirmative answer to Q1 through the per-triple variance refinement.

This refinement requires no change to UCB-Bernstein+ or to the main proof technique; it simply avoids the step-wise worst-case relaxation that was used in the previous presentation. The original step-wise bound follows immediately as a simpler corollary by upper bounding each local variance by Qh⋆\mathbb{Q}_h^\star. The strengthened result and its corresponding proof are provided in the updated version.

W2: Dependence on log⁡T\log T

By “approximately logarithmic” in Section 5, our intended meaning was that the empirical regret exhibits very slow, polylogarithmic-type growth in TT, rather than polynomial growth. We did not intend this phrase to claim that Theorem 3.1 establishes an exact O(log⁡T)O(\log T) regret rate. To make this intended meaning explicit, in the revised version we replace “suggesting approximately logarithmic cumulative regret growth” with “suggesting polylogarithmic-type cumulative regret growth.”

More precisely, the first two terms in Theorem 3.1 scale with ι1=log⁡(SAT)\iota_1=\log(SAT), while the residual term is H6SAι12β\frac{H^6SA\iota_1^2}{\beta}. Thus, for fixed problem-dependent quantities, Theorem 3.1 has a polylogarithmic dependence on TT, with an O(log⁡2T)O(\log^2 T) lower-order residual term. The relatively flat Regret(T)/log⁡(K+1)\mathrm{Regret}(T)/\log(K+1) curves in Section 5 were intended to illustrate this slow empirical growth, rather than to distinguish between an exact O(log⁡T)O(\log T) and O(log⁡2T)O(\log^2 T) dependence over the finite range of KK considered in the experiments.

The O(log⁡2T)O(\log^2 T) dependence originates from the lower-order O(ι/n)O(\iota/n) term that commonly appears in variance-aware exploration bonuses, including those used in our algorithm and in [1,2,3,4]. The precise meaning of nn depends on the algorithm: in our algorithm and [1], nn is the number of visits collected in the preceding stage, while in non-stage-wise variance-aware algorithms [2,3,4] it typically denotes the cumulative number of visits to the corresponding state-action-step triple. In either case, the confidence factor ι\iota itself contains a logarithmic dependence on TT.

To bound regret, one needs to accumulate the exploration bonuses over the entire learning process. Consequently, the lower-order O(ι/n)O(\iota/n) component is also accumulated over all relevant updates and visits. This accumulation contributes another logarithmic dependence on TT on top of the logarithmic dependence already contained in the confidence factor ι\iota, resulting in the squared logarithmic dependence. In our analysis, this effect is captured by Lemma E.2 with α=1\alpha=1, which yields Equation (78):

\lesssim \frac{H^4SA\iota^2}{\beta}.$$ This contribution is subsequently incorporated into Equation (76) and ultimately gives rise to the $\iota^2$-dependent residual term in Theorem 3.1. Hence, the squared logarithmic dependence in the residual term is a natural consequence of accumulating the $O(\iota/n)$ component of the variance-aware exploration bonus throughout the learning process: one logarithmic dependence is contained in the confidence factor $\iota$, while the other arises from controlling the cumulative contribution of the inverse-visit-count term, as formalized by Lemma E.2 and Equation (78). Reducing the resulting $O(\log^2 T)$ dependence to $O(\log T)$ would require a sharper way of controlling this lower-order bonus contribution, and whether such a refinement is possible remains an open question.
AuthorsOpenReview ↗

Rebuttal by Authors (Part II)

W3: Multiple-MDP Robustness and Controlled Gap/Variance Experiments

We thank the reviewer for this comment. In the revised version, we address this concern in two complementary ways.

First, in Appendix B.3, we repeat the main-algorithm and state-of-the-art baseline comparisons over five independently generated MDPs for each experimental scale, with 10 independent runs on each MDP. The results in Figures 5--7 remain consistent with the original experiments, supporting that the empirical advantages of UCB-Bernstein+ are not specific to a single randomly generated MDP.

Second, in Appendix B.4, we introduce a controlled 3×33\times3 family of synthetic MDPs in which the suboptimality-gap structure and the conditional variance Vs,a,h(Vh+1⋆)\mathbb V_{s,a,h}(V_{h+1}^\star) are varied systematically. For each fixed variance level, the transition kernel and the complete conditional-variance table remain unchanged while the entire positive gap table increases coordinate-wise. Conversely, for each fixed gap level, the complete gap table remains unchanged while the conditional-variance table increases coordinate-wise. These properties are verified exactly using dynamic programming.

The corresponding results are shown in Figures 8--13. UCB-Bernstein+ maintains favorable regret and switching-cost performance across all nine controlled settings. Moreover, Figures 12--13 directly isolate the two effects: at a fixed gap table, regret increases with the conditional variance, while at a fixed transition kernel and variance table, regret decreases as the gap structure becomes more favorable, qualitatively consistent with Theorem 3.1.

Further details of the multiple-MDP robustness and controlled gap/variance experiments are provided in Appendices B.3 and B.4 of the revised paper.

W4: Theoretical coefficients

We thank the reviewer for pointing out this distinction. We agree that the parameterization used in our main experiments is a practical version of UCB-Bernstein+, rather than the parameterization used in the theoretical analysis. We have clarified this distinction in Section 5 of the revised paper.

The constants in the theoretical bonus are chosen to ensure the concentration and optimism properties needed for the high-probability analysis. They are consequently conservative for the finite interaction budgets considered in our experiments. In particular, the theoretical bonus contains both a variance-dependent term with coefficient C1=8C_1=8 and a lower-order term with coefficient C2=119C_2=119, as well as a Hoeffding-type cap with coefficient C3=2C_3=2. Together with the logarithmic factor ι\iota, these terms can remain large during the early stages of learning. Since the QQ-estimates are initialized optimistically and updated by taking a minimum with the new estimate, a large bonus can delay decreases in the QQ-values and, in turn, changes in the greedy policy.

For illustration, consider (H,S,A,K)=(5,3,2,3×104)(H,S,A,K)=(5,3,2,3\times10^4) and an outer failure probability δ=0.1\delta=0.1. Under the high-probability setting in our analysis, ι=log⁡(16SAT/δ)≈18.79\iota=\log(16SAT/\delta)\approx18.79. The theoretical Hoeffding-type cap is then 2Hι/n2H\sqrt{\iota/n}, which is at least HH when the current stage count satisfies n≤4ιn\leq 4\iota. Over this range, the lower-order term 119Hι/n119H\iota/n is also larger than HH. Hence, the resulting optimistic bonus can be large relative to the horizon and can delay informative value and policy updates.

At the finite interaction budgets considered in our experiments, directly using these theoretical constants would therefore leave a substantial initial portion of learning under an extremely conservative optimistic bonus, making such a performance comparison largely uninformative about the practical behavior of the algorithm. We therefore explicitly distinguish the theoretical and practical parameterizations and provide the calculation above to quantify the conservativeness of the theoretical choice.

For a reasonably fair empirical comparison across algorithms, our main experiments therefore use the same coefficient 2\sqrt{2} for the leading 1/n1/\sqrt{n} exploration terms and coefficient 11 for the lower-order 1/n1/n terms whenever applicable. This parameterization is intended to compare the practical behavior of the algorithms under comparable bonus scales, rather than to reproduce the conservative constants used in the theoretical analysis. Similar practical choices are made in [4,5]; moreover, some baseline analyses specify bonus coefficients only as sufficiently large universal constants.

We have also added a bonus-coefficient sensitivity experiment in the appendix. It varies the leading exploration coefficients over a common range while keeping the lower-order terms fixed, and the comparison remains consistent across these choices.

(continued in the next part)

AuthorsOpenReview ↗

Rebuttal by Authors (Part III)

W5: Presentation and Clarifications.

Thank you for pointing out these presentation issues. We have revised the manuscript accordingly.

First, we agree that Lemma I.1 is important for understanding the gap-dependent local switching-cost result. We have therefore moved its variance-aware statement to the main text immediately before Theorem 3.2, while retaining its detailed proof in the appendix.

Second, we have revised the presentation of Theorem 3.2 so that its variance-aware form is stated explicitly before taking the worst-case relaxation. In particular, the bound now retains its dependence on Qh⋆\mathbb{Q}_h^\star and the analysis parameter β\beta. The previous bound is then presented as a simplified consequence obtained by using Qh⋆≤H2\mathbb{Q}_h^\star\leq H^2 and setting β=H\beta=H. This makes the variance adaptivity underlying the switching-cost result explicit.

Third, we have corrected the notation in the introduction. Previously, the same symbol ZZ was used for different notions of optimal triples in the bounds of Xu et al. and Zhang et al. We now use separate notation for the two quantities and state their definitions explicitly.

Fourth, we have clarified the dependence on the known time horizon. In our theoretical guarantees, the confidence parameter is chosen as a function of TT (e.g., δ=1/(8T)\delta=1/(8T) in Theorem 3.1), and hence the theoretical parameter setting assumes that TT is known in advance. We now state this explicitly when introducing ι=log⁡(2SAT/δ)\iota=\log(2SAT/\delta). This is separate from our experimental implementation, where ι\iota is treated as a positive tuning parameter.

Finally, UCB-Bernstein is one of the simplest model-free algorithms using variance-based exploration bonuses, yet a fine-grained gap-dependent analysis for model-free algorithms with such bonuses was previously unavailable. This gap is an important motivation for our work. Starting from UCB-Bernstein+, a refined version of UCB-Bernstein that incorporates a stage-wise update design and a refined variance-based bonus, we introduce pseudo value functions to address the main technical difficulty arising from the state-action-dependent variance-based bonus structure and establish a fine-grained gap-dependent analysis. Although our current theorem is established for UCB-Bernstein+ rather than the original UCB-Bernstein, the pseudo-value-function technique is designed to handle the variance-aware bonus structure itself and may therefore be useful for analyzing variance-aware bonuses more broadly, potentially including the corresponding gap-dependent analysis of the original UCB-Bernstein. We have revised the introduction by removing the potentially ambiguous statement about the original UCB-Bernstein and clarifying that our contribution addresses the question of fine-grained gap-dependent analysis with variance-based bonuses.

Q2: Coarse-grained comparison

Our revised Theorem 3.1 contains two distinct levels of instance adaptivity that are absent from their coarse-grained bound.

First, the revised theorem retains the triple-specific conditional variances Vs,a,h(Vh+1⋆),\mathbb{V}_{s,a,h}(V_{h+1}^{\star}), rather than replacing them by the global maximum conditional variance Q⋆:=max⁡s,a,hVs,a,h(Vh+1⋆).\mathbb{Q}^{\star}:=\max_{s,a,h}\mathbb{V}_{s,a,h}(V_{h+1}^{\star}). Second, our result is fine-grained in the suboptimality gaps: it depends on the individual Δh(s,a)\Delta_h(s,a), the step-dependent Δmin⁡,h\Delta_{\min,h}, and the structure of the optimal sets Zopt,hZ_{\mathrm{opt},h}, rather than only on the global minimum gap Δmin⁡\Delta_{\min}. Therefore, if either the local variance structure or the fine-grained gap structure is favorable, Theorem 3.1 can be substantially smaller than its fully coarse-grained relaxation. In particular, a few high-variance triples do not force all triples to pay the global worst-case variance, and a few small-gap triples do not force all suboptimal triples to pay 1/Δmin⁡1/\Delta_{\min}.

To obtain a direct coarse-grained comparison, we now remove both sources of instance adaptivity. Specifically, let Vs,a,h(Vh+1⋆)≤Q⋆\mathbb{V}_{s,a,h}(V_{h+1}^{\star})\leq \mathbb{Q}^{\star} for all (s,a,h)(s,a,h), replace all positive gaps by Δmin⁡\Delta_{\min}, use Δmin⁡,h≥Δmin⁡\Delta_{\min,h}\geq\Delta_{\min}, and upper bound ∣Zopt,h∣≤SA|Z_{\mathrm{opt},h}|\leq SA. For the suboptimal-triple term, this gives

O\left(\frac{(\mathbb{Q}^{\star}+\beta^2)H^4SA}{\Delta_{\min}}\right).$$ Similarly, the optimal-pair contribution satisfies $$\sum_{h=1}^H\frac{H\left(\sum_{t=h+1}^H\sqrt{\sum_{(s,a)\in Z_{\mathrm{opt},t}}(\mathbb{V}_{s,a,t}(V_{t+1}^{\star})+\beta^2)}\right)^2}{\Delta_{\min,h}} \leq O\left(\frac{(\mathbb{Q}^{\star}+\beta^2)H^4SA}{\Delta_{\min}}\right).$$ Hence, the fully coarse-grained version of our Theorem 3.1 is $$O\left(\frac{(\mathbb{Q}^{\star}+\beta^2)H^4SA\log(SAT)}{\Delta_{\min}}+\frac{H^6SA\log^2(SAT)}{\beta}\right).$$ (continued in the next part)
Reviewer_FREAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies gap-dependent regret and local policy switching costs in episodic tabular model-free RL. The best fine-grained gap-dependent bounds suffer from high polynomial dependence on the horizon. This leads the question of improving fine-grained gap-dependent regret bound. The authors propose UCB-Bernstein+; in which the main theoretical contribution is a fine-grained gap-dependent regret bound that incorporates both individual suboptimality gaps and transition-dependent variances, which improve over previous Hoeffding-based results. The paper also establishes improved gap-dependent local switching guarantees and worst-case regret and switching bound relative to original UCB-Bernstein algorithm.

Critical strengths

I think the strongest part of this paper is that it addresses an interesting technical problem: extending fine-grained gap-dependent regret analysis to model-free RL algorithms using variance-based exploration bonuses. This is a meaningful gap in the literature, and I appreciate the authors' use of pseudo-value functions to handle the additional difficulty introduced by variance-dependent bonuses. Theorem 3.1 is particularly interesting because it provides improved inverse-gap dependence in deterministic or low-variance settings.

I also liked that the authors looked at local policy switching costs alongside standard regret. Theorems 3.2–3.4 manage to provide improved gap-dependent switching guarantees without sacrificing the best-known worst-case order for model-free RL. Achieving this with a fairly straightforward stage-wise Q-learning design is a solid contribution.

Critical weaknesses

My main concerns are about two aspects of the paper: the strength of the theoretical improvements and the empirical validation provided.

For the first point, although I find Theorem 3.1 interesting, I am not entirely convinced on how broadly it improves over existing results. Specifically, the bound relies on the maximum conditional variance across state-action pairs at each step, which seems to really limit its instance adaptivity. It also picks up additional terms involving β\beta, including H6SA/βH^6SA/\beta, and contributions from optimal state-action pairs. Under the worst-case variance regime, the leading gap-dependent terms recover the same horizon dependence as previous Hoeffding-based analyses, while the worst-case regret bound in Theorem 3.3 is still worse than the best-known model-free rate by approximately a factor of H\sqrt{H}. Because of this, it's hard to tell when this bound actually offers a major step up for general stochastic MDPs, outside of the specific low-variance settings highlighted in the text.

For the second point, I do not think the experiment as is highlight enough the primary advantage of the proposed variance-aware approach. The synthetic experiments rely on just one randomly generated MDP per configuration across 10 independent trials, lacking a systematic variation of the suboptimality gaps or transition variances, despite these being central to the theoretical contribution. There seems to also be lack of comments into the sensitivity of exploration-bonus parameters. A more controlled evaluation across different variance and gap regimes would strengthen the empirical claims.

Questions for authors

My questions are as follow:

  1. Can the authors clarify the specific regimes where Theorem 3.1 yields a strict improvement over existing fine-grained Hoeffding-based bounds, especially once you factor in the H6SA/βH^6SA/\beta term and the optimal state-action pairs contributions? Also, is relying on the step-wise maximum conditional variance absolutely necessary for the proof, or is there a path to making the bound more instance-adaptive?

  2. Can the authors provide more empirical evidence showing that these regret and switching-cost improvements actually hold up across multiple independently generated MDPs when varying the suboptimality gaps and transition variances? It would also be great to see if these gains hold steady when tweaking the exploration-bonus parameters and properly tuning the baselines.

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part I)

We thank you for your thoughtful comments. Below are our point-by-point responses.

W1.1 and Q1.2: Stepwise maximum variance

We thank the reviewer for pointing this out. The step-wise maximum conditional variance is not essential to our proof. In fact, our original analysis first obtains a bound in terms of the triple-specific conditional variance Vs,a,h(Vh+1⋆),\mathbb{V}_{s,a,h}(V_{h+1}^{\star}), and only subsequently upper bounds it by Qh⋆=max⁡s,aVs,a,h(Vh+1⋆)\mathbb{Q}_h^\star=\max_{s,a}\\{\mathbb{V}_{s,a,h}(V_{h+1}^{\star})\\} to simplify the presentation.

Motivated by the reviewer’s comment, we have strengthened Theorem 3.1 in the uploaded revision by retaining the triple-specific conditional variances throughout the analysis. In particular, the suboptimal-triple contribution now depends on ∑h=1H∑Δh(s,a)>0(Vs,a,h(Vh+1⋆)+β2)H3ι1Δh(s,a),\sum_{h=1}^H\sum_{\Delta_h(s,a)>0}\frac{\big(\mathbb{V}_{s,a,h}(V_{h+1}^{\star})+\beta^2\big)H^3\iota_1}{\Delta_h(s,a)}, rather than using the step-wise maximum variance for every state-action pair. Similarly, the contribution from optimal state-action pairs is refined from a term involving (Qt⋆+β2)∣Zopt,t∣(\mathbb{Q}_t^\star+\beta^2)|Z_{\mathrm{opt},t}| to one involving ∑(s,a)∈Zopt,t(Vs,a,t(Vt+1⋆)+β2).\sum_{(s,a)\in Z_{\mathrm{opt},t}}\big(\mathbb{V}_{s,a,t}(V_{t+1}^{\star})+\beta^2\big). Thus, a single high-variance state-action pair no longer forces all pairs at the same step to pay the corresponding worst-case variance cost. This refinement requires no change to UCB-Bernstein+ or to the main proof technique; it simply avoids the step-wise worst-case relaxation that was used in the previous presentation. The original step-wise bound follows immediately as a simpler corollary by upper bounding each local variance by Qh⋆\mathbb{Q}_h^\star. The strengthened result and its corresponding proof are provided in the updated version.

W1.2 and Q1.1: Regimes where Theorem 3.1 improves over fine-grained Hoeffding-type bounds

We thank the reviewer for raising this question. We first note that the strengthened Theorem 3.1 in our uploaded revision now retains the triple-specific conditional variances Vs,a,h(Vh+1⋆)\mathbb{V}_{s,a,h}(V_{h+1}^{\star}), instead of relying on the step-wise maximum conditional variance. For a transparent comparison with the existing Hoeffding-type fine-grained bound, in the discussion below we further upper bound these triple-specific variances by Qh⋆:=max⁡s,aVs,a,h(Vh+1⋆)\mathbb Q_h^\star:=\max_{s,a}\mathbb{V}_{s,a,h}(V_{h+1}^{\star}). This relaxation is used only for the following regime interpretation; the revised theorem itself is more instance-adaptive and can be strictly sharper than this step-wise relaxation.

The Hoeffding-type bound in Zhang et al. (2026a) is O~(∑h=1H∑Δh(s,a)>0H5Δh(s,a)+∑h=1HH3(∑t=h+1H∣Zopt,t∣)2Δmin⁡,h+H3SA).\tilde{O}\left(\sum_{h=1}^H\sum_{\Delta_h(s,a)>0}\frac{H^5}{\Delta_h(s,a)}+\sum_{h=1}^H\frac{H^3(\sum_{t=h+1}^H\sqrt{|Z_{\text{opt},t}|})^2}{\Delta_{\min,h}}+H^3SA\right). Let DD denote the first two terms in the bound above. Then the bound can be written as O~(D+H3SA)\tilde O(D+H^3SA).

After upper bounding the triple-specific variances in our strengthened theorem by Qh⋆\mathbb Q_h^\star, our bound in Theorem 3.1 can be equivalently written as I:=O~(∑h=1H∑Δh(s,a)>0Qh⋆H3Δh(s,a)+∑h=1HH(∑t=h+1HQt⋆∣Zopt,t∣)2Δmin⁡,h+inf⁡β∈(0,H](β2DH2+H6SAβ)).I:=\tilde{O}\left(\sum_{h=1}^H\sum_{\Delta_h(s,a)>0}\frac{\mathbb{Q}_h^\star H^3}{\Delta_h(s,a)}+\sum_{h=1}^H\frac{H\left(\sum_{t=h+1}^H\sqrt{\mathbb{Q}_t^\star|Z_{\text{opt},t}|}\right)^2}{\Delta_{\min,h}}+\inf_{\beta\in (0,H]}\left(\frac{\beta^2D}{H^2}+\frac{H^6SA}{\beta}\right)\right). Let JJ denote the first two terms in the bound above. Since Qh⋆≤H2\mathbb Q_h^\star\le H^2, we know J≤DJ\le D.

We emphasize that this comparison already includes the contributions from optimal state-action pairs. More explicitly, let D=Dsub+DoptD=D_{\text{sub}}+D_{\text{opt}} and J=Jsub+JoptJ=J_{\text{sub}}+J_{\text{opt}}, where DoptD_{\text{opt}} and JoptJ_{\text{opt}} denote the second terms in the Hoeffding-type and Bernstein-type bounds above, respectively. Define q:=max⁡h∈[H]Qh⋆q:=\max_{h\in[H]}\mathbb Q_h^\star. Then Jsub≤qH2Dsub,Jopt≤qH2Dopt,J_{\text{sub}}\le\frac{q}{H^2}D_{\text{sub}},\qquad J_{\text{opt}}\le\frac{q}{H^2}D_{\text{opt}}, and therefore J≤qH2D.J\le\frac{q}{H^2}D. Hence, the contribution from optimal state-action pairs is not an additional deterioration relative to the corresponding Hoeffding-type term. Rather, it is its variance-aware counterpart and enjoys the same variance-dependent improvement.

The remaining additional terms in Theorem 3.1 are the β\beta-dependent terms. Importantly, the quantity β2DH2+H6SAβ\frac{\beta^2D}{H^2}+\frac{H^6SA}{\beta} already incorporates the β2\beta^2 contributions from both the suboptimal-triple and optimal-state-action terms, as well as the residual term H6SA/βH^6SA/\beta. Thus, the following optimization jointly accounts for all of these additional terms. The minimizer is \beta^\star=\min\left\\{H,\left(\frac{H^8SA}{2D}\right)^{1/3}\right\\}.

(continued in the next part)

AuthorsOpenReview ↗

Rebuttal by Authors (Part II)

If D≥12H5SAD\ge\frac{1}{2}H^5SA, which is satisfied in a common situation where there exists Ω(SH)\Omega(SH) state-step pairs (s,h)(s,h) such that each has only O(1)O(1) optimal actions, including the case that the optimal action is unique at every (s,h)(s,h), then β⋆=(H8SA2D)1/3,I=O~(J+H103D13(SA)23)≤O~(D).\beta^\star=\left(\frac{H^8SA}{2D}\right)^{1/3},\qquad I=\tilde{O}\left(J+H^\frac{10}{3}D^\frac{1}{3}(SA)^\frac{2}{3}\right)\le\tilde{O}(D). Thus, even after accounting for the β\beta-dependent residual term and the optimal-state-action contribution, our bound is no worse than the leading terms of the Hoeffding-type bound in this regime. We next characterize when the improvement is strict.

Suppose the HH-dependence of DD scales as HαH^\alpha with α≥5\alpha\ge 5, while omitting the dependencies on SS, AA, the gaps, the optimal-set sizes, and logarithmic factors.

Deterministic MDPs: q=0q=0. In this case, J=0J=0, and hence I=O~(H103D13(SA)23).I=\tilde{O}\left(H^\frac{10}{3}D^\frac{1}{3}(SA)^\frac{2}{3}\right). Since DD depends linearly on the aggregated inverse gaps, the sublinear dependence D1/3D^{1/3} of our Bernstein-type bound provides a substantial improvement when the aggregated inverse gaps are large, for example when some suboptimality gaps are small. For the horizon dependence, the Hoeffding-type bound scales as HαH^\alpha, whereas our Bernstein-type bound scales as H(α+10)/3H^{(\alpha+10)/3}. Therefore, the improvement in the exponent of HH is 2(α−5)3\frac{2(\alpha-5)}{3}. The improvement becomes larger as the HH dependence of DD increases. Similar improvements occur in the dependence on SS or AA when the corresponding dependence in DD becomes superlinear.

Low-variance MDPs: q=O(1)q=O(1). In this case, I≤O~(D/H2+H103D13(SA)23).I\le \tilde{O}\left(D/H^2+H^\frac{10}{3}D^\frac{1}{3}(SA)^\frac{2}{3}\right). When 5≤α≤85\le\alpha\le 8, the upper bound scales as H(α+10)/3H^{(\alpha+10)/3}, yielding an improvement of 2(α−5)3\frac{2(\alpha-5)}{3} in the exponent of HH compared to the Hoeffding-type bound. When α≥8\alpha\ge 8, the bound scales as Hα−2H^{\alpha-2}, yielding an improvement by a factor of H2H^2.

Moderate-variance MDPs: q=O(H)q=O(H). In this regime, I≤O~(D/H+H103D13(SA)23).I\le\tilde{O}\left(D/H+H^\frac{10}{3}D^\frac{1}{3}(SA)^\frac{2}{3}\right). When 5≤α≤6.55\le\alpha\le 6.5, the upper bound scales as H(α+10)/3H^{(\alpha+10)/3}, yielding an improvement of 2(α−5)3\frac{2(\alpha-5)}{3} in the exponent of HH compared to the Hoeffding-type bound. When α≥6.5\alpha\ge 6.5, the bound scales as Hα−1H^{\alpha-1}, yielding an improvement by a factor of HH.

High-variance MDPs: q=Θ(H2)q=\Theta(H^2). Under this coarse step-wise relaxation, no strict improvement in the horizon dependence over the Hoeffding-type bound can be guaranteed based on qq alone, and the leading gap-dependent terms recover the same horizon dependence as the Hoeffding-type analysis in Zhang et al. (2026a). This is consistent with the worst-case-variance observation raised by the reviewer.

Importantly, however, this does not mean that the strengthened per-triple result cannot improve in such an MDP. Since the revised Theorem 3.1 depends directly on Vs,a,h(Vh+1⋆)\mathbb{V}_{s,a,h}(V_{h+1}^{\star}), a large variance at a small number of state-action pairs only affects those individual contributions. Therefore, even when q=Θ(H2)q=\Theta(H^2), the revised bound can still be strictly smaller than the Hoeffding-type bound if the high variance is concentrated on relatively few state-action-step triples or on triples that contribute little to the gap-dependent terms.

If D<12H5SAD<\frac{1}{2}H^5SA, then the constrained minimizer is β⋆=H\beta^\star=H, and the Bernstein-type analysis yields I=O~(H5SA).I=\tilde{O}(H^5SA). In this regime, the resulting guarantee II no longer exhibits an explicit dependence on the variance quantities, and the residual term has a worse dependence on HH than the Hoeffding-type bound O~(D+H3SA)\tilde{O}(D+H^3SA). However, UCB-Bernstein+ also includes the Hoeffding-type cap C3H2ιnC_3\sqrt{\frac{H^2\iota}{n}} in its exploration bonus. Therefore, instead of using the Bernstein-type branch in this regime, we can apply the existing Hoeffding-type fine-grained analysis together with the stage-wise update structure of UCB-Bernstein+. In particular, the Hoeffding cap depends only on the stage-wise visit count, and the effects of using preceding-stage samples and propagating estimation errors across stages are controlled by the same stage-wise visit-count and weighted-recursion arguments already established in our analysis. Following the standard Hoeffding-type fine-grained argument then yields the corresponding guarantee O~(D+H3SA).\tilde{O}(D+H^3SA). Consequently, although the Bernstein-type analysis alone provides no variance-aware improvement in this regime, the Hoeffding cap allows UCB-Bernstein+ to recover the same order as the existing Hoeffding-type fine-grained guarantee.

(continued in the next part)

AuthorsOpenReview ↗

Rebuttal by Authors (Part III)

W1.3: Horizon Dependence in Theorem 3.3.

We agree with the reviewer that the worst-case regret bound in Theorem 3.3 remains worse than the minimax-optimal model-free rate by a factor of H\sqrt{H}. Our goal in this work is to establish improved fine-grained gap-dependent regret guarantees for algorithms with variance-based exploration bonuses while simultaneously maintaining low switching cost. Accordingly, we build on UCB-Bernstein and develop the refined UCB-Bernstein+. The leading term O~(H3SAT)\tilde{O}(\sqrt{H^3SAT}) in our worst-case regret bound is inherited from the underlying UCB-Bernstein-type structure, rather than being an additional loss introduced by our fine-grained gap-dependent analysis or by the variance-aware refinement developed in this paper.

In particular, our contribution is orthogonal to the line of work targeting minimax-optimal worst-case regret. The latter typically requires additional algorithmic machinery, such as reference-advantage decomposition or variance-reduction techniques [1,2], which reduce the accumulation of estimation errors across the horizon and achieve the minimax rate O~(H2SAT)\tilde{O}(\sqrt{H^2SAT}). Incorporating such techniques into our fine-grained variance-aware framework is an interesting direction, but would require a substantially different algorithm and analysis.

We therefore do not claim that UCB-Bernstein+ is minimax-optimal in the worst case. Rather, Theorem 3.3 shows that the refinements introduced for fine-grained gap-dependent analysis and low switching cost preserve the O~(H3SAT)\tilde{O}(\sqrt{H^3SAT}) leading worst-case dependence of the UCB-Bernstein family while removing its superlinear dependence on SS and AA in the lower-order term. More importantly, the gap-dependent guarantee in Theorem 3.1 can be substantially sharper than Hoeffding-type fine-grained bounds in low- and moderate-variance instances, as discussed above. Thus, the remaining H\sqrt{H} gap in the worst-case regime does not preclude the instance-dependent improvements that are the main focus of this work.

[1] Zihan Zhang, Yuan Zhou, and Xiangyang Ji. "Almost optimal model-free reinforcement learning via reference-advantage decomposition." Advances in Neural Information Processing Systems 33 (2020): 15198-15207.

[2] Gen Li, Laixi Shi, Yuxin Chen, Yuantao Gu, and Yuejie Chi. "Breaking the sample complexity barrier to regret-optimal model-free reinforcement learning." Advances in Neural Information Processing Systems 34 (2021): 17762-17776.

(continued in the next part)

Reviewer_t4XeOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper extends fine-grained gap-dependent regret analysis from Hoeffding-type exploration bonuses to Bernstein-type bonuses, with a control on policy switching costs. The proposed algorithm, UCB-Bernstein+, combines a stage-wise update rule with a careful construction of empirical variance bonuses. This is an improvement to a previous ICLR paper Q-Learning with Fine-Grained Gap-Dependent Regret, Zhang et al.

Critical strengths
  1. I find the introduction of the pseudo value function an interesting technical tool. Conceptually, it seems related to the reference-value ideas in model-free RL.
  2. The proof seems technically sound for the part I find critical, though I have not checked every detail of the whole proof.
Critical weaknesses
  1. The variance dependence might be improvable. In particular, in the high variance regime Qh=Θ(H2)\mathbb Q_h = \Theta(H^2), the bound in Theorem 3.1 could be worse than Theorem 4.5 in Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learning, Zhang et al. I have not checked if there is some simple workaround for such worst-case though.
  2. The experiments are conducted on few MDP instances. If I understand correctly, for each synthetic configuration, the authors generate only one MDP and repeat the algorithms ten times. It would be more informative to evaluate across multiple independently generated MDPs, especially with varying variances and suboptimality gaps.
Questions for authors

My main concern is whether the dependence on Qh\mathbb Q_h can be improved, especially in the worst case.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part I)

We thank you for your thoughtful comments. Below are our point-by-point responses.

W1 and Q1: High-variance regime

We thank the reviewer for raising this comparison. The revised per-triple variance form of Theorem 3.1 also helps clarify the behavior in the high-variance regime. In particular, a large step-wise maximum variance alone does not force every state-action-step triple in our bound to incur the corresponding worst-case variance cost, since the revised theorem retains the local conditional variance Vs,a,h(Vh+1⋆)\mathbb{V}_{s,a,h}(V_{h+1}^{\star}) for each triple. Therefore, the most unfavorable high-variance regime arises when the relevant local conditional variances are themselves all of order Θ(H2)\Theta(H^2).

To compare this extreme case with Theorem 4.5 of Zhang et al., suppose that Vs,a,h(Vh+1⋆)=Θ(H2)\mathbb{V}_{s,a,h}(V_{h+1}^{\star})=\Theta(H^2) for all relevant triples. Define the fine-grained gap complexity

+\frac{1}{H^2}\sum_{h=1}^H\frac{\left(\sum_{t=h+1}^H\sqrt{|Z_{\mathrm{opt},t}|}\right)^2}{\Delta_{\min,h}}.$$ Then, up to logarithmic factors, the first two terms of our revised Theorem 3.1 become $(H^2+\beta^2)H^3\mathcal C_{\mathrm{fg}},$ and the full bound takes the form $$\widetilde O\left((H^2+\beta^2)H^3\mathcal C_{\mathrm{fg}}+\frac{H^6SA}{\beta}\right).$$ For comparison, Theorem 4.5 of Zhang et al., specialized to $M=1$ and $\mathbb Q^\star=\Theta(H^2)$, has the form $$\widetilde O\left(\frac{(H^2+\beta^2H)H^3SA}{\Delta_{\min}}+\frac{H^7SA}{\beta}\right).$$ Thus, the comparison depends not only on the variance level but also on how favorable the fine-grained gap structure is. To make this dependence explicit, define $$G_{\mathrm{sub}}:=\Delta_{\min}\sum_{h=1}^H\sum_{\Delta_h(s,a)>0}\frac{1}{\Delta_h(s,a)}$$ and $$G_{\mathrm{opt}}:=\frac{\Delta_{\min}}{H^2}\sum_{h=1}^H\frac{\left(\sum_{t=h+1}^H\sqrt{|Z_{\mathrm{opt},t}|}\right)^2}{\Delta_{\min,h}}.$$ Then $$\mathcal C_{\mathrm{fg}}=\frac{G_{\mathrm{sub}}+G_{\mathrm{opt}}}{\Delta_{\min}}.$$ Consequently, the $\beta$-independent high-variance contribution in our result is $$\widetilde O\left(\frac{H^5(G_{\mathrm{sub}}+G_{\mathrm{opt}})}{\Delta_{\min}}\right),$$ compared with $$\widetilde O\left(\frac{H^5SA}{\Delta_{\min}}\right)$$ in Zhang et al. Therefore, our leading high-variance term is smaller whenever $$G_{\mathrm{sub}}+G_{\mathrm{opt}}<SA.$$ This condition highlights the instance-dependent benefit captured by our fine-grained bound. The quantity $G_{\mathrm{sub}}$ can be substantially smaller than its worst-case value when only a limited number of suboptimal triples have gaps close to $\Delta_{\min}$ and most suboptimal gaps $\Delta_h(s,a)$ are considerably larger. Similarly, $G_{\mathrm{opt}}$ can be small when the number of optimal state-action pairs is limited and the step-dependent minimum gaps $\Delta_{\min,h}$ are substantially larger than the global minimum gap for most relevant steps. Therefore, our high-variance contribution can still improve over the coarse-grained dependence on $SA/\Delta_{\min}$ whenever the combined effective complexity $G_{\mathrm{sub}}+G_{\mathrm{opt}}$ is smaller than $SA$. More importantly, even if neither source of instance adaptivity is favorable, so that the relevant local variances are all $\Theta(H^2)$ and both the suboptimal-gap and optimal-pair structures attain their worst-case scaling, the degradation relative to the coarse-grained result remains limited. In general, $$G_{\mathrm{sub}}\leq HSA,\ G_{\mathrm{opt}}\leq O(HSA),$$ and hence $$G_{\mathrm{sub}}+G_{\mathrm{opt}}\leq O(HSA).$$ Therefore, even in this joint worst-case regime, the $\beta$-independent variance contribution in our theorem is at most $$\widetilde O\left(\frac{H^6SA}{\Delta_{\min}}\right),$$ which differs by only one additional factor of $H$ from the corresponding $$\widetilde O\left(\frac{H^5SA}{\Delta_{\min}}\right)$$ term in Zhang et al. Importantly, this worst-case factor-$H$ difference is confined to the $\beta$-independent variance term and does not apply to the entire regret bound. (continued in the next part)
AuthorsOpenReview ↗

Rebuttal by Authors (Part II)

Before comparing the β\beta-dependent terms, we emphasize that β\beta plays different roles in the two results. In our Theorem 3.1, β∈(0,H]\beta\in(0,H] is an auxiliary parameter introduced only in the theoretical analysis and does not appear in the UCB-Bernstein+ algorithm; accordingly, our final bound takes the infimum over β\beta. In contrast, β\beta in Zhang et al. is an algorithmic tuning parameter used by Q-EarlySettled-LowCost, and their guarantee is stated for the algorithm instantiated with a given β\beta. Therefore, the comparison below is between the corresponding terms for a common numerical value of β\beta, while our final guarantee can additionally optimize over β\beta at the analysis level without changing the algorithm.

For the β2\beta^2-dependent part, the same worst-case bound Gsub+Gopt≤O(HSA)G_{\mathrm{sub}}+G_{\mathrm{opt}}\leq O(HSA) gives O~(β2H4SAΔmin⁡),\widetilde O\left(\frac{\beta^2H^4SA}{\Delta_{\min}}\right), which is of the same order as the corresponding β2\beta^2-dependent term in Zhang et al. Moreover, our burn-in term is O~(H6SAβ),\widetilde O\left(\frac{H^6SA}{\beta}\right), which improves by a factor of HH over their O~(H7SAβ)\widetilde O\left(\frac{H^7SA}{\beta}\right) term.

Therefore, even in the high-variance regime, our fine-grained bound can be strictly smaller when the instance has sufficiently favorable gap and optimal-pair structure. In the least favorable regime, the possible deterioration is confined to an additional factor of HH in the β\beta-independent variance contribution, while the β\beta-dependent gap term remains of the same order and the burn-in term is improved by a factor of HH.

Further improving the β\beta-independent variance term in this extreme high-variance regime likely requires an additional variance-reduction mechanism, such as the reference-advantage decomposition used in Zhang et al. Our pseudo-value-function technique addresses a different technical difficulty: it enables us to control the non-uniform variance-based bonuses while preserving the weighted recursive structure required for fine-grained gap-dependent analysis. Combining the two techniques is nontrivial, since a variance-reduced analysis would need to simultaneously control the reference and advantage variance terms, their estimation errors, and their interaction with the fine-grained weighted recursion. We view developing such a variance-reduced fine-grained analysis as an interesting direction for future work.

W2: Robustness Across MDP Instances and Controlled Gap/Variance Experiments

We thank the reviewer for this suggestion. In the revised version, we substantially expand the empirical evaluation in two directions. First, in Appendix B.3, we repeat both the main-algorithm and state-of-the-art baseline comparisons over five independently generated MDPs for each experimental scale, with 10 independent runs per MDP. The results in Figures 5--7 remain consistent with the original experiments: UCB-Bernstein+ achieves the lowest regret among the four main algorithms and maintains favorable regret and switching-cost performance relative to the state-of-the-art baselines.

Second, in Appendix B.4, we construct a controlled 3×33\times3 family of MDPs that independently varies the suboptimality-gap structure and the conditional variance Vs,a,h(Vh+1⋆)\mathbb V_{s,a,h}(V_{h+1}^\star). At fixed variance, the complete gap table increases coordinate-wise, while at fixed gap, the complete gap table remains unchanged and the conditional-variance table increases coordinate-wise. Figures 8--13 show that UCB-Bernstein+ remains competitive across all nine settings, while its regret increases with variance and decreases as the gap structure becomes more favorable, qualitatively consistent with Theorem 3.1.

Further details of these experiments are provided in Appendices B.3 and B.4 of the revised paper.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer t4Xe,

As the discussion stage is approaching its end, we would like to kindly follow up on our response. We sincerely appreciate your positive evaluation and hope that our rebuttal has addressed your remaining concerns.

If any concerns remain, we would be happy to provide further clarification before the discussion period ends.

Thank you again for your time and thoughtful feedback.

Best regards,

Authors of Submission 9344

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.