- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper analyses sigmoid smoothing of contact activation in first-order pathwise gradients. Near a single regular switching surface, it expresses the gradient of the expected return as the expected within-branch gradient plus a boundary term that hard branch differentiation misses. Under a smooth gated-return representation and endpoint-matching conditions, the expected sigmoid gate gradient converges to this boundary term as the gate becomes sharp. With additional regularity, the paper derives finite-sharpness bias and second-moment expansions and the resulting tradeoff between sharpness and sample size. Numerical experiments in simple contact configurations, including a curved switching surface with two-dimensional noise, validate the predicted boundary terms, asymptotic orders, and coefficients.
Critical strengths
- Careful mathematical analysis with clearly stated scope. The paper explicitly identifies the boundary contribution omitted by hard pathwise differentiation and states sufficient conditions under which sigmoid gating recovers it in expectation. It distinguishes gate-term consistency from full-gradient consistency and does not claim optimisation guarantees. These distinctions make the analysis useful for understanding what contact smoothing can and cannot justify.
- Detailed numerical validation. The experiments check the boundary-term values, asymptotic bias and second-moment orders, and expansion coefficients. They also examine cancellation of the leading bias term and a curved switching surface where both bulk and boundary contributions are present. This supports the internal consistency of the analysis.
Critical weaknesses
- The incremental mathematical contribution relative to directly relevant prior work is insufficiently established. Shekhovtsov (2021) already analyses asymptotic unbiasedness, generic O(1/k) bias, O(1/k^2) bias for a linear downstream function, and O(k) variance growth for binary Gumbel-Softmax, writing k for inverse temperature. Its proof also changes variables to the activation, recovers the endpoint difference, and expands the density to obtain bias and second-moment coefficients. Wagner et al. (2024) studies sigmoid smoothing of discontinuous programs and establishes gradient consistency and second-moment bounds under its assumptions, together with an optimisation procedure. Neither work is discussed in the current manuscript. The present paper has differences in its setting, including local switching-surface geometry and its contact interpretation, but does not establish which differences yield substantive new results, technical advances, or applicability beyond prior work. The amount of mathematical derivation alone does not resolve this concern. [1, 2]
- The additional insight contributed to contact simulation is insufficiently demonstrated. With direct precedents for the core smoothing mechanisms, the paper needs to clarify what its adaptation establishes specifically for contact simulation: for example, a previously unresolved applicability question, a new distinction between successful and unsuccessful smoothing schemes, guidance that changes training or simulator design, or a previously overlooked contact phenomenon. The current experiments mainly confirm the mathematical predictions in simple configurations. They do not yet establish such an additional contribution. I do not require a large robotics benchmark or a new training algorithm; a precise and consequential contact-specific conclusion would be sufficient evidence of significance.
[1] Alexander Shekhovtsov. Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators. 2021. [2] Dominik Wagner, Basim Khajwal, C.-H. Luke Ong. Diagonalisation SGD: Fast & Convergent SGD for Non-Differentiable Models via Reparameterisation and Smoothing. AISTATS 2024.
Questions for authors
- Comparison with prior theory: Please compare Theorem 1 and Proposition 3 with Shekhovtsov (2021), particularly Propositions 1–2 and the expansions in Appendix A, and with the gradient-consistency and second-moment results of Wagner et al. (2024). Which conclusions have direct precedents, which are substantive extensions, and which changes in assumptions or technical arguments are essential to those extensions?
- Contact-specific contribution: What concrete question in contact simulation is resolved or substantially clarified by this analysis beyond existing smoothing theory? Can you identify a conclusion that changes our understanding of an existing method, its applicability, or its design? In particular, after downstream dynamics are represented by Lambda, what role does the contact structure play beyond that of a general nonlinear downstream function?
- Demonstrated applicability: The gated representation and endpoint matching are modelling assumptions, and the higher-order co-area regularity required for the finite-sharpness expansions is additional. For which contact-simulation configurations beyond the single effective gate examples are these conditions established? An explicit instance showing what the theory newly explains, or where its applicability fails, would help assess the significance of the claimed extension.
2 comments
I thank the authors for the candid response and for revising the contribution statement to credit Shekhovtsov (2021) and Wagner et al. (2024).
The response, however, confirms rather than resolves my central concern. The authors state that "the mechanism is Shekhovtsov's" and that "once the dynamics are represented by Λ, the contact structure plays no further role in the proofs." The remaining contribution is the extension of the scalar argument to a switching surface in multi-dimensional noise via the co-area formula, with explicit constants; for affine Λ, the surface-integral case is already covered by Liu & Hong (2011). This is a technical generalization of known results, not a new mechanism. The response also does not identify a new insight specific to contact simulation, and it provides neither a task-level experiment nor a multi-contact example. Proposition 5 is a relevant addition, but its scalar case has the straight-through Gumbel-Softmax precedent, as the authors note.
I also note for the discussion that the revision no longer presents returns nonlinear in the activation as a new case. This was one of the main strengths identified in other reviews.
The mathematical analysis remains rigorous, but its significance for contact simulation is not established. I therefore maintain my rating.
Thank you for these two references. We did not know them, and they change what we can claim. The revised PDF now cites both and no longer presents returns nonlinear in the activation as the new case; these changes are marked in orange (Sections 1 and 3).
W1, Q1. Direct precedents. Shekhovtsov (2021) treats the binary Gumbel-Softmax estimator: scalar logistic noise and a differentiable function f of the relaxed variable. With κ=1/τ, Λ=f, g=θ+ξ and η=−θ, this is our gate term with scalar logistic noise. His Propositions 1 and 2 give the conclusions of our Theorem 1, Proposition 2 and the bias orders of Proposition 3 for that case, by the same change of variables and the same expansion of the density. His result on the straight-through variant is the scalar precedent of our hard-forward Proposition 5. Wagner et al. (2024) prove gradient consistency and a second-moment bound for sigmoid-smoothed programs in the reparameterisation setting. This is a precedent for our Corollary 2 and for the O(κ) bound of Proposition 2. Their work includes nested conditionals, which we exclude (L1), and an optimisation result that goes beyond our Remark 2.
What remains, to our reading. (i) For a return nonlinear in the activation on a switching surface in multi-dimensional noise, we identify the limit of the gate term with the boundary surface integral (Theorem 1). Shekhovtsov has this for scalar noise; Wagner et al. prove convergence of the full gradient. (ii) In the same setting we give the finite-κ expansions with explicit constants (Proposition 3). Shekhovtsov has them for scalar noise; we found no rate in Wagner et al. The technical change is the co-area formula in place of the scalar substitution. The mechanism is Shekhovtsov's. Our noise assumption is narrower than in both papers: compact support (L7).
W2, Q2. Once the dynamics are represented by Λ, the contact structure plays no further role in the proofs. It fixes the inputs: the gap is a function of the state, so in general the surface is curved, the parameters move it non-uniformly, and the noise that decides contact also enters the return. For contact simulation our statements are limited to one regular contact event. Hard first-order gradients miss exactly the boundary term (Corollary 1). Simulating the smoothed model recovers it as κ→∞ (Theorem 1). A hard forward pass with a smoothed backward pass is guaranteed to recover it only for returns at most quadratic in the activation (Proposition 5). We give no rule for choosing κ and have no task-level experiment (L2).
Q3. The conditions are established only for a single effective gate: one gated Moreau step, where they hold exactly, and a 20-step unroll with one contact event, where the other gates change the return by at most 0.08% at κ=200 (measured, not proved). The theory does not apply when smoothing an earlier contact changes a later contact event (L1), and we have no such example.