acceptodds
Under review as a conference paper at ICLR 2027

Sigmoid Contact Smoothing Recovers a Local Boundary Term in First-Order Pathwise Gradients

Abstract

First-order pathwise gradients through rigid-contact simulation can be biased: the return of a rollout may jump when a contact event switches on or off, and differentiating a fixed rollout never sees that jump. We analyze one deterministic remedy used in recent differentiable locomotion pipelines, in which hard contact-mode selection is replaced by a sigmoid gate on the contact impulse, and identify precisely which part of the true gradient the gate supplies. Near a single regular switching surface in the space of rollout randomness, the gradient of the expected return equals the expectation of the pathwise gradient plus a boundary integral over that surface, weighted by the jump of the return. This decomposition is classical in form, a transport (Reynolds) identity; we give a rigorous local proof of it under mild hypotheses, from which it follows that hard pathwise differentiation misses exactly the boundary term. Our main result concerns the gate: whenever the smoothed return depends on the contact activation through a differentiable interpolation whose endpoints reproduce the two hard branches, the gate part of the smoothed pathwise gradient converges in expectation to the boundary term as the gate hardens. In this limit, analytic gate smoothing supplies exactly the term that hard differentiation discards. We complement this with a finite-sharpness theory: the second moment of the gate term grows at most linearly in the sharpness, and under one extra smoothness hypothesis its bias and second moment have explicit expansions whose orders are set by the readout and the noise density at the surface, giving a bias–variance scaling law in the batch size; we illustrate the results numerically on a single-contact model, including a two-dimensional-noise configuration.

Primary area:
reinforcement learning
Keywords:
differentiable simulation, contact dynamics, policy gradients, analytic smoothing, bias-variance tradeoff

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ReBwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper analyses sigmoid smoothing of contact activation in first-order pathwise gradients. Near a single regular switching surface, it expresses the gradient of the expected return as the expected within-branch gradient plus a boundary term that hard branch differentiation misses. Under a smooth gated-return representation and endpoint-matching conditions, the expected sigmoid gate gradient converges to this boundary term as the gate becomes sharp. With additional regularity, the paper derives finite-sharpness bias and second-moment expansions and the resulting tradeoff between sharpness and sample size. Numerical experiments in simple contact configurations, including a curved switching surface with two-dimensional noise, validate the predicted boundary terms, asymptotic orders, and coefficients.

Critical strengths
  1. Careful mathematical analysis with clearly stated scope. The paper explicitly identifies the boundary contribution omitted by hard pathwise differentiation and states sufficient conditions under which sigmoid gating recovers it in expectation. It distinguishes gate-term consistency from full-gradient consistency and does not claim optimisation guarantees. These distinctions make the analysis useful for understanding what contact smoothing can and cannot justify.
  2. Detailed numerical validation. The experiments check the boundary-term values, asymptotic bias and second-moment orders, and expansion coefficients. They also examine cancellation of the leading bias term and a curved switching surface where both bulk and boundary contributions are present. This supports the internal consistency of the analysis.
Critical weaknesses
  1. The incremental mathematical contribution relative to directly relevant prior work is insufficiently established. Shekhovtsov (2021) already analyses asymptotic unbiasedness, generic O(1/k) bias, O(1/k^2) bias for a linear downstream function, and O(k) variance growth for binary Gumbel-Softmax, writing k for inverse temperature. Its proof also changes variables to the activation, recovers the endpoint difference, and expands the density to obtain bias and second-moment coefficients. Wagner et al. (2024) studies sigmoid smoothing of discontinuous programs and establishes gradient consistency and second-moment bounds under its assumptions, together with an optimisation procedure. Neither work is discussed in the current manuscript. The present paper has differences in its setting, including local switching-surface geometry and its contact interpretation, but does not establish which differences yield substantive new results, technical advances, or applicability beyond prior work. The amount of mathematical derivation alone does not resolve this concern. [1, 2]
  2. The additional insight contributed to contact simulation is insufficiently demonstrated. With direct precedents for the core smoothing mechanisms, the paper needs to clarify what its adaptation establishes specifically for contact simulation: for example, a previously unresolved applicability question, a new distinction between successful and unsuccessful smoothing schemes, guidance that changes training or simulator design, or a previously overlooked contact phenomenon. The current experiments mainly confirm the mathematical predictions in simple configurations. They do not yet establish such an additional contribution. I do not require a large robotics benchmark or a new training algorithm; a precise and consequential contact-specific conclusion would be sufficient evidence of significance.

[1] Alexander Shekhovtsov. Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators. 2021. [2] Dominik Wagner, Basim Khajwal, C.-H. Luke Ong. Diagonalisation SGD: Fast & Convergent SGD for Non-Differentiable Models via Reparameterisation and Smoothing. AISTATS 2024.

Questions for authors
  1. Comparison with prior theory: Please compare Theorem 1 and Proposition 3 with Shekhovtsov (2021), particularly Propositions 1–2 and the expansions in Appendix A, and with the gradient-consistency and second-moment results of Wagner et al. (2024). Which conclusions have direct precedents, which are substantive extensions, and which changes in assumptions or technical arguments are essential to those extensions?
  2. Contact-specific contribution: What concrete question in contact simulation is resolved or substantially clarified by this analysis beyond existing smoothing theory? Can you identify a conclusion that changes our understanding of an existing method, its applicability, or its design? In particular, after downstream dynamics are represented by Lambda, what role does the contact structure play beyond that of a general nonlinear downstream function?
  3. Demonstrated applicability: The gated representation and endpoint matching are modelling assumptions, and the higher-order co-area regularity required for the finite-sharpness expansions is additional. For which contact-simulation configurations beyond the single effective gate examples are these conditions established? An explicit instance showing what the theory newly explains, or where its applicability fails, would help assess the significance of the claimed extension.
2 comments

Thank you for these two references. We did not know them, and they change what we can claim. The revised PDF now cites both and no longer presents returns nonlinear in the activation as the new case; these changes are marked in orange (Sections 1 and 3).

W1, Q1. Direct precedents. Shekhovtsov (2021) treats the binary Gumbel-Softmax estimator: scalar logistic noise and a differentiable function ff of the relaxed variable. With κ=1/τ\kappa=1/\tau, Λ=f\Lambda=f, g=θ+ξg=\theta+\xi and η=−θ\eta=-\theta, this is our gate term with scalar logistic noise. His Propositions 1 and 2 give the conclusions of our Theorem 1, Proposition 2 and the bias orders of Proposition 3 for that case, by the same change of variables and the same expansion of the density. His result on the straight-through variant is the scalar precedent of our hard-forward Proposition 5. Wagner et al. (2024) prove gradient consistency and a second-moment bound for sigmoid-smoothed programs in the reparameterisation setting. This is a precedent for our Corollary 2 and for the O(κ)O(\kappa) bound of Proposition 2. Their work includes nested conditionals, which we exclude (L1), and an optimisation result that goes beyond our Remark 2.

What remains, to our reading. (i) For a return nonlinear in the activation on a switching surface in multi-dimensional noise, we identify the limit of the gate term with the boundary surface integral (Theorem 1). Shekhovtsov has this for scalar noise; Wagner et al. prove convergence of the full gradient. (ii) In the same setting we give the finite-κ\kappa expansions with explicit constants (Proposition 3). Shekhovtsov has them for scalar noise; we found no rate in Wagner et al. The technical change is the co-area formula in place of the scalar substitution. The mechanism is Shekhovtsov's. Our noise assumption is narrower than in both papers: compact support (L7).

W2, Q2. Once the dynamics are represented by Λ\Lambda, the contact structure plays no further role in the proofs. It fixes the inputs: the gap is a function of the state, so in general the surface is curved, the parameters move it non-uniformly, and the noise that decides contact also enters the return. For contact simulation our statements are limited to one regular contact event. Hard first-order gradients miss exactly the boundary term (Corollary 1). Simulating the smoothed model recovers it as κ→∞\kappa\to\infty (Theorem 1). A hard forward pass with a smoothed backward pass is guaranteed to recover it only for returns at most quadratic in the activation (Proposition 5). We give no rule for choosing κ\kappa and have no task-level experiment (L2).

Q3. The conditions are established only for a single effective gate: one gated Moreau step, where they hold exactly, and a 20-step unroll with one contact event, where the other gates change the return by at most 0.08% at κ=200\kappa=200 (measured, not proved). The theory does not apply when smoothing an earlier contact changes a later contact event (L1), and we have no such example.

I thank the authors for the candid response and for revising the contribution statement to credit Shekhovtsov (2021) and Wagner et al. (2024).

The response, however, confirms rather than resolves my central concern. The authors state that "the mechanism is Shekhovtsov's" and that "once the dynamics are represented by Λ, the contact structure plays no further role in the proofs." The remaining contribution is the extension of the scalar argument to a switching surface in multi-dimensional noise via the co-area formula, with explicit constants; for affine Λ, the surface-integral case is already covered by Liu & Hong (2011). This is a technical generalization of known results, not a new mechanism. The response also does not identify a new insight specific to contact simulation, and it provides neither a task-level experiment nor a multi-contact example. Proposition 5 is a relevant addition, but its scalar case has the straight-through Gumbel-Softmax precedent, as the authors note.

I also note for the discussion that the revision no longer presents returns nonlinear in the activation as a new case. This was one of the main strengths identified in other reviews.

The mathematical analysis remains rigorous, but its significance for contact simulation is not established. I therefore maintain my rating.

Reviewer_Pwr3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper analyzes sigmoid contact gating, the smoothing used by Schwarke et al. (2025), in which the contact impulse of a Moreau time-stepper is scaled by a(g) = σ(−κg) of the signed gap g. The analysis is local: one regular switching surface Σ in the space of rollout randomness ξ, with compactly supported, bounded noise.

Critical strengths
  1. Rigor. The hypotheses are local and clearly stated, and the transport identity and the co-area density regularity are proved from scratch, not cited. I checked the main proofs and found no errors.

  2. A new identification. The paper pinpoints the gate term as the part of the smoothed gradient that recovers the boundary correction hard differentiation misses, for a gate inside the dynamics and returns nonlinear in the activation. The kernel view (8) adds insight: the kernel is shaped by the readout and is in general neither symmetric nor positive.

  3. A useful finite-κ result. The O(κ) second-moment bound needs only the standing assumptions and explains the documented difficulty of training through stiff contact.

Critical weaknesses

W1 (main concern): ∇F is not the gradient the motivating algorithm needs. In Schwarke et al. (2025), the gate acts inside the forward solver. Policies are trained entirely on the smoothed model at a fixed κ, and the hard model appears only in post-hoc evaluation. There, E[∇_θL_κ] = ∇F_κ exactly, so the "bias" studied here is a gap between two models, not an error in the algorithm's gradient. For an optimizer that decreases F_κ, ∇F_κ is the consistent direction. The authors should either identify a setting where ∇F is operationally needed, or connect the gradient gap to a transfer-relevant quantity such as F(θ*_κ) − min F. In hybrid hard-forward/soft-backward pipelines, the estimator differs from Z_κ when Λ is nonlinear in a.

W2: no practical guidance on κ. The authors state that Cor. 3 is "not a rule for choosing κ" (Remark 1). The experiments confirm only the exponent of κ*(N), computed with oracle knowledge of B, and never vary κ in a task measuring performance or transfer.

Questions for authors
  1. In which concrete pipeline is ∇F, rather than ∇F_κ, the gradient the optimizer needs? If the answer is hybrid hard-forward/soft-backward pipelines, do the results extend to the estimator that evaluates ∂_aΛ at the hard activation?

  2. Can the gradient gap be connected to a transfer-relevant quantity such as F(θ*_κ) − min F?

1 comment

Thank you for checking the proofs. We agree with W1 and W2. The statements below are now in the paper; the additions made for this review are marked in green in the revised PDF.

W1. You are right. In Schwarke et al. the gradient is exact for the smoothed model, so what we call the finite-κ\kappa bias is a gap between two models. Section 4 now says this where the bias is introduced, and the new Remark 2 states it.

Q1. ∇F\nabla F is the target in hard-forward/soft-backward schemes, where the simulated model is the hard one. There the result extends, with a different limit. Under Assumptions 1–2, the gate term with ∂aΛ\partial_a\Lambda evaluated at the hard activation converges to BB with the jump L+−L−=−∫01∂aΛ(a)daL^+-L^-=-\int_0^1\partial_a\Lambda(a) da replaced by its trapezoid value −12[∂aΛ(0)+∂aΛ(1)]-\frac12[\partial_a\Lambda(0)+\partial_a\Lambda(1)]. The proof is that of Theorem 1, with half of the gate mass falling on each side of Σ\Sigma. The limit therefore equals BB when Λ\Lambda is at most quadratic in aa, as for every readout in Section 5, and not in general. This is the new Proposition 5 (Appendix D, with proof), summarized in a new paragraph after Corollary 2. For scalar logistic noise this limit has a precedent, the straight-through Gumbel-Softmax analysis of Shekhovtsov (2021), which Reviewer ReBw pointed out.

Q2. Yes, and this is why the gap matters even though the algorithm never uses ∇F\nabla F: the policy is evaluated on the hard model. At a stationary point θκ∗\theta_\kappa^\ast of FκF_\kappa, ∇F(θκ∗)=−(∇Fκ−∇F)(θκ∗)\nabla F(\theta_\kappa^\ast)=-(\nabla F_\kappa-\nabla F)(\theta_\kappa^\ast). So if the gap has norm at most ε(κ)\varepsilon(\kappa), smoothed training stops at an ε(κ)\varepsilon(\kappa)-stationary point of the hard objective. If FF also satisfies a Polyak–Łojasiewicz inequality with constant μ\mu near its minimiser (for instance, local strong convexity), then F(θκ∗)−min⁡F≤ε(κ)2/(2μ)F(\theta_\kappa^\ast)-\min F\le\varepsilon(\kappa)^2/(2\mu). In the configurations of Section 5, ε(κ)=O(κ−r)\varepsilon(\kappa)=O(\kappa^{-r}). In general we have a rate only for the gate part of the gap, at a fixed parameter. This is now stated in Remark 2.

W2. Agreed. Corollary 3 is a scaling law, not a rule for choosing κ\kappa, and we have no experiment that varies κ\kappa on a task. This is a limitation of the paper, now stated in L2.

We hope this resolves the soundness concern.

Reviewer_uEecOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper analyzes sigmoid smoothing of contact activation in differentiable simulation. It shows that, under a smooth interpolation assumption near a single regular switching surface, the expected gate derivative converges to the boundary contribution missed by hard pathwise differentiation. It also derives finite-sharpness bias and second-moment expansions, leading to a bias-variance scaling law.

Critical strengths

The paper gives precise connection between contact smoothing and the missing boundary contribution. The analysis accommodates returns that depend nonlinearly on contact activation and explains how this dependence can affect the bias order. The numerical examples closely reproduce the predicted limits and rates, and the authors explicitly acknowledge the limitations of their results.

Critical weaknesses
  • The paper is quite badly written, and it seems to me that it is written by AI in many parts (Page 7, lines 324 - 375). The central message is repeated too often, without any help in improving readability, understandability of the paper. The experimental section mixes findings with excessive verification detail. The distinction between established machinery and the new contribution is difficult to extract. The paper acknowledges prior results, but the reader must work to identify precisely what the nonlinear interpolation adds.
  • The experimental results are convincing as numerical checks of the theory, but weak as evidence of practical usefulness. I would recommend authors to add more experiments/evidences to show that the extension matters beyond the constructed single-contact setting.
Questions for authors
  • Could you provide a nontrivial multistep contact example in which Assumption 2 holds, or explain concrete conditions under which it can be verified? In particular, how does the representation handle cases where smoothing an earlier contact changes subsequent contact events? This would help clarify the scope of the theory beyond the single-contact examples.
  • Could you explain more explicitly which result cannot be obtained directly from existing boundary-correction and kernel-smoothing analyses, for examples Liu and Hong (2011) and Yang and Zhang (2026)? A concise comparison of assumptions and conclusions would help establish the significance of allowing nonlinear dependence on contact activation.
2 comments

Thank you for the careful reading. We agree that the presentation was a main weakness and have revised the text. The changes are rewording and reorganization; no results, assumptions, or numbers have changed. The revision also contains additions made for Reviewer Pwr3 (Remark 2 and Proposition 5), marked in green; the rewritten Section 5 is marked in blue.

W1 (writing). We rewrote the passage you point to (page 7, lines 324–375) and the paragraphs around it. Section 5 is now split into paragraphs with headings, and the three sanity checks come first, separate from the predicted and measured rates. We removed the repeated statements of the main claim from the abstract, Section 5 and the conclusion, and split long sentences throughout. The contributions now say which part is known and which is new, and Section 3 ends with a short paragraph on the relation to Yang & Zhang (2026); see Q2.

W2 (experiments). Agreed. The experiments check the theory on a model where the assumptions hold exactly; they do not show a practical benefit on a locomotion task, and this is a limitation of the paper. Within this scope, one experiment shows that the extension matters: for the kinetic-energy readout, which is quadratic in the activation, the bias decays as κ−1\kappa^{-1} (measured slope −1.04-1.04), whereas for returns affine in the activation it is always O(κ−2)O(\kappa^{-2}) (Remark 1). The optimal sharpness then scales as N1/3N^{1/3} instead of N1/5N^{1/5} (measured 0.320.32).

Q1 (multi-step contact). Assumption 2 holds when exactly one gate is in its transition layer on the noise support and the gaps at all other steps stay bounded away from zero, say ∣gj∣≥δ\lvert g_j\rvert\ge\delta. Then Λ(a,θ,ξ)\Lambda(a,\theta,\xi) is the return of the rollout with activation aa at the contact step, and a=0a=0 and a=1a=1 reproduce the free and the engaged branch of the hard simulator. The other gates differ from their hard values by at most e−κδe^{-\kappa\delta}, so the rollout deviates from the gated form by an exponentially small amount. The 20-step unroll of Appendix F is of this type: the 19 inert gates change the return by at most 0.08% at κ=200\kappa=200 and by nothing beyond double precision for κ≥750\kappa\ge750. It still has a single contact event, so it is not the nontrivial multi-contact example you ask for.

If smoothing an earlier contact changes a later contact event, the later gap depends on the earlier activation, and the return depends on two activations that vary at the same time. This is the case of interacting events that our theory does not cover (Limitation L1). We now state this explicitly in L1.

Q2 (relation to prior work). Both papers treat payoffs l⋅1{h≥0}l\cdot\mathbf 1\lbrace h\ge0\rbrace, which are linear in the indicator; in our notation, Λ\Lambda is affine in aa. In that case ZκZ_\kappa is exactly the kernel estimator of the boundary term of Liu & Hong (2011, Eq. 5), with the logistic density as kernel and bandwidth 1/κ1/\kappa; their Eq. (4) already reads it as the derivative of the payoff with the indicator replaced by the kernel's distribution function, here a sigmoid. With the logistic sigmoid it is also the estimator of Yang & Zhang (2026). For affine Λ\Lambda, Theorem 1, the O(κ)O(\kappa) second moment of Proposition 2, the O(κ−2)O(\kappa^{-2}) bias of Remark 1 and the N−4/5N^{-4/5} rate of Corollary 3 with r=2r=2 are therefore known in substance (Liu & Hong obtain the n−2/5n^{-2/5} error rate at bandwidth ∝n−1/5\propto n^{-1/5}), and we do not claim them as new. What does not follow from these works is the case of Λ\Lambda nonlinear in aa, which arises when the gate acts inside the dynamics and the readout is a nonlinear function of the post-contact state:

  • consistency still holds, because only ∫01∂aΛda=−[[L]]\int_0^1\partial_a\Lambda da=-[[L]] enters the limit (Theorem 1);
  • the kernel in (8) is the logistic density reweighted by ∂aΛ\partial_a\Lambda, so it is set by the readout and is in general neither symmetric nor positive;
  • the leading bias is generically O(κ−1)O(\kappa^{-1}) (C1≠0C_1\neq0) rather than O(κ−2)O(\kappa^{-2}), so the bias order and the scaling of κ\kappa with NN depend on the readout (W2).

Our assumption on the noise is more restrictive than that of Liu & Hong, who allow general distributions under moment and Lipschitz conditions; we need compact support (Limitation L7).

Liu & Hong; Yang & ZhangThis paper
smoothing acts onindicator in the payoffcontact impulse in the dynamics
dependence on aaaffineC1C^1, possibly nonlinear
kernelsymmetric, positiveset by the readout
leading biasO(κ−2)O(\kappa^{-2})C1κ−1C_1\kappa^{-1}, or C2κ−2C_2\kappa^{-2} if C1=0C_1=0
MSE at optimal κ\kappaO(N−4/5)O(N^{-4/5})O(N−2r/(2r+1))O(N^{-2r/(2r+1)}), r∈{1,2}r\in\lbrace1,2\rbrace

We hope the revised text is easier to follow and would be grateful if you could take another look at Sections 1, 3 and 5.

Reviewer_uEecOpenReview ↗

Raising the score

I thank the authors for taking into account the weaknesses that I mentioned. I also thank the authors for exaplaining to me the questions that I asked for.

The writing looks better to me now, the original submission was indeed very difficult to follow. After receiving the answers to the questions and reread the paper, I find it a bit difficult to be convinced by the performance of the method, though I'm not really an expert in the field.

My decision is, I'll raise the writing score to 2, and the final score to 2 also, for your rewriting and explaining more about the limitation of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.