acceptodds
Under review as a conference paper at ICLR 2027

Advancing Optimal Subset Oracle via Learning Relaxation of Neural Set Functions

Abstract

Learning neural set functions is pivotal to a wide range of important applications, including compound selection in AI-driven drug discovery and product recommendation. Recent work has introduced optimal subset oracles to implicitly learn set functions under practical weakly supervised settings, where model parameters are optimized through mean-field variational inference. However, these frameworks rely on Monte Carlo sampling to estimate gradients of the evidence lower bound when updating the variational distribution. Repeated sampling across iterations incurs substantial computational overhead, while the resulting stochasticity can destabilize the optimization trajectory. In this work, we reinterpret the evidence lower bound as a continuous relaxation of the set function and learn a surrogate objective that replaces sampling-based gradient estimation during variational optimization. The learned surrogate provides stable and efficient gradients throughout the continuous domain, thereby reducing computational overhead and accelerating inference. Furthermore, we establish an approximation guarantee for the proposed framework under submodular maximization and characterize its connection to variational free energy. Experiments on a variety of real-world tasks demonstrate consistent improvements over existing baselines.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
neural set functions, subset selection, variational inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_39UDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies a specific issue within the mean-field variational inference process of optimal subset (OS) oracle frameworks (such as EquiVSet and INSET): the estimation of the gradient of a multilinear extension via Monte Carlo (MC) sampling. According to the authors, this step is computationally expensive and introduces stochasticity that destabilizes the optimization trajectory. To address this, the authors propose "ReSet," which reinterprets the ELBO as a continuous relaxation of a set function and learns a differentiable surrogate function, gθ:[0,1]∣V∣g_\theta: [0,1]^{|V|}, to replace the MC-based gradient estimates with the gradient of gθg_\theta. Specifically, the method initializes ψ(0)\psi^{(0)} using a recognition network, performs KK iterations of gradient ascent, incorporating an entropy gradient and Langevin noise into ∇gθ\nabla g_\theta. And trains the entire framework by minimizing the binary cross-entropy (BCE) loss with respect to the final ψ(K)\psi^{(K)}. The step size α\alpha and temperature TT are also treated as learnable parameters.

Critical strengths

1. Consistent performance and efficiency improvements. Both the EquiVSet and INSET frameworks demonstrate higher performance than existing OS oracles, including iDiffMF, on most of the 16+ datasets. At the same time, inference time and memory are reduced.

2. A direct and proven solution to cost issues. Since removing sampling eliminates the evaluation of the per-sample set function, the proposed method directly and logically resolves the computational cost problem among the two issues raised by the authors. Their analysis quantitatively demonstrate this effect by varying the sample size and set size.

Critical weaknesses

1. The claimed optimization instability is not empirically demonstrated, and the paper’s own results contradict it. Instability is shown only in a toy example. Gradient variance and seed-to-seed variance are never measured on real tasks. In Table 11, increasing MC samples from 10 to 50 barely changes performance, and the gap to ReSet remains. This suggests that variance is not the bottleneck. The large gains are therefore more likely due to the change in model class than to more stable gradients. No control experiment separates these two explanations.

2. The theoretical framing does not match the implemented algorithm. gθg_\theta is trained only with BCE on ψ(K)\psi^{(K)} and is not trained to approximate any multilinear extension. The optimized objective is therefore not the ELBO of pθ(S∣V)p_\theta(S|V). With K=1K=1 in all main experiments, the method reduces to a feed-forward predictor. Thm 3.1 relies on γ\gamma-weak DR-submodularity, which is neither enforced nor measured, and it analyzes continuous greedy from ψ(0)=0\psi^{(0)}=0 rather than the actual RecNet-initialized update. The learnable temperature TT changes by less than 1% during training, so the claimed adaptive trade-off is not supported.

Questions for authors
  1. Demonstration of instability. Are there actual results comparing the gradient variance of MC estimation and ReSET, and the variance of the learning curve between seeds, in the assignment (e.g., Toys, Two Moons)? Also, how can the finding in Table 11 that increasing the number of MC samples barely improves performance be reconciled with the claim that "stochasticity induces a suboptimal solution"?

  2. Isolation of the cause of improvement. Is there a comparison between defining Fθ(S)F_\theta(S) using the same gθg_\theta architecture as ReSet (including MHA) and estimating the multilinear gradient using the traditional method via MC? Please clarify whether "EquiVSet-MHA" in Table 8 is this setting. This comparison is "learned relaxation vs. I believe it is the only way to separate the MC's effect.

  3. Source of the sharp increase. What caused the achievement of 99.7 in Two Moons, where all baselines remained around 58–59? The same question is also asked for Furniture (17→36, 280 training samples). Can these datasets demonstrate the MHA-augmented baseline and the K=0K=0 results?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 39UD

We thank the reviewer for their feedback and respond point by point below. All newly added or revised content is highlighted in the revised manuscript.

Demonstration of Instability

Original Table 11 (now Table 12) does not support the blanket claim that more MC samples barely improve performance: increasing samples from 10 to 50 raises EquiVSet’s MJC from 71.6 to 75.9 (+4.3 percentage points), although INSET gains only 0.4 points.

We added measurements on Toys in Appendix F.5 (Table 13): increasing samples from 10 to 50 reduces gradient variance by approximately 80%, including for MHA baselines.

ModelM=10M=50Reduction
EquiVSet20.19754.070479.8%
INSET1.91600.387679.8%
EquiVSet-MHA15.26103.035180.1%
INSET-MHA6.32641.273579.9%

ReSet eliminates MC sampling variance by directly differentiating the learned relaxation. We also measured per-epoch validation MJC variance across training seeds on Toys. Figure 8 shows that, after an initial peak, EquiVSet-R and INSET-R have 53.0% and 7.9% lower mean variance than their baselines during the second half of training. Lower gradient variance need not proportionally improve MJC, as the final top-kk selection may remain unchanged.


Surrogate Approximation of the Multilinear Extension

ReSet backpropagates BCE through unrolled refinement for optimal-subset prediction, avoiding costly explicit multilinear-matching targets—a core motivation. The objective is thus a learned variational surrogate, not an assumed exact ELBO of pθ(S∣V)p_\theta(S\mid V). The original Appendix F.1 and Figure 7 already demonstrate strong alignment with the induced multilinear extension’s gradients on synthetic inputs: median cosine similarities are 0.845–0.952 for EquiVSet-R and 0.944–0.982 for INSET-R.


Controlled Ablations and Sharp Gains

The original -MHA baselines already matched their ReSet counterparts’ architectures. New ablations in Appendix F.2 (Table 9) isolate MC estimation versus learned relaxation by controlling other factors, showing higher mean MJC on all six datasets in both families. See our response to Reviewer 94Ud.

A plausible explanation for the sharp gains is that ReSet learns a set objective tailored to finite-step inference and removes MC gradient noise. Improved inclusion-probability rankings may then yield substantial MJC gains after discrete selection.


K=1K=1 versus K=0K=0

Calling K=1K=1 merely feed-forward overlooks its variational optimization stage. As in EquiVSet and INSET, recognition initializes a subset distribution, which optimization adapts to the learned set objective. In contrast, K=0K=0 returns recognition marginals directly—a restricted DeepSet-type predictor in our implementation.

The new comparison in Appendix F.6 (Table 14) shows gains on all six datasets in both families. Dataset-averaged MJC (%) is:

ModelK=0K=0K=1K=1Gain (pp)
EquiVSet-R31.5056.28+24.78
INSET-R31.6356.88+25.25

These gains demonstrate the substantial benefit of one variational optimization step. Both EquiVSet and INSET use K=1K=1 by default, supporting this choice for amortized inference and ensuring fair comparison.


Theorem 3.1: Assumptions and RecNet Initialization

  • Structural assumptions. Appendix F.8 (Tables 17–18) presents two surrogate implementations guaranteeing γ=L=1\gamma=L=1 by construction, with numerical verification. Their measured mean normalized approximation ratios are 0.9875 and 0.9922 across two synthetic and six real datasets. See our response to Reviewer eZVm.

  • RecNet initialization. Zero initialization follows standard continuous greedy and avoids dependence on initialization quality. We extend the approximation and refinement analyses to feasible RecNet initialization in Appendices A.1 and A.4, respectively, establishing compatibility with our framework while accounting for the initial point in the approximation bound.


Learnable Temperature

The original temperature’s limited change may mainly reflect the small shared learning rate and its decay. We added a relative-temperature strategy in Appendix F.6: T(V)=λs(V)T(V)=\lambda s(V), where s(V)s(V) accounts for energy/entropy gradient scales and positive λ\lambda is learned through the original supervision with a separate learning rate. We measure the effective trade-off by r(V)=∥T(V)∇H∥2/∥∇G∥2r(V)=\|T(V)\nabla H\|_2/\|\nabla G\|_2. Across both families and six product datasets, Table 16 shows relative changes in mean rr from −76.61% to +179.53%, exceeding 10% in magnitude in nine of twelve comparisons. Average MJC gains over fixed T=1T=1 are 0.18 and 1.49 percentage points for EquiVSet-R and INSET-R, respectively, supporting adaptive energy–entropy balancing with comparable or improved performance.

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer 39UD,

Thank you again for your detailed feedback. We have added variance measurements (Appendix F.5), controlled ablations (Appendix F.2), and a comparison of K=0K=0 versus K=1K=1 to examine the role of the optimization step (Appendix F.6).

As the Author–Reviewer Discussion period draws to a close, we would appreciate your thoughts on these results and the clarifications in our response. We would be happy to discuss any concerns that remain.

Best regards,

The Authors

Reviewer_94UdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces ReSet (Relaxation of Set Functions), a method for learning neural set functions from optimal set examples, without the need to calculate utility for every subset. Unlike prior works, which use Monte Carlo to estimate gradients via repeated subset sampling, ReSet proposes learning a direct, differentiable continuous relaxation of the set function, which enables calculating the gradients through standard autodiff. The authors demonstrate theoretical approximation and convergence guarantees under weak DR-submodularity and smoothness assumptions, as well as a connection to variational free energy. Empirical results on simulated data, along with product recommendation, set anomaly detection, and compound selection, demonstrate the utility of the proposed methods in terms of both predictive performance and computational efficiency.

Critical strengths
  1. The method proposes a learned continuous relaxation that avoids the Monte Carlo sampling overhead and improves the computational efficiency of variational optimization.
  2. The authors demonstrate extensive empirical results showcasing the utility of the proposed method across a wide range of applications. The authors also performed a thorough investigation of computational efficiency, sensitivity to Monte Carlo sample count, and architectural components through ablation studies.
Critical weaknesses

The experimental comparisons and ablation studies do not sufficiently isolate the contribution of the learned relaxation from architectural and optimization changes. For example, Table 8 demonstrates the effects of multi-head attention, learnable temperature, and Langevin dynamics. However, it doesn’t fully establish how much of the performance improvement is specifically due to replacing Monte Carlo gradient estimation with the learned relaxation.

Questions for authors
  1. Can the authors please provide the percent improvement, in addition to the final performance, in Tables 1-3? Some of the improvements seem to be marginal (e.g., Table 2, Feeding, INSET vs. INSET-R), whereas some others are quite substantial (e.g., Table 2, Furniture, INSET vs. INSET-R).
  2. Could the authors provide a more controlled comparison that isolates the contribution of the learned relaxation from other architectural and optimization modifications?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 94Ud

We thank the reviewer for their feedback and respond point by point below. All newly added or revised content is highlighted in the revised manuscript.

More Direct Ablation Comparisons

The -MHA variants in the original Table 8 already used the same network architectures as their corresponding ReSet variants. To isolate the contribution of learned relaxation more directly, we have added a comparison between -MHA and R(T=1,w/o L)\mathrm{R}_{(T=1,\text{w/o L})}, using identical architectures, fixing T=1T=1, disabling Langevin noise, and matching all remaining settings. The only difference is MC estimation of the multilinear gradient versus direct differentiation of the learned relaxation. Test MJC (%) is reported as mean ± standard deviation below.

ModelToysFurnitureCarseatsDiaperSafetyMedia
EquiVSet-MHA75.7 ± 1.415.1 ± 1.820.6 ± 3.288.5 ± 0.713.9 ± 0.662.9 ± 2.8
EquiVSet-R(T=1,w/o L)\mathrm{R}_{(T=1,\text{w/o L})}78.0 ± 0.833.8 ± 4.327.6 ± 3.990.3 ± 0.740.2 ± 1.367.8 ± 1.0
INSET-MHA77.8 ± 1.317.4 ± 1.523.5 ± 2.090.2 ± 1.714.0 ± 0.565.5 ± 2.4
INSET-R(T=1,w/o L)\mathrm{R}_{(T=1,\text{w/o L})}78.1 ± 0.936.6 ± 1.131.6 ± 1.090.3 ± 0.435.9 ± 6.968.8 ± 0.5

The learned-relaxation variants achieve higher mean MJC on all six datasets in both model families, supporting their benefit beyond architectural changes, temperature learning, and Langevin noise. We have incorporated this comparison and clarified the shared architectures in Appendix F.2, Table 9, with the revisions highlighted.


Reporting Percent Improvements in Tables 1–3

We have added a summary of the final MJC scores and relative improvements corresponding to Tables 1–3 in Appendix E.7, Table 8. Relative improvement is calculated as 100×(MJCR−MJCbase)/MJCbase100\times(\mathrm{MJC}_{\mathrm{R}}-\mathrm{MJC}_{\mathrm{base}})/\mathrm{MJC}_{\mathrm{base}}. Averaging these percentages across all 20 datasets yields 23.86% for EquiVSet-R and 19.68% for INSET-R. EquiVSet-R improves mean MJC on all 20 datasets, while INSET-R improves on 18, with one tie and one slight decrease.

As the reviewer observes, the magnitude varies. For example, INSET-R improves Feeding from 88.2 to 88.6 (+0.45%) and Furniture from 16.9 to 36.5 (+19.6 percentage points; +115.98%). More generally, this variation may reflect differences in baseline strength and how well the baseline’s inductive biases match each task. Stronger baselines leave less room for improvement, whereas a learned relaxation may provide greater benefits when the baseline is less well matched to the data. Relative percentages also depend on the baseline score: the same absolute improvement produces a larger relative gain when the baseline is lower.

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer 94Ud,

Thank you again for your helpful suggestions. We have added a controlled ablation isolating the contribution of learned relaxation (Appendix F.2, Table 9), along with a summary of relative performance improvements across datasets (Appendix E.7, Table 8).

As the Author–Reviewer Discussion period draws to a close, we would appreciate your feedback on whether these additions address your questions. We would be glad to discuss any remaining concerns.

Best regards,

The Authors

Thank you for providing additional results and clarifications. The new controlled results provide more convincing evidence that the reported improvement is due to learned relaxation. I also appreciate including relative improvements in Table 8, as I believe this helps contextualize the varying degrees of improvement reported.

I consider my main concerns adequately addressed and will maintain my positive score of 3 (Weak Accept).

Thank you for your reply. We are happy that our response addressed your concern, and we thank you again for your constructive feedback.

Reviewer_eZVmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses learning neural set functions under the optimal subset oracle, where the model is trained from pairs of a ground set and its optimal subset and inference proceeds by mean-field variational optimisation of a Bernoulli parameter vector. Existing frameworks estimate the ELBO gradient by Monte Carlo sampling over subsets, which is expensive and noisy. The authors instead learn a differentiable surrogate, implemented as a permutation-invariant DeepSets network with multi-head attention, and train it together with a recognition network by propagating a cross-entropy loss on the final variational parameter back through the unrolled inner optimisation. The method is presented as a plug-in for existing optimal subset oracles, giving the variants EquiVSet-R and INSET-R. Theoretically the authors import approximation guarantees from continuous DR-submodular maximisation and relate the variational objective to the Helmholtz free energy. Experiments cover two synthetic datasets, product recommendation on the Amazon baby registry, set anomaly detection on four image benchmarks, and compound selection on PDBBind and BindingDB.

Critical strengths
  • The authors present a careful examination of model performance on several subset selection tasks, along with architectural ablations showing that the gains are not explained by the architectural changes alone.
  • The proposed method demonstrates a clear efficiency gain over its Monte Carlo counterpart, reporting faster inference for EquiVSet-R and INSET-R, at accuracy that is at least comparable.
  • Besides indirect evidence from downstream performance, the authors directly measure how closely the gradient of the learned surrogate aligns with that of the multilinear extension it stands in for, which few papers substituting a learned approximation bother to do.
Critical weaknesses
  • Section 3 explains everything around the method except the method. It sets up the mean-field factorisation, the ELBO, the multilinear extension, the Monte Carlo estimator being replaced and the approximation guarantee, but never says what trains the surrogate. The only statement on the point is a clause in 3.2 noting that \theta is updated through the loss of Eq. (5) and that this loss is propagated through the optimisation process. Algorithm 1 does not fill the gap: although presented as "a brief description of our method", it contains no parameter update, no loss and no outer loop, and it forward-references Eqs. (22) and (24) from Section 4, so it cannot be read where it appears. The construction procedure is Algorithm 2 in Appendix D.2 on page 23, and it belongs in the main text.
  • Neither theoretical contribution advertised in the abstract is specific to the proposed framework. Theorem 3.1 holds for any non-negative, L-smooth, \gamma-weakly DR-submodular function on the unit box; the proof in Appendix A.1 uses those three properties and nothing else, so renaming g_\theta leaves statement and proof unchanged, and 3.2 in fact attributes the guarantees to Pedramfar et al. (2024) and Hassani et al. (2017) while the abstract says they are established here. The properties are also unverified: nothing in Eq. (24) enforces weak DR-submodularity, \gamma and L are never estimated for any trained model.
  • The second advertised contribution is unrelated to the task: Section 3.3 restates the definition of variational free energy and the standard identity relating it to the negative ELBO, both of which hold for any Boltzmann distribution and any variational family, with the mean-field assumption playing no role.
  • The abstract's claim of consistent improvements over existing baselines holds only against the weaker of the two oracles. For INSET difference is less substantial and is often within the measured standed deviation.
Questions for authors
  • Could Algorithm 2 be moved into Section 3? As the main text stands, the section explains how \psi is optimised given the surrogate but not how the surrogate is obtained.
  • Is Theorem 3.1 presented as a new result or as a restatement? Section 3.2 attributes it to Pedramfar et al. and Hassani et al. while the abstract says it is established here. Since the proof requires only non-negativity, L-smoothness and \gamma-weak DR-submodularity, is there anything in the statement specific to the learned relaxation as opposed to any function with those properties? Could you also report estimates of \gamma and L for a trained surrogate, since the approximation factor vanishes as \gamma goes to zero?
  • Is there a consequence of the free-energy equivalence in Section 3.3 beyond motivating a learnable temperature? If not, presenting it as an interpretive remark rather than as one of the paper's contributions would be clearer, given that the statements hold for any Boltzmann distribution and any variational family.
5 comments
AuthorsOpenReview ↗

Response to Reviewer eZVm

We thank the reviewer for their feedback and respond point by point below. All newly added or revised content is highlighted in the manuscript.

Clarification of the Training Procedure and Algorithmic Details in Section 3

We agree that Section 3 should present the training procedure more explicitly. However, the training objective was already specified in the original submission: Section 2, Eqs. (2)–(5), introduced the general OS-oracle learning framework and explicitly described parameter updates through the supervised loss in Eq. (5). As a general plug-in module, ReSet replaces MC-based gradient estimation with a learned relaxation while retaining this framework and the same supervision objective. Section 3.2 also stated that the surrogate parameters are updated through the marginal-based loss in Eq. (5). For clarity, we have moved the original Algorithm 2 into Section 3.2 and further refined the presentation to improve readability.


Attribution and Empirical Verification of Properties in Theorem 3.1

  • Theoretical Contribution and Prior Work. Our contribution concerns the learned-relaxation formulation for optimal subset oracles and its optimization analysis, rather than a new general continuous-greedy bound. The architecture-independent conditions in Theorem 3.1 allow prior optimization analyses to apply to learned surrogates within our framework. Our analysis connects the learned relaxation to discrete subset selection under an additional directional-consistency condition (Appendix A.3) and characterizes entropy-regularized refinement and the resulting utility correction (Appendix A.4). These framework-specific results constitute our own theoretical contribution beyond the underlying general optimization tools. We have revised the wording to clarify this distinction and how our analysis builds on prior work.

  • Condition Guarantees and Approximation Ratio Measurement in Theorem 3.1. We have added Appendix F.8 to present two concrete implementations of the learned surrogate that satisfy the structural assumptions of Theorem 3.1. The first constructs the surrogate from frozen model predictions, retaining the same predictions and performance without retraining. The second imposes explicit structural constraints during training and achieves comparable or better MJC (Table 17). Both implementations guarantee γ=L=1\gamma=L=1 by construction, supported by numerical gradient and Hessian measurements.

    We further directly measure empirical approximation ratios. Across two synthetic and six real-world datasets, the implementations without and with retraining achieve mean ratios of 0.9875 and 0.9922, with minima of 0.9545 and 0.8136, respectively (Table 18). These results demonstrate that the structural assumptions can be concretely satisfied within our framework without material performance loss. Moreover, the measurements on real-world datasets provide strong empirical evidence that the resulting outputs attain near-optimal objective values on average.


Positioning of Section 3.3 and Role of the Mean-Field Assumption

We agree that the general free-energy identity is standard and does not require a mean-field family; however, its specialization here is directly relevant to optimal subset learning. The product-Bernoulli factorization represents the variational distribution through elementwise inclusion probabilities, makes the expected set utility a multilinear extension (Eq. (7)), and yields the separable entropy in Eq. (19). This explains why ReSet⁡\operatorname{ReSet} learns the computationally demanding utility relaxation while retaining the analytic entropy gradient in Eq. (12). The significance also extends beyond motivating a learnable temperature: the resulting entropy-regularized surrogate objective connects directly to the deterministic projected refinement analyzed in Appendix A.4, including temperature-dependent smoothness, stationarity conditions, and entropy-dependent utility corrections. These consequences concern the specified objective and refinement conditions, rather than following from the generic identity alone. Following the reviewer’s suggestion, we now present the standard statements as Remarks 3.2 and 3.3 in Section 3.3 and explicitly clarify the role of mean-field factorization.


Performance Improvements over Baselines

The results support improvements for both OS-oracle families. Across 20 datasets, EquiVSet-R and INSET-R achieve higher MJC on 20/20 and 18/20 datasets, with average relative gains of 23.86% and 19.68%, respectively (Appendix E.7, Table 8). Substantial INSET improvements include 46.2 → 73.2 on Double MNIST and 16.9 → 36.5 on Furniture. “Consistent” describes the overall pattern of improvements across datasets. Ablations with matched architectures show higher MJC in both families (Appendix F.2, Table 9), further supporting learned relaxation. See our response to Reviewer 94Ud.

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer eZVm,

Thank you again for your constructive feedback. Following your suggestions, we have clarified the training procedure and the scope of our theoretical contribution, and added concrete surrogate constructions and empirical approximation-ratio measurements in Appendix F.8.

As the Author–Reviewer Discussion period draws to a close, we would appreciate your feedback on whether our response and revisions address your concerns. We would be happy to clarify any remaining points.

Best regards,

The Authors

I would like to thank the authors for their response. I appreciate the additional empirical results and think that among the weaknesses specified in my initial review the note about the improvements the method brings was not fair.

However, I still find the explanation of the method and the proposed theoretical links hard to follow and understand. As I can see, the reviewer 39UD could not match the implemented algorithm with the theoretical framing as well. Most of the exposition elements are well-known (like the analytic entropy of product-Bernoulli), but I cannot see how they are linked together to bring the proposed method.

The existence of a surrogate the theorem (appendix f.8) does not bring additional insights about the learned surrogate. Could the relevant reasoning in Appendix A.4 be stated as a self-contained lemma or proposition, with explicit assumptions and a clear explanation of which conclusions apply to the implemented procedure?

For now, I would retain my original score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal and Revisions

Dear AC, SAC, and PCs,

Thank you for handling our submission. We have posted our responses and revised manuscript and have not yet heard back from the reviewers. As the discussion period draws to a close, we would be grateful if you could encourage them to take a look. To facilitate your assessment, we summarize the key revisions and additional evidence below.

  1. Benefit of learned relaxation (94Ud, 39UD). Controlled ablations show higher MJC with learned relaxation in both model families (Appendix F.2, Table 9). EquiVSet-R and INSET-R improve MJC on 20/20 and 18/20 datasets, respectively, with average per-dataset relative gains of 23.86% and 19.68% (Appendix E.7, Table 8).

  2. Theory and structural guarantees (eZVm, 39UD). We clarified the theoretical scope and extended the analysis to feasible RecNet initialization (Appendices A.1 and A.4). We added two structured surrogates satisfying Theorem 3.1's conditions by construction, with γ=L=1\gamma=L=1, and evaluated their performance and empirical approximation ratios (Appendix F.8, Tables 17–18).

  3. Stability and optimization (39UD). We separately measured gradient-sampling variance and cross-seed validation-MJC variability on Toys (Appendix F.5). Compared with separately trained recognition-only variants (K=0K=0), one-step refinement (K=1K=1) improves dataset-averaged MJC by approximately 25 percentage points in both families across 6 datasets (Appendix F.6, Table 14). We also analyzed the energy–entropy trade-off under relative temperature (Table 16).

  4. Clearer presentation (eZVm, 39UD). We clarified the learned-surrogate objective and training procedure (Section 3.2), and presented standard free-energy identities as remarks while explaining the mean-field formulation (Section 3.3).

We would appreciate your consideration of these revisions and additional evidence in your final assessment. Thank you for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.