- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SCULPT, a backpropagation-free framework for adapting frozen linear prediction heads using symbolic feedback. It transforms prediction constraints into parameter-space half-spaces and combines reliability-gated directional updates, covariance-aware fitting, and optional trust-region control. Experiments cover routing, visual Sudoku, and MNIST Addition across different backbones.
Critical strengths
-
The core idea is clear and well motivated. Transforming symbolic constraints into linear-head parameter constraints provides a mathematically precise way to use non-differentiable solver feedback without backpropagation.
-
The evaluation covers diverse tasks and representations. Sudoku raw-cell accuracy improves by 6.44 percentage points over a label-using supervised baseline, and all 21 held-out backbone configurations show positive mean gains over their corresponding frozen heads, although some gains are very small.
Critical weaknesses
-
Improvements over strong baselines are inconsistent. Several routing comparisons are not significant after Holm correction. SCULPT trails matched Adam on Warcraft 12×12 and exact marginalization on multi-digit Addition. It also trails DataSP, although that baseline uses additional ground-truth cost information.
-
The additional mechanisms and practical efficiency need stronger justification. Existing projection and ridge-regression methods provide related updates, while the benefits of reliability gating and mixed routing remain insufficiently isolated. Computational complexity is reported, but measured runtime and memory comparisons are missing.
Questions for authors
-
Beyond the Sudoku controls in Figure 3(c), can the authors isolate the contributions of reliability gating, historical agreement, and mixed-branch updates?
-
Does the Sudoku advantage persist against a supervised ridge-regression baseline under comparable fitting budgets?
6 comments
W2 and Q1 (components): Reliability gating and historical agreement
Response: We separately ablated direction switching, the soft consensus gate, historical agreement weighting, and projection mechanisms. Under noisy Dijkstra feedback, these ablations reduce mean accuracy. On Warcraft, coherent-direction updates are better suited to this setting, and fixing the conflicting direction reduces performance (Table R2).
Table R2. Directional-update component ablations. Dijkstra is evaluated on 18 × 18 OOD maps with 20% feedback corruption; Warcraft uses 18 × 18 maps with no feedback corruption. Results are mean ± SD of exact path match (%). Drop is the respective full SCULPT mean minus the ablation mean (pp); negative values indicate a higher ablation mean. A dash denotes the reference row or an unavailable ablation result.
| Setting | Dijkstra (%) | Drop (pp) | Warcraft (%) | Drop (pp) |
|---|---|---|---|---|
| Full SCULPT | 53.00 ± 9.20 | — | 19.26 ± 0.69 | — |
| Fixed coherent direction (hard switch off) | 41.00 ± 4.18 | 12.00 | 19.26 ± 0.69 | 0.00 |
| Fixed conflicting direction (hard switch off) | 50.67 ± 5.25 | 2.33 | 14.88 ± 0.53 | 4.38 |
| No soft consensus gate | 52.50 ± 6.43 | 0.50 | — | — |
| No historical agreement weighting | 48.17 ± 3.79 | 4.83 | 19.26 ± 0.69 | 0.00 |
| No projection coefficient cap | 52.00 ± 9.12 | 1.00 | 19.58 ± 0.89 | -0.32 |
| No projection term | 52.00 ± 10.28 | 1.00 | 19.14 ± 0.50 | 0.12 |
| No overall trust projection | — | — | 19.26 ± 0.69 | 0.00 |
W2 and Q1 (mixed branches): Combining both feedback types
Response: We compared the directional branch, the covariance branch, and their combination under the same Warcraft setting. The combined update achieves higher mean accuracy. Removing the directional branch causes the larger drop, while the covariance branch provides an additional improvement in mean performance (Table R3). The directional branch obtains a correction direction from path comparisons; the covariance branch uses feature correlations to fit target cost changes.
Table R3. Mixed-branch ablations on Warcraft 18 × 18 with no feedback corruption. Results are mean ± SD of exact path match (%); drops are relative to the combined update (pp).
| Setting | Exact path match (%) | Drop from combined update (pp) |
|---|---|---|
| Combined update (full SCULPT) | 19.26 ± 0.69 | — |
| Directional only (no covariance contribution) | 18.82 ± 1.33 | 0.44 |
| Covariance only (no directional contribution) | 14.82 ± 1.31 | 4.44 |
W1: Comparison with strong baselines
Response: We added Perceptron, PA variants, OGD, and ridge-regression baselines. Table R1 summarizes the best-performing added baseline in each setting. SCULPT achieves higher means on Dijkstra ID, Sudoku, Addition, and larger Warcraft maps, while performing comparably to Perceptron on Dijkstra OOD. The supervised ridge comparison on Sudoku also matches the frozen features, data split, and fitting budget (Tables R4 and R5).
Table R1. Summary of added baselines. Each setting lists the added baseline with the highest test mean; SCULPT results are taken from the main experiments in the paper. Results are mean ± SD (%); Δ is SCULPT minus the baseline mean (pp).
† Supervised reference using ground-truth labels. * Pseudo-label Ridge, Weighted Pseudo-label Ridge, and Supervised Ridge† tie on decoded whole-board accuracy. Differences are computed before rounding and reported to two decimal places.