acceptodds
Under review as a conference paper at ICLR 2027

SCULPT: Backpropagation-Free Linear-Head Adaptation with Symbolic Constraints

Abstract

Non-differentiable solvers and rule engines provide structured feedback on neural predictions, yet translating this feedback into lightweight post-deployment updates remains challenging. We introduce Symbolic Constraint Updates for Lightweight Predictor Tuning (SCULPT), a backpropagation-free framework for adapting linear prediction heads over frozen representations. The key observation is that a linear inequality on prediction scores maps exactly to a half-space in head-parameter space, providing a directly computable update direction. To accommodate feedback with varying reliability and structure, SCULPT combines reliability-gated directions from violated constraints with covariance-aware fitting of consensus-filtered solver targets. A shared feedback router supports either feedback type or their combination, with an optional trust projection for drift control. Experiments on routing, visual Sudoku, and corrupted digit addition demonstrate the utility of this interface, including a gain of 6.44 percentage points in Sudoku cell accuracy over the supervised head. \method provides a unified approach to incorporating actionable symbolic feedback into deployed predictors without retraining their representations or differentiating through the checker.

Primary area:
learning theory
Keywords:
SCULPT, symbolic feedback, backpropagation‑free, constraint‑based tuning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_q1MoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCULPT, a backpropagation-free framework for adapting frozen linear prediction heads using symbolic feedback. It transforms prediction constraints into parameter-space half-spaces and combines reliability-gated directional updates, covariance-aware fitting, and optional trust-region control. Experiments cover routing, visual Sudoku, and MNIST Addition across different backbones.

Critical strengths
  1. The core idea is clear and well motivated. Transforming symbolic constraints into linear-head parameter constraints provides a mathematically precise way to use non-differentiable solver feedback without backpropagation.

  2. The evaluation covers diverse tasks and representations. Sudoku raw-cell accuracy improves by 6.44 percentage points over a label-using supervised baseline, and all 21 held-out backbone configurations show positive mean gains over their corresponding frozen heads, although some gains are very small.

Critical weaknesses
  1. Improvements over strong baselines are inconsistent. Several routing comparisons are not significant after Holm correction. SCULPT trails matched Adam on Warcraft 12×12 and exact marginalization on multi-digit Addition. It also trails DataSP, although that baseline uses additional ground-truth cost information.

  2. The additional mechanisms and practical efficiency need stronger justification. Existing projection and ridge-regression methods provide related updates, while the benefits of reliability gating and mixed routing remain insufficiently isolated. Computational complexity is reported, but measured runtime and memory comparisons are missing.

Questions for authors
  1. Beyond the Sudoku controls in Figure 3(c), can the authors isolate the contributions of reliability gating, historical agreement, and mixed-branch updates?

  2. Does the Sudoku advantage persist against a supervised ridge-regression baseline under comparable fitting budgets?

6 comments
AuthorsOpenReview ↗

W1: Comparison with strong baselines

Response: We added Perceptron, PA variants, OGD, and ridge-regression baselines. Table R1 summarizes the best-performing added baseline in each setting. SCULPT achieves higher means on Dijkstra ID, Sudoku, Addition, and larger Warcraft maps, while performing comparably to Perceptron on Dijkstra OOD. The supervised ridge comparison on Sudoku also matches the frozen features, data split, and fitting budget (Tables R4 and R5).

Table R1. Summary of added baselines. Each setting lists the added baseline with the highest test mean; SCULPT results are taken from the main experiments in the paper. Results are mean ± SD (%); Δ is SCULPT minus the baseline mean (pp).

Task / settingBaselineBaseline (%)SCULPT (%)Δ (pp)
Dijkstra (exact path match)
12×12 IDAveraged Perceptron76.75 ± 9.0687.13 ± 3.66+10.38
18×18 OODPerceptron82.67 ± 3.6582.83 ± 3.52+0.16
Warcraft (exact path match)
12×12PA-I37.70 ± 2.7136.90 ± 0.96-0.80
18×18PA-I19.58 ± 4.6519.26 ± 0.71-0.32
24×24PA-I9.88 ± 1.3611.88 ± 0.83+2.00
30×30PA-II6.50 ± 2.338.22 ± 0.54+1.72
Visual Sudoku
Raw digit accuracySupervised Ridge†69.36 ± 0.5970.49 ± 0.33+1.13
Decoded whole-board accuracyRidge variants*95.00 ± 4.7496.25 ± 4.07+1.25
MNIST Addition (raw digit-pair exact match)
Single-digitSupervised Ridge†77.06 ± 1.0178.74 ± 1.07+1.68
Two-digit IDSupervised Ridge†60.18 ± 0.8161.33 ± 0.55+1.15
Three-digit OODSupervised Ridge†46.45 ± 0.7147.47 ± 0.65+1.02

† Supervised reference using ground-truth labels. * Pseudo-label Ridge, Weighted Pseudo-label Ridge, and Supervised Ridge† tie on decoded whole-board accuracy. Differences are computed before rounding and reported to two decimal places.

AuthorsOpenReview ↗

W2 and Q1 (components): Reliability gating and historical agreement

Response: We separately ablated direction switching, the soft consensus gate, historical agreement weighting, and projection mechanisms. Under noisy Dijkstra feedback, these ablations reduce mean accuracy. On Warcraft, coherent-direction updates are better suited to this setting, and fixing the conflicting direction reduces performance (Table R2).

Table R2. Directional-update component ablations. Dijkstra is evaluated on 18 × 18 OOD maps with 20% feedback corruption; Warcraft uses 18 × 18 maps with no feedback corruption. Results are mean ± SD of exact path match (%). Drop is the respective full SCULPT mean minus the ablation mean (pp); negative values indicate a higher ablation mean. A dash denotes the reference row or an unavailable ablation result.

SettingDijkstra (%)Drop (pp)Warcraft (%)Drop (pp)
Full SCULPT53.00 ± 9.20—19.26 ± 0.69—
Fixed coherent direction (hard switch off)41.00 ± 4.1812.0019.26 ± 0.690.00
Fixed conflicting direction (hard switch off)50.67 ± 5.252.3314.88 ± 0.534.38
No soft consensus gate52.50 ± 6.430.50——
No historical agreement weighting48.17 ± 3.794.8319.26 ± 0.690.00
No projection coefficient cap52.00 ± 9.121.0019.58 ± 0.89-0.32
No projection term52.00 ± 10.281.0019.14 ± 0.500.12
No overall trust projection——19.26 ± 0.690.00
AuthorsOpenReview ↗

W2 and Q1 (mixed branches): Combining both feedback types

Response: We compared the directional branch, the covariance branch, and their combination under the same Warcraft setting. The combined update achieves higher mean accuracy. Removing the directional branch causes the larger drop, while the covariance branch provides an additional improvement in mean performance (Table R3). The directional branch obtains a correction direction from path comparisons; the covariance branch uses feature correlations to fit target cost changes.

Table R3. Mixed-branch ablations on Warcraft 18 × 18 with no feedback corruption. Results are mean ± SD of exact path match (%); drops are relative to the combined update (pp).

SettingExact path match (%)Drop from combined update (pp)
Combined update (full SCULPT)19.26 ± 0.69—
Directional only (no covariance contribution)18.82 ± 1.330.44
Covariance only (no directional contribution)14.82 ± 1.314.44
Reviewer_BSVUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces SCULPT, a backpropagation-free method for adapting linear prediction heads using symbolic constraint feedbacks. The core idea is that the linear inequality constraints in output can be exactly mapped to half-space constraints in the linear-head parameter space, which makes it possible to directly update the parameters without backpropagation. The method is tested on various tasks, including Dijkstra routing, Warcraft shortest paths, visual Sudoku and MNIST Addition, and the results demonstrate substantial improvements on Sudoku and mixed outcomes on the other tasks.

Critical strengths
  1. This paper studies post-deployment adaptation with symbolic feedback from non-differentiable checkers/solvers. This is empirically relevant, particularly when retraining the backbone is not feasible.
  2. This paper aggregates directional constraints and consensus-filtered solver targets into one unified interface, which is clean, modular and interpretable.
  3. The experiments cover a variety of structured prediction tasks and provide detailed analysis, showing that symbolic feedback can improve the model's predictions
Critical weaknesses
  1. The novelty of the core method is relatively limited: half-space mapping is closely related to existing approaches such as Structured Perceptron, Passive-Aggressive, and online projection, while covariance-aware fitting is essentially a form of weighted ridge regression. The authors have not sufficiently clarified the distinct algorithmic contributions of their method.
  2. The paper lacks ablation studies of some of its key components, particularly reliability gating, history agreement, etc. The individual contributions of these components remain unclear.
  3. The effectiveness of the shared feedback router is not sufficiently proven, since most experiments use either the moment branch or the covariance branch, with only some Warcraft experiments combining both.
Questions for authors
  1. Could the authors clarify the key algorithmic innovations of SCULPT compared with Structured Perceptron, Passive-Aggressive, and existing online projection methods?
  2. Could the authors provide more comprehensive ablation studies to demonstrate the necessity of each component and the shared feedback router?
4 comments
AuthorsOpenReview ↗

W1 and Q1: Algorithmic contribution and relation to existing methods

Response: For constraint-based updates, SCULPT selects the direction according to batch agreement and controls feedback influence through historical agreement weighting and a projection coefficient cap. Its weighted ridge solve uses solver targets filtered by perturbation consensus, together with full feature covariance and statistics accumulated across batches, to update the prediction head. We added Perceptron, PA variants, OGD, and ridge-regression baselines to evaluate these designs against simpler update rules (Table R1).

Table R1. Summary of added baselines. Each setting lists the added baseline with the highest test mean; SCULPT results are taken from the main experiments in the paper. Results are mean ± SD (%); Δ is SCULPT minus the baseline mean (pp).

Task / settingBaselineBaseline (%)SCULPT (%)Δ (pp)
Dijkstra (exact path match)
12×12 IDAveraged Perceptron76.75 ± 9.0687.13 ± 3.66+10.38
18×18 OODPerceptron82.67 ± 3.6582.83 ± 3.52+0.16
Warcraft (exact path match)
12×12PA-I37.70 ± 2.7136.90 ± 0.96-0.80
18×18PA-I19.58 ± 4.6519.26 ± 0.71-0.32
24×24PA-I9.88 ± 1.3611.88 ± 0.83+2.00
30×30PA-II6.50 ± 2.338.22 ± 0.54+1.72
Visual Sudoku
Raw digit accuracySupervised Ridge†69.36 ± 0.5970.49 ± 0.33+1.13
Decoded whole-board accuracyRidge variants*95.00 ± 4.7496.25 ± 4.07+1.25
MNIST Addition (raw digit-pair exact match)
Single-digitSupervised Ridge†77.06 ± 1.0178.74 ± 1.07+1.68
Two-digit IDSupervised Ridge†60.18 ± 0.8161.33 ± 0.55+1.15
Three-digit OODSupervised Ridge†46.45 ± 0.7147.47 ± 0.65+1.02

† Supervised reference using ground-truth labels. * Pseudo-label Ridge, Weighted Pseudo-label Ridge, and Supervised Ridge† tie on decoded whole-board accuracy. Differences are computed before rounding and reported to two decimal places.

AuthorsOpenReview ↗

W2 and Q2 (components): Reliability mechanisms and covariance fitting

Response: Under noisy Dijkstra feedback, fixing the update direction or removing the soft consensus gate, historical agreement weighting, or projection mechanisms reduces mean accuracy (Table R2a). On Sudoku and Addition, retaining only diagonal covariance or current-batch statistics substantially reduces performance (Table R2b). Coherent-direction updates are better suited to the Warcraft setting; fixing the conflicting direction reduces performance (Table R2c).

Table R2a. Dijkstra component ablations on 18 × 18 OOD maps with 20% feedback corruption. The metric is exact path match, reported as mean ± SD (%). Drop is the full SCULPT mean minus the ablation mean (pp).

SettingExact path match (%)Drop from full SCULPT (pp)
Full SCULPT53.00 ± 9.20—
Fixed coherent direction (soft gate retained)41.00 ± 4.1812.00
Fixed conflicting direction50.67 ± 5.252.33
No soft consensus gate52.50 ± 6.430.50
No historical agreement weighting48.17 ± 3.794.83
No projection coefficient cap52.00 ± 9.121.00
No projection term52.00 ± 10.281.00

Table R2b. Covariance-branch ablations. Sudoku reports digit accuracy under head perturbation; Addition reports raw digit-pair exact-match accuracy for single-digit addition. Results are mean ± SD (%). Drops relative to full SCULPT are computed before rounding and reported to two decimal places (pp).

SettingSudoku (%)Sudoku drop (pp)Addition (%)Addition drop (pp)
Full SCULPT70.50 ± 0.34—78.74 ± 1.07—
No target filtering or weighting70.49 ± 0.330.0178.70 ± 1.110.04
Diagonal covariance only58.44 ± 0.5212.0665.86 ± 1.4212.88
Current-batch statistics only66.93 ± 0.543.5754.15 ± 2.3824.59
No overall trust projection70.49 ± 0.330.0178.70 ± 1.070.04

Table R2c. Component ablations on Warcraft 18 × 18 with no feedback corruption. Results are mean ± SD of exact path match (%). Drop is the full SCULPT mean minus the ablation mean (pp); negative values indicate a higher ablation mean.

SettingExact path match (%)Drop from full SCULPT (pp)
Full SCULPT19.26 ± 0.69—
Fixed coherent direction (hard switch off)19.26 ± 0.690.00
Fixed conflicting direction (hard switch off)14.88 ± 0.534.38
No historical agreement weighting19.26 ± 0.690.00
No projection coefficient cap19.58 ± 0.89-0.32
No projection term19.14 ± 0.500.12
No overall trust projection19.26 ± 0.690.00
AuthorsOpenReview ↗

W3 and Q2 (routing): Shared feedback routing

Response: The shared feedback router sends directional constraints to the moment branch and solver targets to the covariance branch, combining their updates when both feedback types are available. We compared the two individual branches and their combination under the same Warcraft setting. The combined update achieves higher mean accuracy. Removing the directional branch causes the larger drop, while the covariance branch provides an additional improvement in mean performance (Table R3).

Table R3. Branch ablations for shared feedback routing on Warcraft 18 × 18 with no feedback corruption. Results are mean ± SD of exact path match (%); drops are relative to the combined update (pp).

SettingExact path match (%)Drop from combined update (pp)
Combined update (full SCULPT)19.26 ± 0.69—
Directional only (no covariance contribution)18.82 ± 1.330.44
Covariance only (no directional contribution)14.82 ± 1.314.44

I thank the authors for the additional baselines and ablations. However, the new results do not fully resolve my concerns: the advantage over PA/Perceptron baselines is inconsistent, several components show negligible or even negative contribution, and the shared router is validated in only one setting. I therefore maintain my rating of weak rejection.

Reviewer_tYaSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SCULPT, a backpropagation-free test-time adaptation framework that updates only a linear prediction head while keeping the representation frozen. Its key idea is to convert directional feedback from non-differentiable symbolic checkers into exact linear constraints on the head parameters, and to complement this with covariance-aware fitting of consensus solver targets. The method is evaluated on Dijkstra/Warcraft routing, clue-free visual Sudoku, and corrupted MNIST Addition, including feedback-budget, robustness, and backbone transfer experiments.

Critical strengths
  1. Clean formulation of symbolic feedback in parameter space. The exact mapping from a checker constraint a⊤y≥ba^\top y\ge b to a half-space over the linear-head parameters is elegant and provides a principled basis for gradient-free adaptation.
  2. Broad and reasonably careful evaluation. The paper tests multiple task types, feedback budgets, corruption levels, and frozen backbones rather than relying on a single benchmark.
Critical weaknesses
  1. The empirical advantage over strong baselines is inconsistent. Several reported gains are small or statistically insignificant. SCULPT loses to DataSP on Warcraft and exact marginalization on multi-digit Addition. Thus, the experiments support SCULPT as a useful adaptation interface, but less strongly support it as a generally superior adaptation method.
  2. The method relies on unusually informative checker feedback. SCULPT requires directional constraints or feasible solver targets rather than simple accept/reject feedback, and the routing experiments directly use reference/oracle paths. This makes the practical applicability narrower than the general framing of “non-differentiable checker feedback” initially suggests.
Questions for authors
  1. How much of the reported improvement comes from the proposed update rules versus simply having access to highly informative oracle/reference-path or solver-generated targets? In particular, can you provide a stronger comparison using the same symbolic information but a simpler update rule?
  2. Given that several main comparisons do not remain significant after multiple-comparison correction, what specific empirical claim do the authors believe is most strongly supported by the experiments?
  3. How expensive are the perturb-and-MAP queries relative to ordinary gradient-based adaptation? Since SCULPT's practical motivation is lightweight adaptation, this computational cost seems important.
4 comments
AuthorsOpenReview ↗

W1 and Q2: Empirical contribution and baseline comparisons

Response: The central empirical finding is that SCULPT uses symbolic feedback to improve raw predictions on frozen representations, handling directional constraints and feasible targets through a shared adaptation interface. We added simpler update baselines, including Perceptron, PA variants, and OGD; Table R1 reports the best-performing added baseline in each setting and its difference from SCULPT. In the paper, DataSP has a higher mean on the smallest Warcraft maps, and Exact marginal has higher means on multi-digit Addition. The comparisons of SCULPT against DBB on the smallest Warcraft maps and against the original supervised head on Sudoku raw cell accuracy remain significant after the multiple-comparison correction reported in Appendix D.

Table R1. Added simpler update baselines. Each setting lists the highest test mean among the added Perceptron, Averaged Perceptron, PA variants, and OGD controls. SCULPT results are taken from the main experiments in the paper. Results are mean ± SD (%); Δ is SCULPT minus the baseline mean (pp), computed before rounding and reported to two decimal places.

Task / settingBaselineBaseline (%)SCULPT (%)Δ (pp)
Dijkstra (exact path match)
12 × 12 IDAveraged Perceptron76.75 ± 9.0687.13 ± 3.66+10.38
18 × 18 OODPerceptron82.67 ± 3.6582.83 ± 3.52+0.16
Warcraft (exact path match)
12 × 12PA-I37.70 ± 2.7136.90 ± 0.96-0.80
18 × 18PA-I19.58 ± 4.6519.26 ± 0.71-0.32
24 × 24PA-I9.88 ± 1.3611.88 ± 0.83+2.00
30 × 30PA-II6.50 ± 2.338.22 ± 0.54+1.72
Visual Sudoku
Raw cell accuracyPA-II66.55 ± 1.3470.49 ± 0.33+3.94
Decoded whole-board accuracyPerceptron91.25 ± 5.1396.25 ± 4.07+5.00
MNIST Addition (raw digit-pair exact match)
Single-digitPA-I77.01 ± 1.4578.74 ± 1.07+1.73
Two-digit IDPA-II59.91 ± 1.0961.33 ± 0.55+1.42
Three-digit OODPA-II46.19 ± 0.4047.47 ± 0.65+1.28
AuthorsOpenReview ↗

W2 and Q1 (feedback): Symbolic information used by each task

Response: SCULPT takes directional constraints or feasible solver targets as input; a Boolean accept/reject signal alone does not determine an update. Routing tasks compare reference and predicted paths. Sudoku generates feasible boards from the current prediction scores and row, column, and block constraints. Addition generates consistent digit combinations from the current prediction scores and the observed sum. The latter two tasks do not access ground-truth individual digit labels during adaptation. We will make these inputs more explicit in the task definitions and experimental setup, specifying the scope as symbolic feedback interfaces that provide directional constraints or feasible targets.

AuthorsOpenReview ↗

Q1 (update rules): Adaptation gains under matched feedback

Response: The added simpler update baselines use the same adaptation data, feedback information, and solver-call budgets as SCULPT. PG-Adam and Self-training in the paper provide further gradient-based controls (Table R2), while Figure 3 compares SCULPT and Self-training across matched solver-call budgets. In the covariance ablations, retaining only diagonal covariance or current-batch statistics reduces performance, demonstrating the role of feature correlations and cross-batch feedback accumulation in using solver targets (Table R3).

Table R2. Gradient-based controls reported in Appendix Tables 6 and 7 of the paper. Routing tasks use PG-Adam; visual tasks use Self-training. Methods use frozen features and the data and checker budgets specified in the paper. Results are mean ± SD (%); Δ is SCULPT minus the baseline mean (pp), computed from the displayed paper means.

Task / settingGradient baselineBaseline (%)SCULPT (%)Δ (pp)
Dijkstra (exact path match)
12 × 12 IDPG-Adam78.75 ± 1.8287.13 ± 3.66+8.38
18 × 18 OODPG-Adam72.50 ± 6.8282.83 ± 3.52+10.33
Warcraft (exact path match)
12 × 12PG-Adam37.88 ± 1.2136.90 ± 0.96-0.98
18 × 18PG-Adam18.84 ± 1.0719.26 ± 0.71+0.42
24 × 24PG-Adam11.94 ± 0.9711.88 ± 0.83-0.06
30 × 30PG-Adam7.50 ± 0.598.22 ± 0.54+0.72
Visual Sudoku
Raw cell accuracySelf-training63.95 ± 0.8570.49 ± 0.33+6.54
Decoded whole-board accuracySelf-training88.75 ± 4.1996.25 ± 4.07+7.50
MNIST Addition (raw digit-pair exact match)
Single-digitSelf-training70.82 ± 1.2278.74 ± 1.07+7.92
Two-digit IDSelf-training45.21 ± 1.9361.33 ± 0.55+16.12
Three-digit OODSelf-training30.04 ± 1.7047.47 ± 0.65+17.43

Table R3. Covariance-update ablations. Sudoku reports cell accuracy under head perturbation; Addition reports raw digit-pair exact-match accuracy for single-digit addition. Results are mean ± SD (%). Drops are relative to the full SCULPT model in this ablation study (pp).

SettingSudoku (%)Drop (pp)Addition (%)Drop (pp)
Full SCULPT70.50 ± 0.34—78.74 ± 1.07—
Diagonal covariance only58.44 ± 0.5212.0665.86 ± 1.4212.88
Current-batch statistics only66.93 ± 0.543.5754.15 ± 2.3824.59
AuthorsOpenReview ↗

Q3: Runtime and memory relative to gradient-based adaptation

Response: We measured runtime and memory for SCULPT, Self-training, and OGD using the same frozen features, initial head, minibatch sequence, and solver-call budget. Each method generates symbolic targets online from its own current predictions. Target generation accounts for most of the adaptation time, while head updates take only a small fraction. SCULPT has lower total adaptation time than both gradient-based controls in these measurements, with the differences arising mainly during target generation; head-update times are of a similar order across the three methods (Tables R4 and R5).

Table R4. Sudoku adaptation time and memory usage from the same measurement campaign. Results are mean ± sample SD. Each Δ is SCULPT minus the corresponding baseline mean, computed before rounding and reported to three decimal places; negative values indicate lower usage by SCULPT.

MethodAdaptation time (s)Δ time (s)Incremental GPU peak (MiB)Δ GPU (MiB)Sampled CPU peak RSS (MiB)Δ CPU (MiB)
SCULPT18.633 ± 0.526—0.549 ± 0.000—1129.952 ± 8.289—
Self-training23.113 ± 1.077-4.4800.271 ± 0.000+0.2781250.066 ± 12.291-120.115
OGD21.859 ± 1.095-3.2260.676 ± 0.000-0.1271104.339 ± 3.532+25.613

Each method was run over five seeds on an RTX 4090, measured sequentially in separate processes after warmup. Each adaptation run uses 20 steps with 4 boards per step and 160 CP-SAT calls; the solver uses 8 workers with a 0.25 s limit per call. Timing includes adaptation initialization, symbolic target generation, and head updates, and excludes feature preparation, warmup, and final evaluation. Self-training uses Adam and OGD uses projected SGD. SCULPT and Self-training retain FP32; OGD retains FP64 from its original configuration.

GPU memory is peak allocated memory during adaptation minus the pre-adaptation baseline. CPU memory is peak process RSS sampled every 2 ms, including resident data and libraries.

Table R5. Adaptation time breakdown. Results are mean ± sample SD (s); each Δ is SCULPT minus the corresponding baseline mean, computed before rounding and reported to three decimal places. Target generation includes batch preparation, prediction scoring, perturbation, solver calls, and consensus processing. Head updates include statistics accumulation, solving, and projection, or loss computation, backpropagation, and optimizer updates. Initialization and other loop overhead are included in the total time in Table R4.

MethodTarget generation (s)Δ target generation (s)Head update (s)Δ head update (s)
SCULPT18.580 ± 0.490—0.048 ± 0.044—
Self-training23.065 ± 1.082-4.4840.046 ± 0.023+0.002
OGD21.826 ± 1.095-3.2460.031 ± 0.001+0.018
Reviewer_M857OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes SCULPT, a backpropagation-free method for adapting a linear prediction head on top of a frozen representation using feedback from non-differentiable checkers or solvers. The key observation is that, under a linear head, a violated output constraint aᵀy ≥ b returned by a checker corresponds exactly to a half-space ⟨aφ(x)ᵀ, W⟩_F ≥ b in head-parameter space, so the checker’s feedback directly yields an update direction. SCULPT combines two update branches:

a reliability-gated moment direction aggregated from violated directional cuts; a covariance-aware ridge solve on consensus-filtered perturb-and-MAP solver targets.

An optional trust projection limits drift from the deployed head. The method is evaluated on Dijkstra routing, Warcraft shortest paths, clue-free visual Sudoku, and corrupted MNIST Addition, together with a backbone-transfer study using CNN, ResNet-18, and ViT encoders.

Critical strengths

The setting is practical: adapting only the head of a deployed model with feedback from a non-differentiable checker, without retraining the backbone or differentiating through the checker. The half-space formulation is simple and handles two types of feedback (directional constraints and feasible targets) within one framework. The reporting is transparent: complete baseline tables, significance tests, and open discussion of the settings where SCULPT underperforms.

Critical weaknesses

Insufficient ablation. Section 3.2 introduces many hyperparameters (e.g., βs, κ, s₀, T_s, β_h, μ, T_h, τ, η_f, η_p, η_h, and the choice of preconditioner), and they are tuned separately for each task (Appendix A.1). However, there is no ablation showing which components actually matter, such as the consensus gate, the history-agreement weighting, or the capped projection. Since the plain Perceptron baseline performs close to SCULPT on Dijkstra (Table 6), it is unclear whether the gains come from the proposed design or from per-task tuning. Please provide component-wise ablations and a hyperparameter sensitivity analysis.

inconsistent results. Table 2 reports 40.36% for the CNN backbone on Warcraft 12×12, whereas Table 1 reports 36.90% for the same setting. I could not find an explanation for this difference in the main text. The two tables appear to use different hyperparameter configurations (Appendix A.1 vs. Table 4), but this is not stated. Notably, 40.36% would exceed the best baseline in Table 1, DataSP (40.06%). Please clarify which configuration is used where and why the two results differ.

Weak backbones in the backbone study. Some backbones produce very weak features. For example, ResNet-18 reaches only 0.19% on two-digit MNIST Addition (and 0.01% on three-digit), with almost no gain from SCULPT. In such cases the bottleneck is the representation rather than the adaptation method, so these results say little about SCULPT’s generality across backbones. Please use reasonably trained backbones or report their linear-probe accuracy.

Questions for authors

N/A

5 comments
AuthorsOpenReview ↗

W1(a): Component-wise ablation

We added component-wise ablations on Dijkstra, Sudoku, and MNIST Addition. Under noisy Dijkstra feedback, fixing the update direction or removing the soft consensus gate, historical agreement weighting, or projection mechanisms reduces mean accuracy (Table R1a). On Sudoku and Addition, retaining only diagonal covariance or current-batch statistics substantially reduces performance, supporting the use of feature correlations and feedback accumulated across batches (Table R1b).

Table R1a. Dijkstra component ablations on 18 × 18 OOD maps with 20% feedback corruption. The metric is exact path match, reported as mean ± SD (%). Drop is the full SCULPT mean minus the ablation mean, in percentage points (pp).

SettingExact path match (%)Drop from full SCULPT (pp)
Full SCULPT53.00 ± 9.20—
Fixed coherent direction (soft gate retained)41.00 ± 4.1812.00
Fixed conflicting direction50.67 ± 5.252.33
No soft consensus gate52.50 ± 6.430.50
No historical agreement weighting48.17 ± 3.794.83
No projection coefficient cap52.00 ± 9.121.00
No projection term52.00 ± 10.281.00

Table R1b. Covariance-branch ablations. Sudoku reports digit accuracy under head perturbation; Addition reports raw digit-pair exact-match accuracy for single-digit addition. Results are mean ± SD (%). Drops relative to full SCULPT are computed before rounding and reported to two decimal places (pp).

SettingSudoku (%)Sudoku drop (pp)Addition (%)Addition drop (pp)
Full SCULPT70.50 ± 0.34—78.74 ± 1.07—
No target filtering or weighting70.49 ± 0.330.0178.70 ± 1.110.04
Diagonal covariance only58.44 ± 0.5212.0665.86 ± 1.4212.88
Current-batch statistics only66.93 ± 0.543.5754.15 ± 2.3824.59
No overall trust projection70.49 ± 0.330.0178.70 ± 1.070.04
AuthorsOpenReview ↗

W1(b): Hyperparameter sensitivity

Response: We varied one parameter at a time on Dijkstra, Visual Sudoku, and Warcraft, keeping the backbone, adaptation budget, and remaining parameters fixed. Update step sizes, the branch-switching threshold, and overly restrictive trust radii have larger effects on performance; the other parameters have smaller effects within the tested ranges (Table R2). All main-experiment configurations remain unchanged.

Table R2. SCULPT hyperparameter sensitivity over five seeds per setting. Accuracy ranges span the tested parameter values; maximum drops are relative to the reference accuracy for each task (pp).

Task (reference accuracy)ParameterAccuracy range (%)Max. drop (pp)
Dijkstra OOD (82.83%)Coherent step, ηf\eta_{f}61.83–84.5021.00
Dijkstra OOD (82.83%)Switch threshold, κ\kappa71.83–83.3311.00
Dijkstra OOD (82.83%)Projection coefficient, ηp\eta_{p}74.50–82.838.33
Dijkstra OOD (82.83%)Conflict step, ηh\eta_{h}76.17–83.006.67
Dijkstra OOD (82.83%)History EMA, βh\beta_{h}82.00–82.830.83
Dijkstra OOD (82.83%)Other gate/history/cap parameters*79.67–83.333.17
Visual Sudoku (70.51%)Trust radius58.91–70.5111.60
Visual Sudoku (70.51%)Target scale69.03–70.551.49
Visual Sudoku (70.51%)Ridge coefficient69.90–70.640.61
Visual Sudoku (70.51%)Threshold / perturbation*70.48–70.540.03
Warcraft 18 × 18 (19.26%)Directional update scale16.18–20.523.08
Warcraft 18 × 18 (19.26%)Covariance weight / projection / ridge*18.92–19.640.34

* Grouped entries summarize separate one-at-a-time sweeps of related parameters.

AuthorsOpenReview ↗

W2: Differences between Tables 1 and 2

Response: The two tables use different experimental protocols. Table 1 presents the main baseline comparison using the configuration in Appendix A.1, with an Adam step size of 0.010 and a projection coefficient cap τ\tau of 0.080. Table 2 evaluates transfer across backbones and task configurations; its CNN uses the configuration in Table 4, with an Adam step size of 0.020 and τ\tau of 0.050. The configuration selected on the source setting is then fixed for the target settings. We will identify the configuration source in each table caption.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.