acceptodds
Under review as a conference paper at ICLR 2027

Calibrated Tool Use via Learned Adapters to Compiled Neural Programs

Abstract

Language models typically call a tool with a single set of arguments, even when several readings of the input remain plausible. This discards useful information, and as a result, reinforcement learning for tool use must estimate credit by sampling many trajectories. We propose SPACE-LLM: Sampling-free Probabilistic Adapters for Calibrated Execution in Large Language Models. Learned adapters over LLM hidden states predict distributions over the arguments of a compiled neural program, a symbolic program written as tensor operations that execute inside the model's own forward pass. Retaining these argument belief distributions allows inference to combine support for an answer across ambiguous readings and enables training to compute analytic gradients from a ground truth answer. We show that maintaining belief distributions at inference, rather than committing to one reading, raises accuracy and improves calibration without a calibration-specific loss. Using only answer-level supervision, we observe comparable or stronger out-of-distribution generalization than fully supervised finetuning and a comparable Brier score, from a single pass without sampling or a recalibration step. On identical training sets, SPACE matches or exceeds GRPO accuracy with a lower Brier score more than 50 times faster, and the tool runs inside the model's forward pass. These advantages are limited to domains in which maintaining the factored belief state is tractable, the answer carries enough information to credit each argument, and the order of operations is known in advance.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
neuro-symbolic learning, large language models, differentiable programming, probabilistic inference, tool use, uncertainty calibration, out-of-distribution generalization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MqwROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SPACE-LLM, which reads distributions over program arguments from intermediate LLM/VLM hidden states and executes a fixed symbolic program as tensor operations inside the forward pass. Answer supervision trains the reader through analytic gradients, and a learned result token allows the upper model layers to verbalize the output. Across two backbones and five benchmarks, the approach learns effectively on CLUTRR, CUB-200, and Super-CLEVR, but fails on GSM8K and SATBench. The paper compares soft and committed readouts, SFT and GRPO, training time, probabilistic scores, and several architectural and inference controls. Its main contribution is a concrete account of when internal differentiable execution provides efficient answer-supervised learning.

Critical strengths

1. Sound methodology. The paper presents a coherent approach to answer-supervised tool integration.

2. Extensive experiments. The paper includes comprehensive evaluations and controls to support its findings.

Critical weaknesses
  • Limited novelty. Probabilistic neurosymbolic inference and end-to-end learning from answer supervision have been explored in DeepProbLog (NeurIPS 18) and Scallop (PLDI 23), while Neural Comprehension / CoNN (ICLR 24) already integrates compiled symbolic modules into language models. SPACE’s contribution therefore appears primarily architectural, but its incremental benefit over these established designs remains insufficiently isolated. The Soft/Commit controls support retaining uncertainty, yet do not establish the specific advantage of executing programs inside the LLM and injecting their results into its residual stream. The authors should clarify this distinction and compare against an LLM encoder connected to an existing differentiable probabilistic reasoner under matched supervision and training budgets.

  • Unreliable method. Answer-only supervision rewards incorrect arguments whenever they produce the correct answer. This risk is evident in CUB-200, where learned attribute accuracy does not exceed the majority baseline despite strong final classification. Thus, successful predictions do not establish semantically faithful tool arguments.

  • Lack theoretical justification. The only formal guarantee is a standard Bayes-optimality result assuming correct argument distributions and exact marginalization, conditions not established for the evaluated models. It does not guarantee that answer-only training learns faithful arguments or produces calibrated predictions.

  • Limited related-work discussion. The brief coverage of related work makes it difficult for readers to understand the broader research context and distinguish the proposed approach from existing methods.

Questions for authors
  1. What benefits specifically arise from in-model execution and residual-stream result injection compared with an LLM encoder connected to an existing differentiable probabilistic reasoner under matched supervision and training budgets?

  2. How can answer-only supervision distinguish faithful arguments from incorrect arguments yielding the same answer, particularly given the near-majority-baseline attribute accuracy on CUB-200? What supports calibration when argument beliefs are inaccurate or marginalization is approximate?

  3. Could the authors expand the related-work discussion and clarify SPACE’s methodological differences from DeepProbLog, Scallop, and Neural Comprehension / CoNN?

1 comment

Thank you for pressing on what answer-only training can identify. That question became Proposition 2, which now explains both of the paper's failures.

Limited novelty

"Probabilistic neurosymbolic inference and end-to-end learning from answer supervision have been explored in DeepProbLog (NeurIPS 18) and Scallop (PLDI 23)"

We agree that SPACE shares the learning principle of these methods. The submission cited all three in Section 2 but did not state the difference, and it now does (p. 2, l. 107): "SPACE shares the learning principle of these methods: neural predictions enter a probabilistic program, and the marginal likelihood of the answer trains them." The difference is where the program runs. Those systems run the program in a separate symbolic engine, while SPACE compiles it into a pretrained VLM's forward pass and writes the result back, so that the model states it. CoNN does place a compiled module inside a language model, but it joins the two outputs with an argmax (their Eq. 1) and trains by standard SFT, so no gradient passes through the tool.

Q1, in-model execution

"What benefits specifically arise from in-model execution and residual-stream result injection compared with an LLM encoder connected to an existing differentiable probabilistic reasoner"

The accuracy gains come from the analytic gradient through the program. In-model execution adds no accuracy of its own, and we do not claim that it does. An LLM encoder feeding an exact differentiable reasoner, as DSR-LM does, computes the same distribution and the same gradient as SPACE, so we did not run it separately. In-model execution adds two practical things: one GPU pass with no external call, and a result the model says in its own words. To isolate the gradient, as you asked, we fixed the adapter, program, rules and priors and changed only the training signal. Sampled rewards lose to the exact gradient in distribution on all four dataset and backbone pairs, and out of distribution on three, with a tie on the fourth (Appendix C.3, Table 11, p. 25).

Unreliable method

"Answer-only supervision rewards incorrect arguments whenever they produce the correct answer."

Correct. Answers alone cannot fix every argument, because a wrong reading can reach the right answer by coincidence. Proposition 2 now states this (p. 4, l. 206-209): "answer-only training cannot prefer the true reading over another reading with the same answers." On CLUTRR the answers mostly fix the arguments, and edges trained from answers reach 0.904 / 0.869 OOD beside 0.910 / 0.891 for SFT on edge labels. On GSM8K many readings give the same answer, and SPACE reaches only 11.1% to 15.5% even with the wiring given (Appendix H, p. 41).

CUB-200 attributes

"learned attribute accuracy does not exceed the majority baseline despite strong final classification"

True, and we make no claim that the attributes are literal. SPACE's attributes agree with the per-image annotations at 0.355 / 0.359, near the 0.376 of always picking the most common value (Appendix F, p. 36). The annotations are hard for any method to match. The most common value for an image's own species matches that image's annotation only 57.3% of the time, and SFT trained directly on the annotations matches them at 0.511 / 0.487. SPACE's attributes work as a code for the species, and naming the species is the task. There SPACE scores 0.864 / 0.818 against SFT's 0.447 / 0.409.

Calibration

"What supports calibration when argument beliefs are inaccurate or marginalization is approximate?"

Calibration is learned together with the beliefs, from the same answer loss. That loss is a proper scoring rule, so at its optimum the answer probabilities are calibrated whether or not the arguments are literal (Proposition 1, p. 4, l. 200). We concede that calibration still depends on the beliefs. The answer probabilities are computed from them, and the proposition gives no guarantee short of the optimum. Under approximation, Super-CLEVR leaves out a median of 2.1% / 4.2% of the probability mass and never renormalizes, so its confidence can only be too low. Moving all of that mass onto the predicted answer changes ECE by 0.03 at most (Appendix G.1, p. 39).

Theory

"The only formal guarantee is a standard Bayes-optimality result"

Section 3.1 now carries the two propositions above, with proofs in Appendix N (p. 52).

Related work

"Could the authors expand the related-work discussion"

Section 2 is more detailed (p. 2, l. 107, to p. 3, l. 132), and a new Appendix L (p. 47) sets SPACE beside DeepProbLog, Scallop, DSR-LM and A-NeSI, beside RASP, Tracr and CoNN, and beside tool use by generation and reinforcement learning (PAL, Toolformer, ViperGPT).

Reviewer_gfUbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SPACE-LLM, a framework for learning tool use through differentiable execution of compiled symbolic programs. Instead of generating discrete tool arguments, a learned adapter predicts probability distributions over arguments from intermediate LLM hidden states. A fixed, domain-specific program propagates these distributions to produce an answer distribution, allowing the adapter to be trained directly from answer-level supervision through analytical gradients. The program's output is subsequently injected into the LLM through a learned virtual token for natural-language generation.

The authors evaluate SPACE on five benchmarks using two pretrained language models, comparing against supervised fine-tuning (SFT) on tool arguments and GRPO using answer rewards. SPACE achieves competitive or superior accuracy on CLUTRR, CUB-200, and Super-CLEVR, often with substantially lower training cost and favorable Brier scores. However, it fails almost completely on GSM8K and SATBench, illustrating limitations of answer-level supervision when program structure is unknown or the output provides insufficient information for credit assignment.

Critical strengths

(1) The central technical contribution is compelling: by integrating a fixed symbolic computation into the model's differentiable computation graph, the authors can learn argument distributions without intermediate argument supervision or sampling-based credit assignment.

(2) The results demonstrate that this approach can be substantially more computationally efficient than reinforcement learning in suitable domains.

(3) The authors also provide some evidence of out-of-distribution generalization to larger problem instances.

Critical weaknesses

(1) One of the central claims is that maintaining distributions over tool arguments improves inference compared with committing to individual arguments. However, the Legal-MAP ablation in Table 2 significantly qualifies this conclusion. On CLUTRR, Legal MAP achieves 78.1% versus 78.0% for Soft on Qwen, and 74.9% versus 73.8% on Gemma. Thus, enforcing the symbolic program's legality constraints appears sufficient to explain essentially the entire improvement over naive commitment. On CUB-200, Soft improves over Commit by less than one percentage point in the main experiments. Super-CLEVR provides stronger evidence for marginalization itself, but its Legal-MAP baseline is approximate, selecting from only 128 sampled assignments. It is unclear to me how much of the remaining performance gap would survive a stronger constrained decoder.

(2) The empirical comparisons do not fully isolate the proposed method's advantages from architectural constraints and computational accounting. SPACE relies on substantial domain-specific engineering. Each benchmark requires a program schema and specialized tensor execution kernels. Super-CLEVR additionally uses lexical and structural priors, whose importance is demonstrated by the ablation in Appendix C.1: removing these priors reduces Qwen's OOD accuracy from approximately 62% to 11%. Moreover, the SFT and GRPO baselines must generate complete textual tool calls, whereas SPACE predicts arguments through a structured adapter with domain-specific constraints. A substantial fraction of baseline errors are invalid tool calls. This complicates attributing performance differences specifically to differentiable training or probabilistic marginalization rather than to the different argument-generation interfaces.

Questions for authors

(1) How much of SPACE's advantage survives stronger structurally constrained baselines? The Legal-MAP control essentially matches Soft on CLUTRR. Could the authors evaluate a stronger Legal-MAP search on Super-CLEVR and compare against SFT/GRPO with grammar-constrained tool generation and comparable domain-specific lexical constraints? Such experiments would help establish which improvements are attributable specifically to marginalization and differentiable training rather than to enforcing valid tool arguments.

(2) What do these symbolic programs look like? How difficult is it to write them? Could it be automated?

(3) How general is this technique? I didn't come away with a good sense of where it could or could not be applied?

2 comments

Thank you for helping us tighten our claims with the Table 2 results. Your reading of that table is now the paper's third contribution.

W1

"enforcing the symbolic program's legality constraints appears sufficient to explain essentially the entire improvement over naive commitment"

You were right, and so we've rebuilt the claim around it. The abstract now reads (p. 1, l. 19-22): "Because SPACE executes the program over the whole distribution of argument readings, the answer is conditioned on the program accepting the reading, a built-in form of constrained decoding." Contribution 3 adds (p. 2, l. 93-97): "Most of SPACE's gain over committing to a single reading comes from this." Over three seeds, Legal MAP on CLUTRR scores 0.7 / 0.9 points above SPACE (Qwen / Gemma).

Q1, Legal MAP

"Could the authors evaluate a stronger Legal-MAP search on Super-CLEVR"

Following your suggestion, we replaced the 128-sample search with an exact one (Appendix G.1, Table 27, p. 40). It proves its answer on 94% to 95% of questions. On the rest, two answers are too close to separate. Your suspicion held. The 128-sample version sat 6.6 / 5.8 points below SPACE. The exact one sits 1.0 / 0.8 below. What survives is the Super-CLEVR specific part counting, where summing is worth 12.3 / 10.6 points.

Q1, constrained baselines

"compare against SFT/GRPO with grammar-constrained tool generation and comparable domain-specific lexical constraints"

Yes, we did this comparison (Appendix C.2, Table 9, p. 24). On CLUTRR, the legality rule adds 6 to 14 OOD points. GRPO then ties SPACE on Qwen (+0.7 points [-2.1, 3.5]) and leads by 3.3 points on Gemma [-0.1, 6.8], while SFT leads by 5.6 on Gemma [2.4, 8.7] and trails by 16.5 on Qwen. The Results now say (p. 7, l. 349): "When the text baselines enforce the same program legality through constrained decoding, GRPO matches SPACE on Qwen and is slightly ahead on Gemma." Still, SPACE gets there in about ten minutes of training, against nine to ten hours for GRPO. On Super-CLEVR, with an execution check that returns an answer on 99.0% to 99.9% of questions, SPACE leads GRPO by 5.8 / 10.8 points. SFT leads SPACE by 3.5 / 1.4 overall and trails it by 12 to 13 on relation chains. With legal sampling, GRPO's CLUTRR OOD Brier score also drops below SPACE's, so the abstract now says "often with a lower Brier score" (p. 1, l. 26).

W2, interface

"This complicates attributing performance differences specifically to differentiable training or probabilistic marginalization rather than to the different argument-generation interfaces."

Agreed. The text comparison can't separate them, and the original submission did not identify this. To separate them, we held the interface fixed and changed only the training signal. We trained the same SPACE adapter, with the same program, rules, priors and data, from sampled rewards in place of the exact gradient. Neither arm can make an invalid call. OOD accuracy, mean of three seeds (Appendix C.3, Tables 11-12, p. 25):

CLUTRR QwenCLUTRR GemmaSC QwenSC Gemma
Sampled reward (tuned)0.7890.6280.6110.600
Exact gradient (SPACE)0.7880.7480.6580.651

In distribution the exact gradient wins all four cells, by 2.6 to 29.9 points. The one OOD tie, on CLUTRR Qwen, needed a nine-run tuning sweep.

W2, priors

"removing these priors reduces Qwen's OOD accuracy from approximately 62% to 11%"

Yes, and the dependence is stronger than originally claimed. Without the priors the correct answer is reachable on only about 30% of training questions, and OOD accuracy is 0.125 / 0.108 (Appendix G.2, Table 28, p. 41). The Discussion now says (p. 9, l. 466): "These priors are domain engineering that SPACE needs and a text tool call does not." Both ablation arms above use them.

Q2

"What do these symbolic programs look like? How difficult is it to write them? Could it be automated?"

Appendix K (pp. 45-46) now lists the five programs, their size and the parts written by hand. Each is a schema of argument domains, a fixed table, composition rules and skeletons that we compile into tensor operations, plus hand-written kernels. We haven't automated the kernels.

Q3

"I didn't come away with a good sense of where it could or could not be applied"

A fair complaint. SPACE works when the algorithm is known exactly in advance and each argument can be read from a local part of the input. Our three working datasets fit: one sentence gives a kinship relation (CLUTRR), one image gives each bird attribute (CUB-200) and one object gives its properties (Super-CLEVR). We expect the same for other attribute-based image classification and for lookups in tables or knowledge graphs. It fails when the model must also choose the order of operations (GSM8K), or when a single yes/no answer depends on every argument at once (SATBench). Proposition 2 (p. 4, l. 206-209) states this condition formally.

Thank you for addressing my questions and adjusting your claims accordingly. I raise my score.

Reviewer_pKvGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper claims that a language model usually calls a tool by writing a single set of arguments as text. This commits to one reading of the input and throws away the model's uncertainty about it. The paper posits that this is sub-optimal in two ways - (i) At inference, probability mass on alternative readings may actually support the correct answer, so committing early can return the wrong one, and (ii) At training, reinforcement learning for tool use (such as GRPO) must estimate credit by sampling many trajectories, which is slow and noisy. To overcome these limitations, the paper presents SPACE, in which a learned adapter predicts distributions over the arguments of a symbolic program written as fixed tensor operations that runs inside the model's own forward pass. The key idea is that the whole program is differentiable, which buys two things at once: (i) at inference the beliefs are propagated through the program and the answer probabilities are summed, so support is combined across ambiguous readings (the "Soft" readout, as opposed to a "Commit" readout that takes the argmax first); and (ii) at training an analytic gradient flows from a ground-truth answer alone, with no argument labels and no sampling. Experiments span five benchmarks (CLUTRR, CUB-200, Super-CLEVR, GSM8K, SATBench) on two VLM backbones (Qwen3.5-4B and Gemma-4-E4B-it), against supervised finetuning (SFT) and GRPO. On the first three, SPACE matches or exceeds SFT test accuracy, beats GRPO by 4–5 points, trains 50–85× faster, and is calibrated comparably to SFT without a calibration-specific loss. On GSM8K and SATBench it fails to learn almost entirely (for example 0.025 versus 0.629 for SFT on GSM8K Qwen), which the authors report and dissect openly.

Critical strengths
  • The core idea is clean and, as far as I can tell, genuinely novel for tool use. Instead of writing the tool arguments as text, the adapter keeps a belief over them. The program then runs on that belief inside the forward pass. This leads to a neat design choice of making the program differentiable, which lets inference marginalize over ambiguous readings, and also lets training get an exact gradient from the answer alone.

  • The efficiency gain is large, and it is evidenced rather than just asserted. SPACE needs a single forward and backward pass per question. GRPO needs eight sampled rollouts before every update. Table 5 makes this concrete. On CLUTRR Qwen, SPACE finishes in 9.3 minutes against 596.7 for GRPO. On Super-CLEVR it is roughly thirty minutes against 29–36 hours. The 50–85× headline is backed by matched iso-data and iso-clock budgets on the same H100.

  • The experimental honesty is well above the norm, and it is what most raised my confidence in the positive results. The method flatly fails on two of the five benchmarks, and the paper says so in the abstract, the results, and the limitations. It then explains each failure mechanistically rather than hiding it. The abstract even states the three conditions under which the approach is expected to help. This kind of scoping is rare, and it makes the wins easier to trust.

Critical weaknesses
  • The title and framing promise more generality than the method delivers. "Calibrated tool use" suggests a broad capability. In practice SPACE works on three of five benchmarks. Those three are exactly the cases where the input already defines the program structure and the readings are local. The two benchmarks where real compositional structure must be recovered, ie (i) arithmetic wiring in GSM8K and (ii) search in SATBench, are the ones where it collapses. So the contribution is narrower than "tool use," and the framing should be read with that in mind.

  • Every tool is a hand-written, compiled tensor program, and tool selection and composition are explicitly out of scope. Real tool use is largely about choosing among many tools and chaining them. SPACE does neither. Someone has to author the domain-specific kernels for each new program. This is a steep practical cost. It also sits awkwardly with the "tool use" framing, which usually implies a general interface to many tools.

  • In all honesty, the baseline comparison is not fully balanced. One seed each of SFT and GRPO is compared against three SPACE seeds. The methods also use different learning-rate schedules, and the comparison is iso-clock rather than iso-FLOPs. The authors are upfront about all of this. Even so, the headline "matches SFT" and "50–85× faster than GRPO" claims rest on baselines that are not seed-matched.

Questions for authors
  • The iso-clock results use one seed for SFT and GRPO but three for SPACE. Could the baselines be run with matched seeds, at least on the three domains where SPACE wins? It would be good to know how much of the "matches SFT" gap sits within baseline seed variance.

  • On GSM8K the failure is pinned on the adapter not recovering the operation wiring from answers alone. Is this fundamental to the factored belief, or is it just a missing structure signal? Concretely, if the operation order were supplied, would SPACE recover SFT-level accuracy there?

  • Can some of the numbers in Figures 4 and 5 be represented as Tables? It is hard to read the values from the images.

2 comments

Thank you for a generous and exact review. Your seed question helped us correct our claim on SFT's test accuracy.

Framing

"the contribution is narrower than 'tool use,' and the framing should be read with that in mind"

By "tool use" we mean calling one fixed tool whose arguments must be read from uncertain input. We kept the title because the abstract's last sentence limits the claim to domains where "maintaining the factored belief state is tractable, the answer carries enough information to credit each argument, and the order of operations is known in advance." Proposition 2 (p. 4, l. 206-209) now makes the middle condition formal. It explains your observation that SPACE wins where readings are local and collapses on GSM8K and SATBench.

Hand-written programs

"Someone has to author the domain-specific kernels for each new program. This is a steep practical cost."

Agreed, and the limitations say so (p. 9, l. 479-482): "Each new tool also has an engineering cost: we write its tensor kernels by hand and compile only their wiring (Appendix K), and tool selection or composition is outside this study." Appendix K (pp. 45-47) lists each program, its size and its hand-written parts. We did not try to automate this step and leave it to future work. The appendix outlines one possible route (p. 47, l. 2519). On Super-CLEVR the adapter also needs lexical priors, and the same passage now states that without them "training does not start".

Matched seeds

"Could the baselines be run with matched seeds, at least on the three domains where SPACE wins?"

Following your request, we trained two more seeds of every SFT and GRPO baseline on CLUTRR, CUB-200 and Super-CLEVR, changing only the seed. The paper now reports three-seed means for every method there (Table 7, p. 22, and Appendix C.4, Table 13, p. 26). As in the original submission, GRPO on CUB-200 didn't fit the training budget. The table gives OOD accuracy, with SPACE minus the baseline and a 95% interval over seeds and questions. Its baselines write the tool call as free text, as in the original. Constrained decoding for them is a separate control (Appendix C.2, Table 9, p. 24), and under it SPACE's CLUTRR lead over GRPO disappears.

CellBaselineSPACESPACE − baseline
CLUTRR Qwen, SFT0.5650.788+22.3 [+10.3, +32.4]
CLUTRR Qwen, GRPO0.7230.788+6.5 [+3.4, +9.8]
CLUTRR Gemma, SFT0.6870.748+6.1 [+1.5, +11.2]
CLUTRR Gemma, GRPO0.7100.748+3.8 [+0.5, +7.2]
CUB-200 Qwen, SFT0.4470.864+41.7 [+40.0, +43.2]
CUB-200 Gemma, SFT0.4090.818+40.9 [+39.4, +42.5]
Super-CLEVR Qwen, SFT0.6810.658−2.3 [−3.6, −1.0]
Super-CLEVR Qwen, GRPO0.6300.658+2.9 [+0.4, +5.9]
Super-CLEVR Gemma, SFT0.6290.651+2.2 [+0.8, +3.7]
Super-CLEVR Gemma, GRPO0.4570.651+19.4 [+15.9, +23.5]

You asked how much of the gap sits within baseline seed variance, and two leads fall within 1.4 baseline SDs (over SFT on CLUTRR Gemma, over GRPO on Super-CLEVR Qwen). However SFT does lead on Super-CLEVR Qwen by 2.3 points, so the Results now read "within about two points of unconstrained SFT's test accuracy" (p. 6, l. 319), up from one.

Schedules and FLOPs

"The methods also use different learning-rate schedules, and the comparison is iso-clock rather than iso-FLOPs."

Both remain true. Each method keeps its own training recipe, so the schedules differ, and the limitations now read (p. 10, l. 492): "As learning-rate schedules differ, Appendix C.1 also reports checkpoints selected on validation data." We compare by clock because training time is the cost a user pays (p. 5, l. 242). We added no iso-FLOPs run, since prompt lengths, batches, backpropagation paths and rollouts all differ and we have no FLOP count we trust across methods. The protocol states "Equal clocks do not establish equal FLOPs" (p. 19, l. 1025), and Table 7 (p. 22) gives every comparison at equal training data too.

GSM8K

"if the operation order were supplied, would SPACE recover SFT-level accuracy there?"

We ran the test you proposed, and supplying the order does not bring SPACE to SFT-level accuracy. With the reference order and wiring fixed in training and at test, so that answers train only the number pointers, accuracy on test questions with a reference program rises from 2.5% to 15.5% (Qwen) and from 2.2% to 11.1% (Gemma). Argument SFT gets 72.8% and 71.3%. Supervising the order in training alone leaves accuracy at 2.6%. Both limits you named are present, and the Results now say (p. 8, l. 393): "Giving the operation order does not close this gap (Appendix H)."

Figures 4 and 5

"Can some of the numbers in Figures 4 and 5 be represented as Tables?"

Yes. Appendix M (Tables 32-35, pp. 49-51) gives every plotted value.

All ten comparisons are now seed-matched, and each difference has a 95% interval that excludes zero.

Thank you for the responses to the questions. I will maintain my rating of 3.

Comments on the paper

2 comments

Thank you all for your careful reviews. Each one helped us produce a tighter submission. Reviewer gfUb read our own Table 2 and saw that Legal MAP ties SPACE on CLUTRR, which rewrote our SPACE Soft inference claim. Reviewer MqwR asked what answer-only training can pin down, which helped us state two propositions. Reviewer pKvG asked for seed-matched baselines. Running them corrected our claim on SFT vs SPACE test accuracy.

Here is each requested change and where it sits in the revised submission:

Reviewer pointChangeLocation
gfUb W1: legality explains the gain over CommitClaim restated as built-in constrained decodingAbstract, l. 19-22. Contribution 3, p. 2, l. 93-97
gfUb Q1: stronger Legal MAPExact search replaces 128 samplesApp. G.1, Table 27, p. 40
gfUb Q1: constrained SFT and GRPOGrammar, legality, lexical and execution constraints applied and measuredApp. C.2, Table 9, p. 24. Results, p. 7, l. 349-357
gfUb W2, MqwR Q1: interface or training signalSame adapter trained from sampled rewardsApp. C.3, Tables 11-12, p. 25
gfUb W2: lexical priorsStated as a limitationp. 9, l. 466 and 480-482. App. G.2, pp. 40-41
gfUb Q2, pKvG W2: hand-written programsAll five programs listedApp. K, pp. 45-46
MqwR: novelty, related workSection 2 extended, new appendixp. 2, l. 107, to p. 3, l. 132. App. L, p. 47
MqwR: faithful arguments, theoryTwo propositions with proofsp. 4, l. 200-209. App. N, p. 52
pKvG W3, Q1: seed-matched baselinesThree seeds of SFT and GRPOTable 7, p. 22. App. C.4, Table 13, p. 26
pKvG Q2: GSM8K with the order suppliedNew experimentp. 8, l. 393. App. H, p. 41
pKvG Q3: Figures 4 and 5 as tablesEvery plotted value tabulatedApp. M, Tables 32-35, pp. 49-51

Two of our claims got narrower:

  • Text GRPO decoded under the CLUTRR legality rule now ties SPACE there.
  • On Super-CLEVR Qwen the three-seed mean puts SFT 2.3 points ahead, so the Results say "within about two points" where the submission said one.

The main claims still hold after every new control. Trained from answers alone, SPACE matches GRPO's accuracy in about 2% of its training time and comes close to argument SFT without any argument labels. Train the same adapter from sampled rewards in place of the exact gradient and accuracy drops everywhere except CLUTRR Qwen OOD, where the two tie.

Dear reviewers,

Thank you for your work reviewing this paper!

The authors have submitted their responses. If you haven't already, please acknowledge them in a comment and adjust your reviews as needed. The discussion period finishes on November 18.

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.