- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SCALELOGIC, a synthetic RL environment that varies proof depth and the set of available logical operators. On Qwen3-4B, the authors find that the number of RL steps needed to reach 90% validation accuracy is well fit by a power law in proof depth, with a larger exponent for richer logical settings. They also report that training on the richer settings transfers better to eight math and science benchmarks, and that curriculum training improves scaling efficiency. The paper includes partial replications across RL algorithms and model families, as well as several controls for prompt length, surface form, alternative rewards, and logical validity.
Critical strengths
-
SCALELOGIC is a useful experimental framework. It provides cheap generation, exact verification, explicit depth control, and more interpretable task variation than standard math or coding datasets. The Z3 audit and randomized predicates, axiom order, and answer order make obvious shortcut explanations less likely.
-
The experimental coverage is unusually strong. The paper studies multiple logical settings, RL algorithms, curricula, model sizes, OOD depths, downstream tasks, proof faithfulness, and several potential confounders. The operator-swap and surface-form controls are particularly helpful.
-
The downstream gains are meaningful rather than confined to the synthetic task. The strongest 4B model improves the eight-benchmark average from 49.39 to 60.05, with gains across most benchmarks. The SATURN comparison also uses the same base model and RL recipe, which is much fairer than copying numbers from the original paper.
Critical weaknesses
-
The evidence supports a power-law fit over the observed range, but not yet a general scaling law. Most points in the central experiment come from one run per configuration, the depth ranges are short and selected after excluding saturated settings, and only power-law and exponential alternatives are compared. The three-seed check covers only the conjunction setting. High R^2 over five or six nearby points is therefore weaker evidence than the abstract suggests. This could be addressed with held-out-depth prediction and uncertainty estimates for the main five exponents.
-
“Expressiveness” is not fully isolated from structural complexity. The hierarchy cumulatively changes rule arity and proof branching; the quantification condition additionally introduces two entities, predicate reuse, and rule reuse. The multi-entity ablation disables quantification and reuse, so it does not separate the effect of universal quantification from reuse. The operator-swap experiment helps for negation and disjunction but does not resolve this final step. A matched-graph ablation that toggles quantification and reuse separately is needed before attributing the exponent increase specifically to logical expressiveness.
-
The claim of more compute-efficient downstream transfer is not established by Figure 3b. That comparison matches approximately 100 optimizer steps, while prompt and response lengths differ substantially across settings. Table 6 itself shows that changing from steps to tokens or FLOPs changes the fitted exponents. The downstream comparison should therefore be repeated at matched generated tokens or FLOPs. The “matched depth” control is also approximate because implication-only uses D = 16 while the other settings use D = 12.
-
The downstream baseline coverage is narrow. The paper cites Logic-RL, SynLogic, Enigmata, Reasoning Gym, and G1, all of which already demonstrate transfer from synthetic verifiable reasoning after RL, but compares experimentally only with SATURN. SCALELOGIC’s controlled scaling analysis remains novel, but the stronger claim that its expressiveness design is responsible for superior transfer would benefit from at least one diverse synthetic-logic baseline, such as SynLogic or Enigmata, retrained with the same model, RL algorithm, and token budget.
-
The architectural conclusion is overstated. Figure 5 shows that performance still approaches random near D_test / D_train = 3. Moreover, SCALELOGIC uses unique derivations, arity two, at most two entities, and a restricted logical fragment. The results show that RL can extend the reasoning horizon within this environment; they do not establish that long-horizon limitations are not architectural.
Questions for authors
-
Do the five power-law models predict convergence cost at held-out depths that were excluded from fitting?
-
What happens when universal quantification and rule reuse are toggled independently while matching rule count, proof size, and prompt length?
-
Does the monotonic downstream ordering remain under matched generated-token or FLOP budgets rather than matched optimizer steps?
-
Can the authors compare against SynLogic, Enigmata, or Logic-RL under the same Qwen3-4B and DAPO setup?
5 comments
Response to Reviewer jxUN (2/4)
Continuation of W1/Q1.
We have also updated Section 4.2 of the main text to highlight these additional analyses:
The power-law model generally outperforms exponential and linear fits in fitting accuracy (Appendix G.1). To further validate the scaling behavior, we fit each model on all but the two deepest data points and predict the held-out depths. The power-law model achieves substantially lower prediction error (11.1% average MAPE, compared to 26.6% for linear and 95.8% for exponential), further supporting the observed power-law scaling behavior.
For uncertainty estimation, we report the standard errors of the fitted exponents in Table 4 of Appendix G.1, with the corresponding error bars shown in Figure 2b of the main paper. We have also updated the caption of Figure 2 to reference these numerical values explicitly:
Error bars denote standard error of the fitted exponent, with the corresponding numerical values reported in Table 4 (Appendix G.1).
Finally, we agree that additional multi-seed experiments across different expressiveness settings would further strengthen the robustness of our scaling results. However, due to computational constraints, a full multi-seed replication across all settings and depths is difficult to complete in the limited time available in the discussion period. In particular, some of the deeper configurations require several days of training on 8 GPUs, making experiments very expensive. We are currently running additional experiments where resources allow, and we will update our response with new results once they become available.
W2/Q2 Ablation on Reuse in Quantification.
We appreciate the reviewer's suggestion and agree that additional ablations would help clarify the effects of quantification and predicate/rule reuse on scaling behavior. To address this, we introduce an additional variant, + Quantification w/o reuse, which preserves universal quantification but disables predicate and rule reuse during problem generation. We compare this variant with the original + Quantification setting and the multi-entity + Disjunction setting as a reference. We have added this ablation study to Appendix G.5 of the revised manuscript, with the results summarized in the table below.
Table: Training steps to reach 90% validation accuracy under different quantification and reuse settings. All three settings use two entities. The + Disjunction results are from the multi-entity control in Appendix G.5.
| + Disjunction | + Quantification w/o reuse | + Quantification | |
|---|---|---|---|
| 6 | 21 | 23 | 49 |
| 8 | 40 | 44 | 98 |
| 10 | 66 | 73 | 186 |
We discuss the key findings in Appendix G.5 of the revised manuscript as follows:
Removing predicate/rule reuse from + Quantification substantially reduces training cost, bringing it close to that of the multi-entity + Disjunction setting. For example, at , + Disjunction with two entities, + Quantification without reuse, and the original + Quantification setting require 66, 73, and 186 training steps, respectively. This is consistent with our construction: without reuse, universally quantified rules effectively behave like entity-specific rules expressed using different natural-language templates, making their reasoning structures similar to those in + Disjunction.
Response to Reviewer jxUN (3/4)
W3/Q3 Downstream transfer under matched compute budgets.
We appreciate the reviewer's suggestion and agree that evaluating downstream performance using generated tokens and training FLOPs would further strengthen our downstream transfer findings. To address this, we added Figure 14 and the following analysis to Appendix H.2 of the revised manuscript:
As shown in Figure 14, the trends are consistent with those based on training steps (Figure 3). In particular, more expressive settings generally achieve larger and more sustained downstream gains as training compute increases. These results suggest that the advantage of more expressive training persists across different measures of training compute.
Furthermore, we repeat the matched-budget comparison in Figure 3b using a fixed training budget of FLOPs. As shown in the table below, downstream performance increases monotonically from 51.71% accuracy for Implication-only to 55.72% for + Quantification, preserving the same trend observed in our original comparison.
Table: Downstream performance (Avg@8) under matched FLOPs.
| Training budget | Implication-only | + Conjunction | + Negation | + Disjunction | + Quantification |
|---|---|---|---|---|---|
| FLOPs | 51.71 | 51.96 | 53.92 | 54.29 | 55.72 |
We have also updated Figure 3b and Section 4.3 of the main text to highlight these findings:
However, the observed downstream trends could be confounded by differences in training compute or inference-time response length. To control for these factors, we further analyze downstream performance as a function of total generated training tokens and training FLOPs, as well as under matched inference-time token budgets (Appendix H.2). The advantages of more expressive settings remain consistent across these comparisons.
For the matched-depth comparison, as noted in the footnote of Section 4.3, the Implication-only setting is relatively easy, so the base model already reaches the 90% validation accuracy threshold at without training. Thus, the base model could have been used for this setting. However, we instead use the RL-trained checkpoint at , which achieves better downstream performance than the base model. This provides a stronger reference point for Implication-only, making the comparison more conservative with respect to the downstream advantages of more expressive settings.
Response to Reviewer jxUN (1/4)
We sincerely thank the reviewer for the detailed review. We address each point below and highlight the corresponding revisions in the manuscript in blue. We would welcome any follow-up in case there are any remaining concerns, so that we may have the opportunity to address them during the discussion period.
W1/Q1 Robustness of the power-law fit.
We appreciate the reviewer's concern and agree that additional robustness analyses of the power-law fit would strengthen our conclusion.
First, we have expanded Appendix G.1 to include a linear model (T=a+cD) alongside the original power-law and exponential models. All three models have two free parameters and are fitted by least squares. The corresponding results are summarized in the table below.
Table: Power-law (T=a⋅Dγ), exponential (T=a⋅ebD) and linear (T=a+cD) fits across expressiveness levels. All models have two free parameters and are fitted by least squares. ΔAICexp=AICexp−AICpow and ΔAIClin=AIClin−AICpow; positive values favor the power law.
We report the following findings in Appendix G.1 of the revised manuscript:
Second, we conducted an additional held-out-depth prediction experiment. For each expressiveness setting, we fit the power-law, exponential, and linear models using only the shallower depths, reserving the two deepest depths for evaluation. The results are summarized in the table below and visualized in Figure 7 of the revised manuscript (Appendix G.1).
Table: Held-out-depth prediction. Mean absolute percentage error (MAPE, %) is computed over the two deepest held-out depths for each setting.
We discuss these findings in Appendix G.1 as follows: