acceptodds
Under review as a conference paper at ICLR 2027

CONGRATTS : Convergence-Guided Resource-Aware Test-Time Scaling for Local LLMs

Abstract

Test-time scaling (TTS) can improve language-model outputs by repeated output sampling or refinement, but the number of scaling rounds required varies across inputs: some outputs stabilize after only a few rounds, while others continue to change, and the cost of another round depends on system resources currently available. We introduce CONGRATTS (**CON**vergence-**G**uided **R**esource-**A**ware **TTS**), a framework for adaptive TTS of locally-hosted models. CONGRATTS decides after each scaling round whether to continue by jointly considering output convergence and live resource feasibility. The same framework supports both self-consistency and self-refinement scaling, without retraining, model modification, hidden-state access, or an external verifier model, and is evaluated across multiple small language models, related datasets, and distinct inference backends. In our evaluation, CONGRATTS reduces self-consistency scaling rounds by 48.2% with only a 0.39 percentage-point accuracy decrease relative to fixed-budget scaling, and self-refinement rounds by 88.3% while improving accuracy by 1.96 percentage points. Under controlled resource contention on both a MacBook Pro (M1 Max) and Jetson AGX Orin, it reduces latency by 34.4-87.6% and energy use by 31.6-87.9% relative to fixed-budget scaling, with only a 0.38 percentage-point average accuracy decrease across both platforms. Our code is available at: https://anonymous.4open.science/r/congratts-E021.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time scaling, adaptive inference, large language models, language models, resource-aware inference, efficient inference, self-consistency, self-refinement

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_XGwROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CONGRATTS, a convergence-guided and resource-aware framework for adaptive test-time scaling of locally hosted LLMs. The method uses a lightweight convergence predictor and runtime resource monitoring to determine when to stop self-consistency sampling or self-refinement. Experiments demonstrate reductions in scaling rounds, latency, and energy consumption across several small language models and hardware platforms.

Critical strengths
  1. The paper addresses a practically relevant problem in efficient on-device LLM inference.
  2. The experimental evaluation is relatively comprehensive, covering multiple models, benchmarks, quantization settings, and real hardware.
Critical weaknesses
  1. Limited methodological novelty. The proposed framework essentially combines a learned convergence-based stopping criterion with a resource-feasibility check. Both components are conceptually straightforward and closely related to existing adaptive stopping and resource-aware inference techniques. The paper does not clearly establish a substantial methodological advance beyond integrating these existing ideas.
  2. Limited improvement over existing adaptive stopping methods. In self-consistency experiments, CONGRATTS does not consistently outperform existing approaches such as CGES in the accuracy–compute trade-off.
Questions for authors

See weakness

3 comments

We thank the reviewer for their assessment. We agree that neither a learned stopping criterion nor a resource-feasibility check is novel in itself. Our contribution is the design of a shared, resource-aware continuation framework that is deliberately not specialised to a single scaling paradigm or deployment configuration.

1. Novelty and motivation

Existing adaptive stopping methods are largely specialised to their scaling paradigm. For example, CGES and ASC exploit statistics specific to repeated self-consistency samples, whereas Semantic-Streak and Edit-Saturation operate on changes between successive self-refinement outputs. CONGRATTS instead learns a common notion of output convergence and uses the same jointly trained predictor for both SC and SR. This same learned model is subsequently reused without retraining across three additional LLMs, quantized inference, and a second inference backend.

The resource-feasibility mechanism is an explicit component of our framework, not an inherent property of the existing stopping methods. To make the reviewer-requested hardware comparison controlled, we augmented CGES and semantic-streak with the same CONGRATTS resource-feasibility guard. This isolates the stopping criterion while holding the resource-aware execution mechanism fixed.

The resulting matched comparison is:

ScalingDeviceSpecialised baseline Acc. / RoundsCONGRATTS Acc. / Rounds
SCM1 MaxCGES: 43.06 / 5.4344.86 / 6.89
SCJetsonCGES: 48.06 / 4.7747.08 / 6.13
SRM1 MaxSemantic-Streak: 22.22 / 3.8624.03 / 3.16
SRJetsonSemantic-Streak: 20.00 / 2.8423.06 / 3.03

These results help clarify our intended contribution. On Mac, CONGRATTS achieves higher SC accuracy than CGES, with query-level analysis showing that additional sampling can recover answers missed by CGES. On Jetson, CGES achieves slightly higher accuracy with fewer rounds, although most CGES-only successes coincide with resource gating that prevents CONGRATTS from refining its initial answer. CONGRATTS also achieves substantially higher SR accuracy on both platforms, using the same learned convergence predictor throughout. We therefore do not claim universal superiority over paradigm-specific stopping methods, particularly as separate hardware executions prevent attributing the differences solely to the stopping criteria.

The objective of CONGRATTS is, instead, to provide a unified transferable controller that supports both parallel and sequential scaling, together with a common resource-feasibility mechanism for live device constraints.

Our design is motivated by the fact that no single scaling paradigm is preferable across applications (as shown in Fig. 1), highlighting the value of supporting different scaling procedures across query and task types. Supporting SC, SR, and potentially other output-sequence scaling paradigms with a common controller reduces the need to develop and maintain separate learned stopping mechanisms for each setting. This is particularly relevant for local inference, where controller complexity and additional model, calibration, and implementation overheads are important.

2. Comparison with existing adaptive stopping methods

We agree that CONGRATTS does not uniformly dominate CGES in the SC accuracy--compute trade-off, and we do not intend to make that claim. In our original evaluation, across the 12 SC model--dataset combinations, CONGRATTS achieves higher mean accuracy than CGES in 9 cases, equal accuracy in one, and lower accuracy in two. Its macro-average accuracy is 62.06% versus 61.67% for CGES, while reducing scaling rounds by 48.2% compared with 62.0% for CGES. The new matched hardware comparison above provides the same picture for SC under identical resource constraints. Importantly, however, the same CONGRATTS predictor also operates on SR, where it achieves higher observed accuracy than Semantic-Streak on both devices and uses fewer rounds on the M1 Max.

Accordingly, we view our contribution as broader than obtaining the best accuracy--compute operating point for every individual SC (or SR) configuration. Unlike existing paradigm-specific stopping methods, CONGRATTS learns a shared convergence model that operates across both dominant scaling paradigms, achieving competitive performance while incorporating live resource feasibility for on-device execution.

We will revise the draft to make this intended contribution clear and to avoid implying universal superiority over specialised stopping methods.

The rebuttal did not address my concern about the novelty. The core method is essentially a combination of learned early stopping and resource-threshold checking, both of which are well-established techniques.

We thank the reviewer for their follow-up. We agree that learned stopping and resource-threshold checking are not individually novel. However, our key contribution is a single unified stopping model that generalises across scaling paradigms, a capability that, to our knowledge, has not been demonstrated by any existing method. Our single jointly trained model supports both parallel self-consistency and sequential self-refinement without retraining, and transfers across multiple LLMs, datasets, quantisation settings, and inference backends. We hope this clarifies our intended contribution.

Reviewer_4DVROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CONGRATTS decides whether locally hosted language models should execute another self-consistency sample or retrieval-augmented refinement round. A lightweight classifier predicts agreement with the output of a fixed-budget run using causal output statistics. Continuation additionally requires a resource-feasibility check. The same jointly trained classifier is reused across several small models and inference backends. The evaluation reports reductions in scaling rounds, latency, energy, and post-base resource violations, with extensive supporting ablations.

Critical strengths
  1. The joint treatment of test-time scaling and resource constraints is a meaningful contribution to practical local LLM inference. Deciding whether another inference round is worthwhile and feasible addresses a concrete problem for local LLM deployment.

  2. The evaluation gives this idea practical support: M1 Max and Jetson measurements cover latency, energy, and resource violations, and the appendix examines calibration, feature groups, classifiers, training-set size, and quantization. Together, these experiments demonstrate value beyond a reduction in the number of inference rounds alone.

Critical weaknesses
  1. The evidence could more clearly isolate the benefit of the learned convergence controller within the complete system. Table 2 compares fixed, resource-only, and full control, whereas Tables 3–4 evaluate stopping methods separately. Combining ASC/CGES or semantic-streak stopping with the same resource gate would make the end-to-end comparison more convincing. Such a comparison would sharpen attribution of the gains.

  2. The scope of the practical claims would benefit from a few clarifications. In SR, model-proposed STOP is replaced with OPEN or fallback SEARCH (Appendix B.1), so a native-STOP reference would help interpret the magnitude of the savings. Appendix D also keeps the workload setting fixed within each query; this supports operation under contention but provides less direct evidence for responding to mid-query load changes.

Questions for authors
  1. Could you clarify whether a comparison with an existing stopping rule using the identical resource gate is available? This would help isolate the contribution of the convergence controller.

  2. What accuracy–cost trade-off results when SR respects its native STOP action? This would clarify the practical significance of the reported savings.

  3. Could you clarify whether the evaluation includes controlled mid-query load changes and align the adaptation claim with the tested conditions?

4 comments

Part 1/4

We’d first like to thank the reviewer for their thoughtful and constructive comments. We address the concerns raised below.

1. Comparison with specialised stopping methods using CONGRATTS resource guard

We agree that evaluating existing stopping methods under identical resource constraints is important for isolating the contribution of the learned convergence controller.

To address this, we conducted additional hardware experiments on both the MacBook Pro M1 Max and Jetson AGX Orin, comparing CONGRATTS against CGES for self-consistency (SC) and Semantic-Streak for self-refinement (SR). For a controlled comparison, we augment CGES and Semantic-Streak with the CONGRATTS resource-feasibility guard, so that the convergence/stopping criterion is the principal difference between the methods. These new experiments evaluate the specialised stopping methods within our common resource-aware execution framework, whilst recognising that they are not inherently resource-aware methods.

We summarise the results below as mean ± standard deviation across three runs with a fixed generation seed. CONGRATTS uses three controller-training seeds, while CGES and Semantic-Streak use stopping parameters calibrated separately across three calibration seeds.

ScalingDeviceSpecialised baseline Acc. / RoundsCONGRATTS Acc. / Rounds
SCM1 MaxCGES: 43.06 / 5.4344.86 / 6.89
SCJetsonCGES: 48.06 / 4.7747.08 / 6.13
SRM1 MaxSemantic-Streak: 22.22 / 3.8624.03 / 3.16
SRJetsonSemantic-Streak: 20.00 / 2.8423.06 / 3.03

View Part 4/4: full hardware results table below for the full, updated hardware results table.

SC. Under the identical resource-feasibility guard, CONGRATTS and CGES occupy different accuracy--efficiency operating points, with neither method dominating across both platforms. On Mac, CONGRATTS achieves 1.80 percentage points higher accuracy than CGES (44.86% vs. 43.06%), while using 6.89 rounds on average versus 5.43. On Jetson, CGES uses fewer rounds (4.77 vs. 6.13) and achieves 0.98 percentage points higher accuracy (48.06% vs. 47.08%). Further analysis of individual queries helps explain the different accuracy outcomes observed across the two devices. On Jetson, most CGES-only successes coincide with resource gating that prevents CONGRATTS from refining its initial answer. On Mac, CONGRATTS more often recovers answers missed by CGES, including cases where additional scaling corrects an early incorrect consensus. These results do not indicate universal superiority of either SC stopping method and instead highlight device-dependent trade-offs, although separate hardware executions prevent attributing the differences solely to the stopping criteria. Importantly, CONGRATTS remains competitive with the specialised SC method while using the same jointly trained convergence predictor that also supports sequential refinement without retraining.

SR. CONGRATTS achieves higher observed accuracy than semantic-streak on both platforms, with different efficiency trade-offs. On Mac, it improves accuracy by 1.81 percentage points while reducing refinement rounds from 3.86 to 3.16, latency by 17%, and energy by 24%. On Jetson, CONGRATTS achieves a larger 3.06-point accuracy advantage, using 3.03 refinement rounds versus 2.84, with correspondingly higher latency and energy.

Compliance with resource bounds. On Jetson, CONGRATTS records slightly fewer controller misses and post-base resource violations than the specialised baselines: 2.50% versus 2.78% for SC, and 1.94% versus 3.47% for SR. On Mac, the specialised baselines record low or zero controller-miss rates (0.14% for CGES and 0% for semantic streak), compared with 0.56% and 0.97% for CONGRATTS. Base-invalid cases occur prior to adaptive stopping and should not be attributed to the stopping policy.

Overall, these additional experiments provide a more complete assessment of the learned controller under identical resource constraints. CONGRATTS improves both accuracy and efficiency over Semantic-Streak on Mac and achieves higher SR accuracy on Jetson, while remaining competitive with specialised SC stopping. Our contribution is not universal superiority over every paradigm-specific method, but a single jointly-trained convergence predictor supporting both SC and SR, integrated with a resource-feasibility guard that can also be flexibly applied to other specialised stopping methods.

We will incorporate these comparisons into the revised hardware evaluation and clarify the complementary roles of the shared convergence predictor and resource guard.

Part 2/4

2. Self-refinement: native STOP comparison

We agree that the fixed-16 baseline can exaggerate the apparent significance of the number of scaling rounds saved, particularly when SR accuracy peaks early. We also agree that evaluating the model's own stopping behavior is important for establishing how much CONGRATTS contributes beyond avoiding unnecessary refinement.

To address this, we conducted additional evaluations on both SR datasets using the ‘native’ STOP actions recorded during our existing SR runs. We reconstructed native stopping by terminating each trajectory at the first model-proposed STOP action. The results are summarised below.

DatasetModelSTOP queriesCorrect at STOPNative acc. (%)Fixed16 acc. (%)Mean steps
CRAGLFM2.531/500 (6.2%)3/31 (9.68%)17.8018.4015.544
Llama-3.20/500 (0.0%)—22.8022.8016.000
Qwen2.54/500 (0.8%)1/4 (25.00%)25.8025.8015.940
SmolLM23/500 (0.6%)0/3 (0.00%)14.4014.4015.960
Overall38/2,000 (1.9%)4/38 (10.53%)20.2020.3515.861
POPQALFM2.550/500 (10.0%)8/50 (16.00%)22.8022.4015.170
Llama-3.20/500 (0.0%)—21.0021.0016.000
Qwen2.514/500 (2.8%)5/14 (35.71%)25.0024.8015.750
SmolLM214/500 (2.8%)1/14 (7.14%)20.2020.4015.872
Overall78/2,000 (3.9%)14/78 (17.95%)22.2522.1515.698

The results use one saved execution per model and dataset.

Native STOP is rarely proposed across both SR datasets. On CRAG, 98.1% of the 2,000 model–query pairs never issue a parsed STOP action within the 16-round budget; on POPQA, the corresponding proportion is 96.1%. Consequently, native stopping returns an answer after an average of 15.861 refinement steps on CRAG and 15.698 on POPQA, saving only 0.87% and 1.89% of the maximum refinement budget, respectively. Average accuracy decreases slightly from 20.35% to 20.20% on CRAG, while increasing marginally from 22.15% to 22.25% on POPQA.

The model-level results show that the frequency of native STOP proposals varies across models, but remains consistently low. Llama-3.2 never proposes STOP on either dataset. On CRAG, Qwen2.5 and SmolLM2 stop on fewer than 1% of queries, while LFM2.5 stops on 6.2%. On POPQA, these frequencies increase to 2.8% for Qwen2.5 and SmolLM2, and 10.0% for LFM2.5.

The model's STOP decisions also do not reliably identify correct answers. On CRAG, only 4 of the 38 answers returned at the first STOP are correct (10.53%), compared with 7 correct fixed-16 answers on those same queries. On POPQA, 14 of the 78 answers returned at STOP are correct (17.95%), compared with 12 correct fixed-16 answers on the corresponding queries. Although STOP events are rare and concern model-selected subsets of queries, these comparisons provide no evidence that native STOP reliably identifies successful refinement.

These results demonstrate that native STOP provides negligible computational savings under our evaluated protocol, while also highlighting the limitations of relying on the model's own termination decisions for adaptive refinement. Importantly, the comparison also shows that overriding the native STOP does not account for the substantial savings achieved by CONGRATTS.

We will add this native-STOP comparison to the ablations and explicitly reference it from the main text, clarifying the limitations of the fixed-budget comparison and the additional benefit of an external early-stopping controller over the model's own termination decisions.

Part 3/4

3. Workload variation and mid-query adaptation.

We appreciate the reviewer highlighting this distinction. Our controller measures the live resource state before every continuation decision, but in the submitted hardware experiments the externally imposed contention level is held fixed within each query and varied across queries. Thus, the experiments evaluate per-round resource-aware decisions under multiple contention levels and naturally varying measured resource usage, but do not include a controlled intervention that changes the imposed workload level partway through an active query or during a single scaling round.

Importantly, the continuation rule itself does not assume a stationary resource state; since feasibility gi,tg_{i,t} is recomputed from the current measurements at every scaling state, a resource change between rounds would be reflected in the next continuation decision without modifying the convergence predictor or stopping rule. The resource guard could also be evaluated during an ongoing generation, without any change in its design, enabling the controller to respond to changes in resource usage within a round, provided the inference backend supports interrupting ongoing generation. However, we have not experimentally established the controller's behaviour under mid-query or within-round workload changes.

We will clarify this distinction in the paper and restrict our empirical claims accordingly. Controlled mid-query workload transitions remain a direction for further evaluation.

Part 4/4: full hardware results table

DeviceScalingMethodAcc. (%)RoundsLatency (s)Energy (J)Controller Miss (%)Base Invalid (%)Post-Base Invalid (%)
MacBook Pro (M1 Max)SCFixed46.25 ± 1.2511.73 ± 0.9970.90 ± 4.691406.6 ± 40.2—3.89 ± 3.248.61 ± 3.39
MacBook Pro (M1 Max)SC+ Resource46.11 ± 0.8711.56 ± 1.2271.11 ± 8.551424.6 ± 105.73.19 ± 2.514.58 ± 5.913.19 ± 2.51
MacBook Pro (M1 Max)SC+CGES43.06 ± 0.875.43 ± 0.2937.01 ± 2.05765.8 ± 41.90.14 ± 0.240.00 ± 0.000.14 ± 0.24
MacBook Pro (M1 Max)SCCONGRATTS44.86 ± 0.486.89 ± 0.8546.54 ± 5.76962.4 ± 111.80.56 ± 0.240.14 ± 0.240.56 ± 0.24
MacBook Pro (M1 Max)SRFixed23.47 ± 0.2413.30 ± 0.8976.77 ± 1.162797.1 ± 321.9—0.42 ± 0.427.36 ± 8.06
MacBook Pro (M1 Max)SR+ Resource24.17 ± 0.0012.87 ± 0.1971.36 ± 1.222800.6 ± 24.01.25 ± 0.420.28 ± 0.241.25 ± 0.42
MacBook Pro (M1 Max)SR+Semantic-Streak22.22 ± 0.963.86 ± 0.5311.41 ± 3.96451.4 ± 143.80.00 ± 0.000.00 ± 0.000.00 ± 0.00
MacBook Pro (M1 Max)SRCONGRATTS24.03 ± 1.053.16 ± 0.579.50 ± 3.29342.0 ± 119.90.97 ± 0.480.69 ± 1.200.97 ± 0.48
Jetson AGX OrinSCFixed48.19 ± 0.2412.29 ± 0.00192.61 ± 1.017189.2 ± 55.6—21.11 ± 0.483.89 ± 0.48
Jetson AGX OrinSC+ Resource47.78 ± 0.6410.28 ± 0.09160.44 ± 2.225780.6 ± 99.12.64 ± 0.2414.58 ± 0.422.64 ± 0.24
Jetson AGX OrinSC+CGES48.06 ± 0.244.77 ± 0.2981.16 ± 5.323056.9 ± 203.42.78 ± 0.6420.14 ± 0.482.78 ± 0.64
Jetson AGX OrinSCCONGRATTS47.08 ± 0.426.13 ± 0.58106.16 ± 8.803913.5 ± 319.82.50 ± 0.8321.81 ± 0.642.50 ± 0.83
Jetson AGX OrinSRFixed22.64 ± 0.2413.35 ± 0.0290.80 ± 0.174180.6 ± 20.3—10.97 ± 0.6414.03 ± 0.64
Jetson AGX OrinSR+ Resource21.11 ± 0.2411.77 ± 0.0379.54 ± 0.183562.4 ± 9.79.17 ± 0.420.69 ± 0.489.17 ± 0.42
Jetson AGX OrinSR+Semantic-Streak20.00 ± 0.422.84 ± 0.3810.69 ± 3.12432.0 ± 137.03.47 ± 0.8716.11 ± 0.243.47 ± 0.87
Jetson AGX OrinSRCONGRATTS23.06 ± 1.343.03 ± 0.5211.55 ± 3.54507.4 ± 161.91.94 ± 1.0510.28 ± 0.241.94 ± 1.05
Reviewer_tRScOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the question of designing a good stopping rule for the amount of test-time compute a small language model should spend on a problem. The authors propose CONGRATTS, which combines a learned estimate of "answer stabilization" with a check that another round fits the device's current resources.

  • A small classifier is trained to predict agreement with a fixed 16-round output, using answer history and token probabilities. The same classifier supports self-consistency and retrieval-based self-refinement, with separate stopping thresholds for the two settings.
  • Across four 1–2B models, the paper reports, on average, ~50% fewer self-consistency samples with a minimal accuracy decrease, and ~90% fewer refinement rounds with a slight improvement in accuracy. Experiments with Qwen2.5-1.5B on a MacBook and Jetson show substantial latency and energy savings against the fixed-budget baseline.
Critical strengths

The question is well motivated: a stopping rule for local devices should consider both whether more computation helps and whether the device can support it. The paper makes a practical case for avoiding long, unproductive runs. There are several clear benefits as I see it:

  • The controller seems easy enough to deploy. Reusing the frozen classifier across models, quantization levels, and two inference backends is useful evidence of portability.
  • The evaluation measures actual deployment costs. Table 2 reports latency, energy, and resource violations on two devices. Saving a given fraction of rounds need not save the same fraction of time, so this is a good comparison.
  • Appendices address several reasonable alternative explanations through ablations. Examining training-set size, feature groups, separate versus joint training, semantic thresholds, embedding models, classifier architecture, and calibration-set size.
Critical weaknesses

My main concern is the comparison with simpler stopping rules on the actual devices.

The savings over the semantic-streak baseline are encouraging, but the hardware experiments do not include that comparison under the same resource constraints.

  • The main device experiments in Table 2 compare against a fixed budget and a resource check alone. Adding the learned rule helps, but I would like to see the simpler stopping rules use the same resource check. That would show whether learning when to stop is better than a simple rule under the same device limits. This seems like the most important comparison for the paper's central idea.
  • The self-refinement setup also makes the headline savings less convincing. Figure 4 shows that accuracy often peaks well before round 16, and Table 4 shows that four rounds already beat 16 on average. The runs also override the model's own decision to stop. I would like to see a comparison that lets the model stop when it asks to. That would help judge how much the controller adds beyond avoiding a long, unproductive run, and make the reported reduction of almost 90% easier to interpret.
Questions for authors

I would focus on the comparisons mentioned in weakness section, and to understand whether stopping loses useful corrections. These would validate the contribution more than just additional variations of the classifier. Some related questions:

  • How do the existing stopping baselines perform on the two devices when they use the same resource check as CONGRATTS?
  • How much of the self-refinement benefit remains when the model's own STOP action is honored?
  • How often does the controller stop on a wrong answer that would have been corrected by continuing? Also, are the small accuracy differences reliable once uncertainty across test questions is included?
6 comments

Part 1/6

We’d first like to thank the reviewer for their thoughtful and constructive comments. We address the concerns raised below.

1. Comparison with specialised stopping methods using CONGRATTS resource guard

We agree that evaluating existing stopping methods under identical resource constraints is important for isolating the contribution of the learned convergence controller.

To address this, we conducted additional hardware experiments on both the MacBook Pro M1 Max and Jetson AGX Orin, comparing CONGRATTS against CGES for self-consistency (SC) and Semantic-Streak for self-refinement (SR). For a controlled comparison, we augment CGES and Semantic-Streak with the CONGRATTS resource-feasibility guard, so that the convergence/stopping criterion is the principal difference between the methods. These new experiments evaluate the specialised stopping methods within our common resource-aware execution framework, whilst recognising that they are not inherently resource-aware methods.

We summarise the results below as mean ± standard deviation across three runs with a fixed generation seed. CONGRATTS uses three controller-training seeds, while CGES and Semantic-Streak use stopping parameters calibrated separately across three calibration seeds.

ScalingDeviceSpecialised baseline Acc. / RoundsCONGRATTS Acc. / Rounds
SCM1 MaxCGES: 43.06 / 5.4344.86 / 6.89
SCJetsonCGES: 48.06 / 4.7747.08 / 6.13
SRM1 MaxSemantic-Streak: 22.22 / 3.8624.03 / 3.16
SRJetsonSemantic-Streak: 20.00 / 2.8423.06 / 3.03

View Part 6/6: full hardware results table below for the full, updated hardware results table.

SC. Under the identical resource-feasibility guard, CONGRATTS and CGES occupy different accuracy--efficiency operating points, with neither method dominating across both platforms. On Mac, CONGRATTS achieves 1.80 percentage points higher accuracy than CGES (44.86% vs. 43.06%), while using 6.89 rounds on average versus 5.43. On Jetson, CGES uses fewer rounds (4.77 vs. 6.13) and achieves 0.98 percentage points higher accuracy (48.06% vs. 47.08%). Further analysis of individual queries helps explain the different accuracy outcomes observed across the two devices. On Jetson, most CGES-only successes coincide with resource gating that prevents CONGRATTS from refining its initial answer. On Mac, CONGRATTS more often recovers answers missed by CGES, including cases where additional scaling corrects an early incorrect consensus. These results do not indicate universal superiority of either SC stopping method and instead highlight device-dependent trade-offs, although separate hardware executions prevent attributing the differences solely to the stopping criteria. Importantly, CONGRATTS remains competitive with the specialised SC method while using the same jointly trained convergence predictor that also supports sequential refinement without retraining.

SR. CONGRATTS achieves higher observed accuracy than semantic-streak on both platforms, with different efficiency trade-offs. On Mac, it improves accuracy by 1.81 percentage points while reducing refinement rounds from 3.86 to 3.16, latency by 17%, and energy by 24%. On Jetson, CONGRATTS achieves a larger 3.06-point accuracy advantage, using 3.03 refinement rounds versus 2.84, with correspondingly higher latency and energy.

Compliance with resource bounds. On Jetson, CONGRATTS records slightly fewer controller misses and post-base resource violations than the specialised baselines: 2.50% versus 2.78% for SC, and 1.94% versus 3.47% for SR. On Mac, the specialised baselines record low or zero controller-miss rates (0.14% for CGES and 0% for semantic streak), compared with 0.56% and 0.97% for CONGRATTS. Base-invalid cases occur prior to adaptive stopping and should not be attributed to the stopping policy.

Overall, these additional experiments provide a more complete assessment of the learned controller under identical resource constraints. CONGRATTS improves both accuracy and efficiency over Semantic-Streak on Mac and achieves higher SR accuracy on Jetson, while remaining competitive with specialised SC stopping. Our contribution is not universal superiority over every paradigm-specific method, but a single jointly-trained convergence predictor supporting both SC and SR, integrated with a resource-feasibility guard that can also be flexibly applied to other specialised stopping methods.

We will incorporate these comparisons into the revised hardware evaluation and clarify the complementary roles of the shared convergence predictor and resource guard.

Part 2/6

2. Self-refinement: native STOP comparison

We agree that the fixed-16 baseline can exaggerate the apparent significance of the number of scaling rounds saved, particularly when SR accuracy peaks early. We also agree that evaluating the model's own stopping behavior is important for establishing how much CONGRATTS contributes beyond avoiding unnecessary refinement.

To address this, we conducted additional evaluations on both SR datasets using the ‘native’ STOP actions recorded during our existing SR runs. We reconstructed native stopping by terminating each trajectory at the first model-proposed STOP action. The results are summarised below.

DatasetModelSTOP queriesCorrect at STOPNative acc. (%)Fixed16 acc. (%)Mean steps
CRAGLFM2.531/500 (6.2%)3/31 (9.68%)17.8018.4015.544
Llama-3.20/500 (0.0%)—22.8022.8016.000
Qwen2.54/500 (0.8%)1/4 (25.00%)25.8025.8015.940
SmolLM23/500 (0.6%)0/3 (0.00%)14.4014.4015.960
Overall38/2,000 (1.9%)4/38 (10.53%)20.2020.3515.861
POPQALFM2.550/500 (10.0%)8/50 (16.00%)22.8022.4015.170
Llama-3.20/500 (0.0%)—21.0021.0016.000
Qwen2.514/500 (2.8%)5/14 (35.71%)25.0024.8015.750
SmolLM214/500 (2.8%)1/14 (7.14%)20.2020.4015.872
Overall78/2,000 (3.9%)14/78 (17.95%)22.2522.1515.698

The results use one saved execution per model and dataset.

Native STOP is rarely proposed across both SR datasets. On CRAG, 98.1% of the 2,000 model–query pairs never issue a parsed STOP action within the 16-round budget; on POPQA, the corresponding proportion is 96.1%. Consequently, native stopping returns an answer after an average of 15.861 refinement steps on CRAG and 15.698 on POPQA, saving only 0.87% and 1.89% of the maximum refinement budget, respectively. Average accuracy decreases slightly from 20.35% to 20.20% on CRAG, while increasing marginally from 22.15% to 22.25% on POPQA.

The model-level results show that the frequency of native STOP proposals varies across models, but remains consistently low. Llama-3.2 never proposes STOP on either dataset. On CRAG, Qwen2.5 and SmolLM2 stop on fewer than 1% of queries, while LFM2.5 stops on 6.2%. On POPQA, these frequencies increase to 2.8% for Qwen2.5 and SmolLM2, and 10.0% for LFM2.5.

The model's STOP decisions also do not reliably identify correct answers. On CRAG, only 4 of the 38 answers returned at the first STOP are correct (10.53%), compared with 7 correct fixed-16 answers on those same queries. On POPQA, 14 of the 78 answers returned at STOP are correct (17.95%), compared with 12 correct fixed-16 answers on the corresponding queries. Although STOP events are rare and concern model-selected subsets of queries, these comparisons provide no evidence that native STOP reliably identifies successful refinement.

These results demonstrate that native STOP provides negligible computational savings under our evaluated protocol, while also highlighting the limitations of relying on the model's own termination decisions for adaptive refinement. Importantly, the comparison also shows that overriding the native STOP does not account for the substantial savings achieved by CONGRATTS.

We will add this native-STOP comparison to the ablations and explicitly reference it from the main text, clarifying the limitations of the fixed-budget comparison and the additional benefit of an external early-stopping controller over the model's own termination decisions.

Part 3/6

3. Missed corrections and over-refinement

We appreciate this question, as it addresses an important limitation of convergence-based stopping. We conducted a new analysis comparing the early-stopped answer with subsequent answer states in the corresponding fixed-16 trajectory. We distinguish three separate outcomes:

  • Missed final correction: The controller stops with an incorrect answer, whereas the fixed-16 final answer is correct.
  • Avoided regression: The controller stops with a correct answer, whereas the fixed-16 final answer is incorrect.
  • Missed temporary correction: The controller stops incorrectly, but a subsequent intermediate state becomes correct, even if that correction is not retained at round 16.

The observed rates of each are summarised in the table below. Percentages are expressed relative to all evaluated steps.

DatasetMissed final correctionAvoided regressionMissed temp correction
GSM8K1.49%1.18%0.49%
MATH-5001.37%0.53%0.25%
SVAMP1.22%1.22%0.17%
CRAG6.08%10.35%15.42%
POPQA6.35%6.00%7.53%

On the SC datasets, missed final corrections are relatively infrequent (1.22-1.49%), although they slightly outweigh the number of avoided regressions on GSM8K and MATH-500. On CRAG, early stopping misses final corrections in 6.08% of outcomes but prevents regressions in 10.35% of outcomes, explaining the net +4.27 percentage-point accuracy improvement. However, 15.42% of trajectories contain a correct intermediate answer after the stopping point. On POPQA, missed final corrections (6.35%) and avoided regressions (6.00%) are almost balanced, yielding a 0.35-point accuracy decrease, while 7.53% of trajectories contain a temporary correction.

To further characterise the accuracy trade-offs of early stopping, we also examined outcomes conditional on the correctness of the controller's returned answer. Specifically, we measured the proportion of incorrect early-stopped answers that are correct at round 16, and the proportion of correct early-stopped answers that become incorrect by round 16. We summarise the results in the table below.

DatasetFixed16 correct given wrong stop, % [95% CI]Fixed16 wrong given correct stop, % [95% CI]
GSM8K5.84 [4.70, 7.12]1.94 [1.45, 2.47]
MATH-5004.01 [2.56, 5.56]1.33 [0.55, 2.26]
SVAMP6.88 [4.20, 10.07]1.63 [0.90, 2.44]
CRAG8.15 [6.65, 9.75]42.16 [37.36, 47.19]
POPQA8.15 [6.60, 9.73]27.27 [22.47, 32.33]

The results include only trajectories where CONGRATTS stops before round 16. The confidence intervals are obtained from 5,000 question-level resamples.

These results further illustrate the non-monotonic effects of continued refinement. On CRAG, only 8.15% of incorrect early-stopped answers are correct at round 16, whereas 42.16% of correct early-stopped answers become incorrect by the final round. On POPQA, these probabilities are 8.15% and 27.27%, respectively. Thus, although early stopping can miss subsequent corrections, continuing refinement also carries a substantial risk of degrading correct answers. The corresponding SC probabilities are much lower, consistent with the different dynamics of independent sampling and iterative refinement.

We will include this conditional analysis alongside the overall missed-correction and avoided-regression rates to provide a more complete characterisation of the accuracy trade-offs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.