acceptodds
Under review as a conference paper at ICLR 2027

Constructing Pareto Frontiers of Optimal Reasoning Termination Methods for Overthinking in Reasoning Models

27 views

Abstract

Reasoning models can benefit from test-time scaling, but continued reasoning may become ineffective or harmful, a phenomenon termed overthinking. Evaluations against full thinking compare accuracy and token usage but lack references for overthinking severity and strategies’ theoretical mitigation potential. To characterizes overthinking at the model–task level, we first introduce Step-wise Truncation Probing (STP), repeatedly generating answers from multiple retained prefixes along each fixed sampled reasoning trajectory. Using each trajectory’s earliest measured position attaining maximum empirical accuracy as a reference, we define Overthinking Ratio and Accuracy Degradation as the fraction of thinking tokens generated afterward and the accuracy decrease from this reference to full thinking, respectively. Secondly, we then formulate dynamic routing, fixed budgets, and dynamic early exit within a unified framework to quantify the theoretical mitigation potential of different termination strategies. Based on this unified framework, we use the complete STP measurements to construct a token–accuracy Pareto frontier for each strategy under a common normalized thinking-token budget. We further develop OTBench, a reusable benchmark built from the STP process, which produced 70.5B reasoning tokens and the corresponding probing measurements across six models and four benchmark groups.

Primary area:
interpretability and explainable AI
Keywords:
Overthinking, Reasoning Model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_w43AOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies overthinking in reasoning language models and explores reasoning termination strategies. The authors first introduce Step-wise Truncation Probing (STP), which samples answers from different prefixes of a fixed reasoning trajectory to estimate how answer accuracy evolves with additional reasoning. In addition, this paper formulates three reasoning termination strategies, including dynamic routing, fixed budgets, and dynamic early exit, within a budget-constrained optimization framework. By optimizing stopping decisions using complete STP measurements, they construct strategy-specific token–accuracy Pareto frontiers. The experiments show that the proposed method could improve reasoning efficiency.

Critical strengths
  • The proposed framework provides a perspective on examining the potential accuracy–efficiency trade-offs, which is an interesting problem for reasoning models.
  • The STP provides an approach to analyzing the effects of reasoning length on accuracy by investigating multiple truncation positions along the same sampled trajectory.
  • The paper organizes dynamic routing, fixed budgets, and dynamic early exit within a unified optimization framework.
Critical weaknesses
  • Equation (1) selects the earliest position achieving the maximum observed accuracy across all probed positions of a trajectory. Since each accuracy estimate is based on only 16 sampled answers, the maximum could show fluctuations. As a result, it could have selection bias.
  • The oracle Pareto frontiers may be optimistic because the same measurements are used for action selection and performance evaluation. The oracle may select stopping positions that happen to have favorable sampling noise.
  • The oracle frontiers rely on complete reasoning trajectories and repeated correctness measurements at every candidate stopping position. Such information is unavailable to a practical termination algorithm when deciding whether to stop. The authors should include stronger online baselines or experiments evaluating attainable performance with limited observation.
  • The main cost metric considers retained reasoning tokens but excludes answer-generation costs, feature extraction, stopping-signal computation, and other inference overhead.
  • There is a typo in Figure 1, "acciracy" -> accuracy.
Questions for authors

Please see the weaknesses.

4 comments

W1: Each accuracy estimate is based on only 16 sampled answers, and the earliest position achieving the maximum observed accuracy is selected across all probed positions of a trajectory. Do fluctuations in the maximum and the resulting selection bias affect the OR and AD estimates?

Yes, they do, and we have now measured the effect by comparing the 16-probe estimates with reference curves built from about 2,048 answers per position on 16 trajectories. The findings are:

  • AD is biased upward at 16 probes: on these trajectories the mean estimate is 29.84 pp against a reference of 25.82 pp. Scoring the selected position on independent answers, with the same 16 answers in total, gives 17.77 pp (8+8 split) and 21.23 pp (rotating 15+1). These values are biased downward, and their RMSE is higher than that of the original estimate (1.7 to 1.9 times as high for a single trajectory). The original AD estimator is the most accurate of the three.
  • OR is affected more strongly, because positions with nearly equal accuracy can be far apart, and small fluctuations decide which of them is selected. The original estimate is 53.13% against a reference of 35.55%.
  • We therefore introduce a posterior estimate of OR, which averages over the probability that each position is the true peak instead of selecting a single position. At 16 probes it reduces the RMSE from 36.32 to 18.56 pp for a single trajectory and from 17.84 to 2.54 pp for the average over the 16 trajectories.
  • Recomputed on all 92,448 trajectories, mean OR changes from 64.34% to 36.93%; AD keeps the original estimator and its values (mean 9.03 pp); the two held-out estimators give 2.36 and 4.78 pp on the full benchmark, which are biased downward.

The details are in our response to Reviewer bSdV, Q1 (five parts): the comparison of the three estimators with bias, SD and RMSE, the estimates at every doubling of the probe count from 16 to 1,024, the posterior estimator, and the recomputed main table. We will add the posterior estimator and the error analysis to the paper.

W2: The same measurements are used for action selection and performance evaluation, and the oracle may select stopping positions that happen to have favorable sampling noise. Could the oracle Pareto frontiers be optimistic?

Yes: at 16 probes the original frontier is optimistic. We measured by how much, and we also tested the natural remedy of scoring the selected actions on independent answers. Independent scoring errs in the other direction, by a similar or larger amount, and its total error is higher. We therefore keep the original estimator and will report its bias.

What is being estimated. For each strategy class, the frontier is the highest accuracy the class can reach at a given thinking budget. An estimator can miss this value in two ways: by scoring the selected actions too favourably, or by selecting actions that are not the best ones.

Experiment. We used 16 trajectories from GPT-OSS-20B and DeepSeek-R1-Distill-Qwen-7B on the four benchmarks: two questions for every model and benchmark, one trajectory per question. At every position we collected about 2,048 answers; the frontier computed from them is the reference. In each of 8,192 repetitions we drew 16 answers per position without replacement and built the frontier in three ways, each with the same 16 answers per position:

  • Original 16: all 16 answers select the policy and score it.
  • 8+8: eight answers select the policy, the other eight score it.
  • Rotating 15+1: fifteen answers select, one scores; averaged over the 16 choices of the held-out answer.

In all three, the full policy is re-selected under the constraints of the strategy class. Results average the eight model–benchmark groups. To fit the rebuttal period we used one trajectory per question; FB and DEE then allow the same actions and coincide, so the gap between strategy classes is FB/DEE accuracy minus DR accuracy.

Results at a budget of 50% of the full-thinking tokens. Accuracies in %, gaps and errors in pp. Bias is the mean estimate minus the reference; SD is the variation over repetitions.

QuantityEstimatorReferenceMean estimateBiasSDRMSE
DR accuracyOriginal 1658.2658.260.001.391.39
DR accuracy8+858.2657.74−0.522.152.21
DR accuracyRotating 15+158.2658.06−0.201.411.42
FB/DEE accuracyOriginal 1670.8475.25+4.411.554.67
FB/DEE accuracy8+870.8463.46−7.383.918.35
FB/DEE accuracyRotating 15+170.8466.36−4.483.875.92
Gap FB/DEE − DROriginal 1612.5816.99+4.411.934.82
Gap FB/DEE − DR8+812.585.72−6.864.148.01
Gap FB/DEE − DRRotating 15+112.588.30−4.283.975.84

The original FB/DEE frontier is 4.41 pp too high. The held-out frontiers are 7.38 and 4.48 pp too low, and their RMSE is higher (8.35 and 5.92 versus 4.67 pp). Independent scoring removes the favourable scoring, but it cannot recover a better policy that was missed during selection, and setting answers aside for scoring leaves fewer for selection. A lower estimate is therefore not a more accurate one. The DR frontier is hardly affected by the choice of estimator, because DR only chooses between the two endpoints of each question.

The gap between strategy classes behaves in the same way: 16.99 pp with the original estimator, 5.72 and 8.30 pp with the held-out estimators, 12.58 pp for the reference. The gap stays positive under every estimator, including the conservative held-out ones. For FB and DEE separately, on the full benchmark with 16 trajectories per question, see our response to Reviewer bSdV, Q1 (part 3).

Figure A (anonymous link) shows the mean estimate, SD and RMSE of the three estimators at all budgets; the pattern at 50% holds over most of the range. We will add this analysis to the paper and state the upward bias of the frontiers at 16 probes next to the frontier results.

W3: The oracle frontiers rely on complete reasoning trajectories and repeated correctness measurements at every candidate stopping position, and such information is unavailable to a practical termination algorithm when deciding whether to stop. Could you include stronger online baselines or experiments evaluating attainable performance with limited observation?

We agree that no online method has the information the oracle uses; the frontier is a reference for what a class of stopping decisions allows, not a claim about what is attainable online. What is attainable with limited observation is measured by the practical methods themselves, and the paper had only one per class. We have added six recent methods, two per class, and now compare nine methods with the frontier of their class. Some of the added methods improve on the original ones, but all nine remain below their frontier. Apart from one fixed-budget method (TALE), the gaps are largest for dynamic early exit.

Methods. Methods added in the rebuttal are marked *.

  • Dynamic routing (DR): ThinkSwitcher, Scout Before You Route *, CARROT (KNN) *
  • Fixed budget (FB): Predictive Scheduling, ThoughtTerminator *, TALE *
  • Dynamic early exit (DEE): ThinkBrake, RCPD *, LYNX *

Information available to the methods. DR and FB methods decide from the question alone. DEE methods are replayed along the trajectory and see only the current and earlier prefixes. No method sees test correctness, later parts of the trajectory or the final thinking length. Learned components are fitted on training data, operating parameters are chosen on development data (highest accuracy, then fewest thinking tokens), and test correctness is used only to score the decisions. Predictive Scheduling uses the corrected supervision (our response to Reviewer bSdV, Q3).

Setup. GPT-OSS-20B and DeepSeek-R1-Distill-Qwen-7B on the same 16 test questions from each of AIME, MATH500, GPQA-Diamond and LiveCodeBench, with 16 trajectories per question and 16 answers per position (2,048 test trajectories). Benchmarks have equal weight; learned methods are averaged over three seeds.

Results at the development-selected operating points. Cost is the retained thinking tokens as a percentage of full thinking, computed per benchmark and then averaged. "Frontier" is the accuracy of the class frontier at the same cost as the method, matched within each benchmark and seed before averaging; the gap is frontier minus method. G: GPT-OSS-20B; D: DeepSeek-R1-Distill-Qwen-7B.

ClassMethodG: cost (%)G: accuracy (%)G: frontier (%)G: gap (pp)D: cost (%)D: accuracy (%)D: frontier (%)D: gap (pp)
DRThinkSwitcher100.079.5680.040.48100.059.9262.042.12
DRScout Before You Route *99.779.5380.040.5184.160.2061.971.77
DRCARROT (KNN) *100.079.5680.040.4886.761.0161.800.79
FBPredictive Scheduling98.379.5281.411.9098.659.9264.654.74
FBThoughtTerminator *95.079.1381.322.2095.759.8464.654.82
FBTALE *28.356.1568.1612.0129.348.9060.3911.49
DEEThinkBrake99.579.4586.396.94100.059.9271.1411.22
DEERCPD *99.579.4786.396.9297.659.8971.1411.25
DEELYNX *92.378.3586.398.0484.256.8271.1414.32

On these test questions, full-thinking accuracy is 79.56% and 59.92%. Most selected settings are close to full thinking because the selection rule puts accuracy first, and for these two models full thinking is close to the most accurate setting of most methods. The methods do stop earlier at other settings: in Figure C, most methods' tested settings range from little thinking to nearly full thinking, and on DeepSeek-R1-Distill-Qwen-7B several of them exceed full-thinking accuracy.

What the comparison shows.

  • Some added methods improve the trade-off. On DeepSeek-R1-Distill-Qwen-7B, CARROT reaches 61.01% at a cost of 86.7%, whereas ThinkSwitcher stays at full thinking (59.92%).
  • Routing methods are close to their frontier (gaps of 0.48 to 2.12 pp), but that frontier is itself the lowest of the three.
  • The largest unused potential is in dynamic early exit (gaps of 6.92 to 14.32 pp). Under rotating 15+1 held-out scoring on the full benchmark, the accuracy ceilings of the three frontiers keep the same order (see our response to Reviewer bSdV, Q1, part 3).

Figure C (anonymous link) shows all tested settings of the nine methods against the frontier of their class. The frontiers in the figure use a common relative budget cap for the four benchmarks, so distances differ slightly from the matched gaps in the table.

We will add these methods and curves to the paper.

W4: The main cost metric considers retained reasoning tokens but excludes answer-generation costs, feature extraction, stopping-signal computation, and other inference overhead. Could these costs also be included in the metric?

We have added the cost of answer generation, which can be measured independently of any particular method: in all eight settings we examined, the thinking tokens saved far exceed the additional answer tokens. The other overheads depend on how a specific method is implemented, so we keep them outside the common metric and state this scope explicitly.

Why the metric counts retained thinking tokens. OTBench compares what three classes of stopping decisions (DR, FB, DEE) allow. Retained thinking tokens are defined in the same way for every method in every class. Feature extraction, predictor inference and stopping-signal computation differ from one method and implementation to another, not from one class to another; including them would turn the comparison of decision classes into a comparison of implementations. Our efficiency statements are about retained thinking tokens, not about latency or FLOPs. The OTBench section and Appendix A.4 of the paper state that the metric counts retained thinking tokens and excludes these overheads; we will repeat it where the results are discussed.

Answer-generation cost. Answers generated after an early stop can be longer than answers after full thinking. We measured this on GPT-OSS-20B and DeepSeek-R1-Distill-Qwen-7B with 64 questions from each of the four benchmarks, one trajectory per model and question (512 trajectories), b = 256 and 16 answers per position. The answer cost at a position is the mean length of its 16 answers.

For each strategy class we consider two operating points on its frontier: the smallest budget that reaches full-thinking accuracy, and the smallest budget that reaches the highest accuracy of the class. The stopping decisions and accuracies are kept as they are, and the cost of one answer is added at each selected position. With one trajectory per question, FB and DEE allow the same actions and coincide.

The table gives the change relative to full thinking, in mean tokens per question, with equal weight for the four benchmarks. A minus sign is a saving, a plus sign an increase.

ModelClassOperating pointThinking tokensAnswer tokensTotal tokens
GPT-OSS-20BDRReaches full-thinking accuracy−2549+82−2467
GPT-OSS-20BDRReaches class maximum−2054+55−1999
GPT-OSS-20BFB/DEEReaches full-thinking accuracy−3345+132−3213
GPT-OSS-20BFB/DEEReaches class maximum−2272+106−2166
R1-Distill-Qwen-7BDRReaches full-thinking accuracy−7005+452−6553
R1-Distill-Qwen-7BDRReaches class maximum−5889+358−5530
R1-Distill-Qwen-7BFB/DEEReaches full-thinking accuracy−7979+673−7306
R1-Distill-Qwen-7BFB/DEEReaches class maximum−5766+520−5246

Entries are rounded independently. For reference, full thinking plus one answer costs 5,570 tokens per question for GPT-OSS-20B and 9,137 for DeepSeek-R1-Distill-Qwen-7B.

Answers do become longer after an early stop, by 55 to 673 tokens, but this offsets only 3% to 9% of the thinking tokens saved. For example, at the FB/DEE budget that reaches full-thinking accuracy, the total saving is 3,213 of 5,570 tokens for GPT-OSS-20B and 7,306 of 9,137 for DeepSeek-R1-Distill-Qwen-7B. Including the answer cost therefore does not change the conclusions drawn from the retained-token metric. We will add this analysis to the appendix.

Typo. Thank you also for spotting "acciracy" in Figure 1; it is corrected in the revision.

Reviewer_NNsyOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies overthinking in reasoning models. It proposes Step-wise Truncation Probing (STP), which truncates a fixed reasoning trajectory at multiple points and resamples answers to build token–accuracy curves, and defines two metrics from them: Overthinking Ratio (OR) and Accuracy Degradation (AD). Across six models and four benchmarks, 64.3% of thinking tokens come after peak accuracy, and full thinking loses 9.03 points. Dynamic routing, fixed budgets, and dynamic early exit are unified as budget-constrained optimization. The released benchmark, OTBench, shows that existing public methods fall well short of these frontiers.

Critical strengths

OTBench offers a large-scale, carefully controlled resource with a unified evaluation protocol and oracle reference frontiers, enabling cheap and comparable offline evaluation of termination methods.

Critical weaknesses
  1. The novelty and contribution of this work are limited. The proposed performance metrics and evaluation method are very simple. Stepwise truncation probing is a common method, such as RCP [1]. Given the extensive analysis, the authors should go further and propose a method that addresses the overthinking problem, rather than only measuring it.
  2. The claimed “theoretical mitigation potential” is overstated, as the Pareto frontiers are empirical hindsight-oracle results rather than theoretically derived bounds.

[1] The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis

Questions for authors

N/A

4 comments

W1 (part 1 of 3): What are the novelty and contributions of this work compared with prior work such as RCP [1]? If this work does not propose a new method that addresses the overthinking problem, does only measuring it constitute a sufficient contribution?

We do not propose a new stopping method. What the paper contributes is knowledge about overthinking that a stopping method does not provide: (1) for each model and task, how much of the thinking comes after the best stopping point and how much accuracy continued thinking costs; (2) that about one trajectory in five loses accuracy after its peak, and that these trajectories carry almost all of the degradation; (3) how much each class of stopping decisions can gain over full thinking, and how far nine existing methods, including RCPD, are from that. Designing online policies that approach these references is the direction named in Appendix E, and it needs such a reference first. We believe this is the kind of contribution described in ICLR's reviewer guidelines ("new, relevant, impactful knowledge (incl., empirical, theoretical, for practitioners, etc)"); "datasets and benchmarks" is also a listed subject area. The comparison with RCP below, for which we ran new experiments, shows what the measurement adds.

Summary. RCP and our work both truncate reasoning and probe the answer, but they measure different things. RCP finds the point where the answer stops changing. We measure whether the answer is correct at every stopping point, and from this we compute how much accuracy and how many thinking tokens each class of stopping methods could gain over full thinking. Our measurements make it possible to test the assumptions behind convergence-based stopping, and they show two limits: accuracy does not always stay at its best level once it is reached, and a converged answer can come later, or be less accurate, than the best stopping point. These limits become visible only when correctness is measured at every stopping point.

What RCP does. RCP [1] probes answers on truncated prefixes and takes the first position at which the length and the semantics of the answers have both stabilized relative to the final answer distribution. Its online detector, RCPD, uses the rank of the end-of-thinking token. Reading this position as the point where reasoning is complete rests on two assumptions, which RCP motivates with case studies but does not test across trajectories:

  1. A trajectory has two phases: exploration, followed by sustained convergence.
  2. Convergence means sufficiency: once the length and semantics of the answer are stable, reasoning is sufficient and later computation is largely redundant.

Because STP measures accuracy at every stopping position, we can check what these assumptions imply for task performance. Part 1 examines the first assumption; Parts 2 and 3 examine the second.

Assumption 1: accuracy does not always stay near its best level. We classified all 92,448 trajectories of the main experiment (six models, four benchmarks) by the shape of their accuracy curve. A curve is rise-stable if, after it first reaches its highest accuracy, the average accuracy of the later positions is within 1/16 of that peak (6.25 pp, i.e. one answer out of the 16 per position). Anything may happen before the peak. Curves that are constant, and curves that reach their peak only at the end, count as rise-stable. This is the performance counterpart of "exploration followed by sustained convergence"; it classifies accuracy curves and does not use RCP's semantic signal.

CriterionRise-stableNot rise-stable
Main: average drop after the peak ≤ 1/1673,114 (79.09%)19,334 (20.91%)
Strict: no drop after the peak48,657 (52.63%)43,791 (47.37%)

About one trajectory in five does not keep its best accuracy, and these trajectories account for 96.60% of the total AD in the main experiment. In other words, almost all of the accuracy lost by continued thinking comes from trajectories whose accuracy does not follow this two-phase pattern. Appendix C.1 ("Representative STP Accuracy Curves") already shows examples of such curves. All groups in Parts 2 and 3 use the main criterion.

[1] The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis. arXiv:2508.17627.

W1 (part 2 of 3): does convergence mark the best stopping point? Comparison against densely probed reference curves

Assumption 2: a converged answer is not always the best stopping point. We compared the position selected by RCP with the position of highest accuracy on the same trajectories.

Setup. We used 16 trajectories from GPT-OSS-20B and DeepSeek-R1-Distill-Qwen-7B: for each model and each of the four benchmarks, one rise-stable and one non-rise-stable trajectory. At every position we collected about 2,048 answers instead of the usual 16, which gives an accurate reference curve for each trajectory. The reference position is the first position with the highest accuracy on this curve. RCP is run as described in its paper, on 16 answers per position. If RCP detects no stopping point, the trajectory is evaluated at full thinking.

Where RCP stops. "Earlier / same / later" compares the RCP position with the reference position. The position difference is the absolute distance between the two, as a percentage of the full thinking length. The last two columns give the share of thinking tokens saved by stopping at the reference position (this is the reference OR) and at the RCP position.

GroupTrajectoriesRCP earlier / same / later (%)Mean absolute position difference (% of full length)Reference OR (%)RCP token savings (%)
Rise-stable887.50 / 12.50 / 0.0026.679.9036.57
Not rise-stable812.50 / 12.50 / 75.0037.9661.2131.98
All1650.00 / 12.50 / 37.5032.3235.5534.28

What accuracy RCP obtains. All accuracies are measured on the reference curves. AD is the peak accuracy minus the full-thinking accuracy.

GroupPeak accuracy (%)RCP accuracy (%)Full-thinking accuracy (%)AD (pp)RCP − full thinking (pp)
Rise-stable99.9598.8299.950.01−1.13
Not rise-stable51.6713.490.0451.63+13.46
All75.8156.1649.9925.82+6.16

On rise-stable trajectories RCP works well. It stops earlier than the reference position in most cases, saves 36.57% of the thinking tokens and loses only 1.13 pp of accuracy. The curve is close to its ceiling there, so a different position costs almost nothing.

On the other trajectories RCP stops too late. It stops after the reference position in 75% of the cases. It recovers 13.46 pp over full thinking but remains about 38 pp below the peak, and it saves 31.98% of the thinking tokens, although 61.21% could be saved. On these trajectories accuracy peaks and then falls to almost zero at full thinking (0.04%), so answers can be stable, as RCP's criterion requires, and still be wrong. This group carries 99.99% of the AD in these 16 trajectories.

The two groups therefore differ sharply: where accuracy stays near its best level, stopping at convergence costs little; where it does not, the converged position misses a much better stopping point. The second group accounts for nearly all of the accuracy degradation.

(The same 16 trajectories and reference curves are used for the error analysis of OR and AD in our response to Reviewer bSdV, Q1.)

W1 (part 3 of 3): comparison on the main benchmark, and our contributions

The same comparison with the standard 16 probes. We drew one trajectory at random for every model and question of the main experiment: 5,778 trajectories, of which 4,606 are rise-stable and 1,172 are not. The reference is posterior OR, the more accurate OR estimator from our response to Reviewer bSdV, Q1 (part 2), and its position t_post. The position columns compare the RCP position with t_post and leave out 236 rise-stable curves on which no answer is correct at any position. Qwen3 stands for the Thinking-2507 models, R1-Distill for DeepSeek-R1-Distill-Qwen.

GroupModelTrajectoriesPosterior OR (%)Original OR (%)RCP token savings (%)RCP later than reference (%)Mean absolute position difference (% of full length)
Rise-stableGPT-OSS-20B78721.4536.3212.4272.4324.99
Rise-stableQwen3-4B76639.6280.1818.6482.6424.58
Rise-stableQwen3-30B-A3B76841.4483.6721.5382.2923.59
Rise-stableR1-Distill-7B64136.4871.2527.4257.2523.00
Rise-stableR1-Distill-14B69536.0070.9727.5057.4121.16
Rise-stableR1-Distill-32B71338.0076.2926.6959.4723.33
Rise-stableAverage437035.5069.7822.3768.5823.44
Not rise-stableGPT-OSS-20B15750.9658.3232.3561.1534.94
Not rise-stableQwen3-4B16444.7961.2516.5183.5431.63
Not rise-stableQwen3-30B-A3B16642.5768.0416.7487.9529.03
Not rise-stableR1-Distill-7B26553.6771.2919.7481.5139.21
Not rise-stableR1-Distill-14B21548.5262.2321.8281.8630.87
Not rise-stableR1-Distill-32B20548.7165.8828.3876.5932.55
Not rise-stableAverage117248.2064.5022.5978.7733.04

AD is peak minus full-thinking accuracy.

GroupModelTrajectoriesPeak accuracy (%)RCP accuracy (%)Full-thinking accuracy (%)AD (pp)RCP − full thinking (pp)
Rise-stableGPT-OSS-20B80693.4983.7493.080.42−9.34
Rise-stableQwen3-4B79992.8590.9192.280.57−1.37
Rise-stableQwen3-30B-A3B79793.9491.9493.580.36−1.64
Rise-stableR1-Distill-7B69885.3580.4484.920.43−4.48
Rise-stableR1-Distill-14B74888.6084.8888.270.33−3.39
Rise-stableR1-Distill-32B75891.1987.8690.970.21−3.11
Rise-stableAverage460690.9086.6390.520.39−3.89
Not rise-stableGPT-OSS-20B15762.9832.3232.8830.10−0.56
Not rise-stableQwen3-4B16474.2040.9336.3937.80+4.54
Not rise-stableQwen3-30B-A3B16683.8958.4353.8030.08+4.63
Not rise-stableR1-Distill-7B26570.3336.7235.4034.93+1.32
Not rise-stableR1-Distill-14B21569.0134.8834.0434.97+0.84
Not rise-stableR1-Distill-32B20572.5637.5938.6633.90−1.07
Not rise-stableAverage117272.1640.1538.5333.63+1.62

The main benchmark confirms the result for trajectories that are not rise-stable. RCP saves about 22% of the thinking tokens in both groups, less than posterior OR (35.50% and 48.20%), and it stops later than the reference position in most cases (68.58% and 78.77%). For the rise-stable group this differs from Part 2 (mostly earlier); both the trajectories and the reference differ (there the first maximum of a densely probed curve, here the posterior position from 16 answers). On rise-stable trajectories it loses 3.89 pp of accuracy relative to full thinking. On the other trajectories it gains 1.62 pp over full thinking but remains 32.01 pp below the peak. On GPT-OSS-20B, RCP lowers accuracy in both groups (−9.34 and −0.56 pp). The online detector RCPD shows the same picture: its gap to the DEE frontier at the same cost is 6.92 pp on GPT-OSS-20B and 11.25 pp on DeepSeek-R1-Distill-Qwen-7B (see our response to Reviewer w43A, W3).

Contributions. These findings are our answer to whether measurement is a sufficient contribution. (1) STP measures correctness at every stopping point of a fixed trajectory, which a convergence signal does not provide. It shows that the best accuracy often does not persist and that convergence can miss the best stopping point. (2) OR and AD separate redundant computation from accuracy degradation, and the comparison above needs both. (3) The class frontiers and OTBench show how far a stopping method, here RCPD and eight others, is from the best that its class of decisions can achieve. New methods can be designed and judged against this reference. We will add the RCP and RCPD comparisons to the paper.

W2: The Pareto frontiers are empirical hindsight-oracle results rather than theoretically derived bounds. Is the claimed “theoretical mitigation potential” overstated?

We agree with your description: the frontiers are empirical. They give the best accuracy–cost trade-off that each class of stopping decisions could have achieved on the trajectories and stopping positions we measured, chosen in hindsight. They are not derived bounds, and they do not guarantee what an online method can reach.

By “theoretical” we meant an idealized reference as opposed to a deployable method. The Introduction describes the frontiers as follows:

“These frontiers represent the best admissible token–accuracy trade-offs under the measured action space and each strategy's decision process, and serve as idealized theoretical references rather than deployable stopping algorithms.”

Appendix E (Limitations) likewise states that offline frontiers are evaluation references rather than deployable stopping policies. We see that “theoretical mitigation potential” can nevertheless be read as a claim about a derived bound. We will replace it with “empirical oracle potential”, revise in the same way every other place in the paper where “theoretical” describes the frontiers or the potential of a strategy, and state in the frontier section that the frontiers are hindsight optima over the measured actions.

Our responses to Reviewer w43A (W2) and Reviewer bSdV (Q1, part 3) quantify how accurately these empirical frontiers are estimated with 16 answers per position.

Reviewer_bSdVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Step-wise Truncation Probing (STP) to measure answer accuracy at several prefixes of a fixed reasoning trajectory. It defines Overthinking Ratio (OR) and Accuracy Degradation (AD) relative to the earliest position with the highest measured accuracy, then constructs offline oracle token–accuracy frontiers for dynamic routing, fixed budgets, and dynamic early exit. OTBench packages these measurements into a common evaluation protocol. Across six models and four benchmark groups, the reported averages are OR 64.3% and AD 9.03 percentage points. The experiments use 963 questions, 16 trajectories per model–question pair, and 16 answer probes per position.

Critical strengths
  1. STP controls an important confound. Holding the reasoning trajectory fixed while resampling answers at different prefixes avoids comparing independently generated short and long trajectories with different content. The protocol also distinguishes trajectory variation from probe-count sensitivity and documents the token-boundary construction in detail.

  2. The strategy comparison has a clear formulation. The paper specifies the decision units and admissible actions for each strategy, constructs exact optima of the measured action table, and uses question-level train/validation/test partitions for practical methods. The feasible-set nesting makes the qualitative frontier ordering explicit; the measured gap sizes are the substantive empirical question.

Critical weaknesses
  1. The reference accuracies and oracle rewards have not been validated on independent probes. Each accuracy estimate uses 16 answers, and the reference is selected by maximizing over many positions. This can inflate the selected accuracy and produce positive AD even when continuation has no underlying accuracy cost. The selected position, and therefore OR, can also be unstable; its bias need not have a fixed direction.

    Appendix B.2 shows that reducing the recorded probe pool from 16 to 4 changes expected OR from 64.34% to 74.69% and AD from 9.03 to 11.07 percentage points. This establishes sensitivity within the recorded experiment, but does not show that 16 probes locate the underlying optimum. Appendix A.5 holds the probe curves fixed during the question/trajectory bootstrap.

    The same concern applies to the frontiers: actions are selected and scored using the same correctness counts. The frontiers are valid optima of the recorded table, but their interpretation as attainable answer accuracy requires independent evaluation of the selected actions. Selection optimism may also affect the gaps between strategies with different action-space sizes.

  2. The practical-method gaps have additional explanations that are not separated from strategy potential. Appendix A.4 trains the fixed-budget predictor on the mean of individual trajectories' optimal retained lengths, although that target differs from the optimal shared-cap action. This mismatch could contribute to the fixed-budget method's distance from its oracle. Only one adaptation represents each strategy, and the token metric excludes feature extraction, predictor inference, and stopping-signal computation. These are disclosed limitations, but they restrict conclusions about deployment efficiency and how much potential practical methods leave unrealized.

Questions for authors
  1. Can you select reference positions and oracle actions using one probe set, then evaluate the selected actions with independent answer probes? Please report the independently evaluated accuracy differences and frontier rewards, together with the stability of reference positions and OR as probe count increases. How do the strategy gaps change?
  2. How do OR, AD, and the frontiers change when answers are actually probed at resolutions b=128 and b=512? Appendix D currently repartitions trajectories and reports step lengths without recomputing the answer measurements.
  3. How much of the fixed-budget gap remains when the predictor is supervised with optimal shared-cap actions on the training questions? Please also report inference overhead for the practical methods, or explain the intended scope of the retained-token comparison.
  4. Please reconcile the 70.5B “reasoning tokens” in the abstract and conclusion with the 576,688,123 stored thinking tokens in Appendix D. Which parts count unique trajectory tokens, repeated prefix processing, or newly generated probe tokens?
9 comments

Q1, W1, W2 and W3 (part 1 of 5): Can you select reference positions and oracle actions using one probe set, then evaluate the selected actions with independent answer probes? Please report the independently evaluated accuracy differences and frontier rewards, together with the stability of reference positions and OR as probe count increases. How do the strategy gaps change?

We ran the independent evaluation you asked for and measured how accurate each estimate is. In brief:

  1. The original estimates are optimistic, as you suspected. With 16 probes, choosing a position and scoring it on the same answers biases AD upward, and the chosen position (and therefore OR) is unstable.
  2. Independent scoring is not a more accurate estimate. It errs in the opposite direction, and by more. With the same 16 answers per position, held-out AD underestimates the reference, and its RMSE is higher than that of the original estimate (1.7 to 1.9 times as high for a single trajectory). On all 92,448 trajectories, held-out AD is 2.36 pp (8+8) and 4.78 pp (rotating 15+1), against 9.03 pp for the original estimator; it is clearly positive on GPQA-Diamond and LiveCodeBench and close to zero or negative on AIME and MATH500 (Part 4).
  3. OR was the metric that needed a better estimator. We now average over the posterior probability of where the peak lies, which roughly halves the per-trajectory error. Recomputed on all 92,448 trajectories, mean OR changes from 64.34% to 36.93%; AD is 9.03 pp as before.

Part 1 compares the estimators at an equal probe budget; Part 2 varies the probe count and introduces the new OR estimator; Part 3 gives the recomputed main table and the strategy gaps; Part 4 gives the held-out results on the full benchmark; Part 5 gives the results by trajectory group and the error expressions.

No estimator of OR or AD is unbiased for every accuracy curve. With a finite number of Bernoulli probes, the expected value of any estimator is a polynomial in the true accuracies. AD is a maximum minus an endpoint and has kinks; OR is the location of the first maximum and has jumps. Neither is a polynomial function of the true accuracies. The practical question is therefore which estimator has the smallest error, so we measured the errors directly.

Estimators. You did not specify a split, so we implemented two. In 8+8, eight answers per position select the reference position and the other eight score it. In rotating 15+1, fifteen answers select, the remaining one scores, and we average over all sixteen choices of the held-out answer. Both use the same 16 answers per position as the original estimator, which selects and scores with all 16.

Reference. We took 16 trajectories from GPT-OSS-20B and DeepSeek-R1-Distill-Qwen-7B: for each model and each of the four benchmarks, one trajectory whose accuracy rises and then stays high, and one whose accuracy degrades after its peak. They were chosen to cover both kinds of curve, not to match their shares in the benchmark; Part 5 reports each group separately. At every position of these trajectories we collected about 2,048 answers instead of 16. The resulting curves give the reference values: OR = 35.55% and AD = 25.82 pp. We then repeatedly drew 16 of these answers per position without replacement and applied each estimator. (Our response to Reviewer NNsy, W1 uses the same reference curves.)

Results at 16 answers per position. OR is in %, AD and errors in pp. SD and RMSE are given for a single trajectory / for the average over the 16 trajectories.

MetricEstimatorMean estimateReferenceBiasSDRMSE
ADOriginal29.8425.82+4.026.33 / 1.589.37 / 4.32
AD8+817.7725.82−8.0516.20 / 4.0518.22 / 9.01
ADRotating 15+121.2325.82−4.5815.37 / 3.8416.11 / 5.98
OROriginal53.1335.55+17.5812.18 / 3.0436.32 / 17.84
OR8+857.9035.55+22.3412.98 / 3.2541.79 / 22.58
ORRotating 15+153.5435.55+17.9910.78 / 2.7036.38 / 18.19

The errors include every effect of finite probing: optimism from reusing answers, accuracy lost by missing the true peak, and fluctuation of the peak itself.

AD. The original estimator is biased upward by 4.02 pp. Both held-out estimators are biased downward by more (−8.05 and −4.58 pp), and their RMSE is higher both for a single trajectory (18.22 and 16.11 versus 9.37 pp) and for the average (9.01 and 5.98 versus 4.32 pp). Holding out answers removes the optimism in scoring, but the position chosen from fewer answers is more often below the true peak, and a score based on eight answers or one answer is noisy.

OR. All three estimators overestimate OR by 17.6 to 22.3 pp, and holding out answers does not help. The difficulty is locating the peak, not scoring it. Part 2 explains why and gives a better estimator.

Q1, W1, W2 and W3 (part 2 of 5): stability as the probe count grows, and a better estimator for OR

With more probes, the original estimates approach the reference: AD quickly, OR slowly. We repeated the comparison of Part 1 with 16 to 1,024 answers per position, drawn from the pool of about 2,048 answers that defines the reference (OR = 35.55%, AD = 25.82 pp). RMSE is shown as single trajectory / average over the 16 trajectories. The last column is the mean absolute distance, in tokens, between the estimated and the reference position.

Probes per positionOR mean (%)OR RMSE (pp)AD mean (pp)AD RMSE (pp)Mean absolute token difference from reference position
1653.1336.32 / 17.8429.849.37 / 4.321307.53
3248.8030.71 / 13.5928.166.32 / 2.631136.18
6444.6325.01 / 9.5327.194.27 / 1.62958.67
12841.4520.65 / 6.4626.602.88 / 1.00804.27
25639.0817.26 / 4.3226.251.93 / 0.61658.46
51236.9313.63 / 2.7426.041.25 / 0.37485.74
102435.279.22 / 1.8225.910.72 / 0.20282.66

The decrease of the errors agrees with the bound for drawing n of N answers without replacement,

RMSE(n) ≤ C · sqrt[(N − n) / (n (N − 1))],

where C depends on the accuracy curve but not on n; it also bounds the absolute bias and the SD. It holds for AD on any fixed set of positions, and for OR when the first peak is strictly higher than every competing position whose answers vary; positions tied with it must have deterministic answers. These reference curves meet the condition (three have ties at 100% accuracy).

Why OR converges slowly. Positions that are far apart can have almost the same accuracy. Many answers are needed to tell which is highest, and each wrong choice moves the position a long way, while AD hardly changes.

Posterior OR: what changes, and why. Posterior OR changes how OR is estimated from the 16 answers. The original estimator commits to the one position with the highest measured accuracy. When several positions cannot be told apart from the best one, noise decides which is chosen, and taking the earliest of them favours early positions, hence a high OR (+17.58 pp in Part 1). Posterior OR instead weights every position by the posterior probability w_i that it has the highest accuracy:

OR_post = Σ_i w_i · (1 − t_i / T)

Here t_i is the thinking length at position i and T the full length. With s_i correct answers out of n, the accuracy at position i has posterior Beta(s_i + 1/2, n − s_i + 1/2) (Jeffreys prior). When the answers identify the peak clearly, the weight concentrates on it and the two estimators agree; when they do not, posterior OR averages over the candidate positions instead of jumping between them. Under this model, OR_post is the estimate with the smallest posterior expected squared error. The table uses the same reference and draws; t_post = T(1 − OR_post) is the corresponding position.

Probes per positionPosterior OR mean (%)Bias (pp)SD (pp)RMSE (pp)Mean absolute token difference from reference position
1637.70+2.145.47 / 1.3718.56 / 2.541002.35
3236.10+0.555.55 / 1.3916.50 / 1.49923.49
6434.86−0.705.56 / 1.3914.62 / 1.55843.85
12833.88−1.685.38 / 1.3413.11 / 2.15768.05
25633.07−2.485.04 / 1.2611.86 / 2.78694.08
51232.34−3.214.61 / 1.1510.52 / 3.41605.12
102431.83−3.723.57 / 0.898.82 / 3.83481.63

At the 16 probes used in our experiments, posterior OR roughly halves the error for a single trajectory: the RMSE falls from 36.32 to 18.56 pp. The gain comes from trajectories whose peak is hard to locate: on rise-stable trajectories the RMSE falls from 48.14 to 19.91 pp; on the others it is unchanged (17.91 and 17.11 pp; Part 5). For the average over the 16 trajectories, the RMSE falls from 17.84 to 2.54 pp and the bias from +17.58 to +2.14 pp; the distance to the reference position falls from 1307.53 to 1002.35 tokens. Its RMSE is below that of the original estimator at every probe count in the table for a single trajectory, and up to 256 probes for the average. At very large probe counts it falls below the reference (−3.72 pp at 1,024), because along a plateau it keeps sharing weight among positions that are practically tied, whereas the reference takes the earliest of them.

Further evidence. The estimate is also less sensitive to the probing resolution (see Q2) and depends little on the prior: on the full benchmark, mean OR is 36.93% (Jeffreys), 37.11% (uniform) and 37.42% (Beta(2, 2)).

Posterior averaging does not improve AD, so AD keeps the original estimator, which has the lowest error in Part 1. We will add posterior OR to the paper as the main OR estimate, together with this error analysis.

Q1, W1, W2 and W3 (part 3 of 5): recomputed main results and the strategy gaps

Main results with posterior OR. We recomputed OR for all 92,448 trajectories from the 16 answers per position that were already recorded. Each cell shows posterior OR (%) / AD (pp). As in the paper, the four benchmarks have equal weight within a model, and the six models have equal weight in the last row.

ModelAIMEMATH500GPQA-DiamondLiveCodeBenchAverage
GPT-OSS-20B20.75 / 3.0227.07 / 1.6628.62 / 16.1532.33 / 9.8927.19 / 7.68
Qwen3-4B-Thinking-250732.38 / 2.5244.42 / 1.2442.23 / 14.2334.15 / 18.2838.30 / 9.07
Qwen3-30B-A3B-Thinking-250731.12 / 1.5345.78 / 1.7341.51 / 10.6935.63 / 8.6338.51 / 5.64
DeepSeek-R1-Distill-Qwen-7B38.66 / 9.7642.18 / 2.8243.25 / 25.6440.35 / 15.3741.11 / 13.40
DeepSeek-R1-Distill-Qwen-14B34.87 / 7.5239.89 / 1.8042.14 / 18.7333.44 / 9.9537.58 / 9.50
DeepSeek-R1-Distill-Qwen-32B33.45 / 4.9642.56 / 1.7540.71 / 16.4238.90 / 12.4038.91 / 8.88
Average31.87 / 4.8840.32 / 1.8339.74 / 16.9835.80 / 12.4236.93 / 9.03

Mean OR changes from 64.34% to 36.93%. The AD values are those of the paper (mean 9.03 pp): we keep the original AD estimator and its measurements, because it has the lowest error of the three estimators in Part 1, and we report its upward bias. Part 4 gives the held-out values for comparison. With the more accurate estimator, more than a third of the thinking tokens still come after the reference position on average, and GPT-OSS-20B still has the lowest OR. The conclusion that overthinking involves both redundant computation and accuracy degradation remains. Across the 24 model–benchmark cells, the Spearman correlation between OR and AD changes from −0.330 to +0.089, so the paper's observation that a larger OR does not imply a larger AD also remains; we will update these numbers. In the revision, the introduction, Table 1 and the related text will report posterior OR; the original OR and the error analysis of Parts 1 and 2 will go to the appendix, where we will also state the upward bias of AD at 16 probes. The intervals in the paper resample questions and trajectories with the recorded answers held fixed (Appendix A.5), so they do not include the probe-level error measured here; we will say so next to them.

Frontiers against the reference. We ran the same three-way comparison for the oracle frontiers on the same 16 reference trajectories. To fit the rebuttal period, this experiment has one trajectory per question, so FB and DEE allow the same actions and coincide. At a budget of 50% of the full-thinking tokens, the gap FB/DEE − DR is 12.58 pp for the reference, 16.99 pp for the original estimator, and 5.72 and 8.30 pp for 8+8 and rotating 15+1. As for AD, the original frontier is optimistic (+4.41 pp), the held-out estimates are pessimistic by a similar or larger amount, and the original estimate has the lowest RMSE. The full table and a figure covering all budgets are in our response to Reviewer w43A, W2.

FB versus DEE. The order of the three classes does not rest on an estimate. As Section 4.2 of the paper states, "A zero or sufficiently large shared cap reproduces either routing endpoint, and independent trajectory-level choices can reproduce any shared-cap assignment", so every DR policy is an FB policy and every FB policy is a DEE policy, and a frontier cannot lie below that of a smaller class. What is estimated is the size of the gaps. Because the experiment above cannot separate FB from DEE, we applied the three estimators to the accuracy ceiling of each class (its frontier without a budget limit) on all 92,448 trajectories, with 16 trajectories per question. Here 8+8 splits the recorded answers into even and odd indices and averages both directions, because the frontier optimization has no closed-form average over all splits.

ClassOriginal 16 (%)8+8 (%)Rotating 15+1 (%)
DR73.1973.1473.16
FB75.9374.8375.16
DEE81.3674.7377.12

Full thinking gives 72.33%. Held-out scoring lowers DEE the most: DEE chooses a position for each trajectory from that trajectory's answers alone, whereas FB chooses one cap per question from the answers of 16 trajectories, so the DEE selection is the noisiest and its held-out value is the most pessimistic (Parts 1 and 5). With only eight answers for selection it falls 0.10 pp below FB; given the guaranteed order, this is estimation error of the 8+8 split. The DEE − FB gap is 5.43 pp with the original estimator and 1.96 pp with rotating 15+1. We will report these values, and the upward bias of the original frontier, next to the frontier results.

Comments on the paper

1 comment

Overview of our response

We thank the reviewers for their careful reading. The paper has three parts: a measurement of overthinking (STP, with the metrics OR and AD), oracle frontiers that show what each class of stopping decisions could achieve, and a benchmark (OTBench) that places practical methods against these frontiers. The reviewers' questions fall under these three parts. This note explains how we approached them; each question is answered as the reviewer posed it in the individual responses.

1. The measurement: OR, AD and the frontiers with 16 answers per position (bSdV Q1, Q2, W1–W3; w43A W1, W2)

Selecting the best position and scoring it on the same answers is optimistic, and the reviewers asked us to score the selected positions on independent answers. We did this for all trajectories of the benchmark and report the results. We also wanted to know which estimate is closer to the truth, because independent scoring leaves fewer answers for choosing the position and gives a noisier score. We therefore probed a set of trajectories far more densely, covering both curves that stay at their best accuracy and curves that degrade, and used them as a reference for the original estimator and for two held-out estimators at the same probe budget.

Three things follow. For AD and for the frontiers, the original estimator is the closest to the reference; we keep it and now state its error next to the results, together with the held-out values. For OR the difficulty is different: the error comes from locating the peak among positions of nearly equal accuracy, which independent scoring cannot fix. We therefore estimate OR by weighting each position by the posterior probability that it is the most accurate one. This roughly halves the error and changes mean OR from 64.34% to 36.93%; the paper's conclusions about OR remain. The order of the three frontiers is not an empirical result that selection could produce: it follows from the nesting of the action sets, and the experiments only measure the size of the gaps. We also replaced the repartition analysis of Appendix D with answers probed at three resolutions; the metrics and the frontiers change little.

2. The contribution: novelty relative to RCP, and the meaning of the frontiers (NNsy W1, W2)

RCP and STP share the truncate-and-probe procedure but ask different questions. RCP finds where the answer stops changing; STP measures whether the answer is correct at every stopping point. Reading a convergence point as the end of useful reasoning assumes that a trajectory explores and then converges for good, and that a converged answer is a sufficient one. A convergence signal cannot test what these assumptions mean for accuracy. Our measurements can, and we ran this test with RCP itself: about one trajectory in five does not keep its best accuracy, these trajectories carry almost all of the accuracy degradation, and on them RCP typically stops late and well below the best accuracy.

This is the kind of knowledge the paper contributes: how much thinking is redundant, how much accuracy continued thinking costs, and what each class of stopping decisions could achieve (ICLR's reviewer guidelines describe such "new, relevant, impactful knowledge" as a contribution). Online methods that approach these references are the natural next step, as Appendix E notes, and they need such a reference to be designed and judged. We agree with the reviewer that the frontiers are empirical hindsight optima rather than derived bounds; by "theoretical" we meant idealized references, and we now say "empirical oracle" to avoid this reading.

3. The benchmark: practical methods, cost and token accounting (w43A W3, W4; bSdV Q3, Q4, W4)

No online method has the information the oracle uses, so a frontier is a reference for a class of decisions, and what is attainable online is shown by practical methods evaluated under the same protocol. The reviewers asked for a broader comparison and for a fixed-budget predictor supervised with the oracle's action. We did both: the predictor is now trained on that action, and each class is represented by three methods instead of one. On cost, the metric counts retained thinking tokens because they are defined identically for every method, whereas other overheads depend on how a method is implemented. The one overhead that does not, the length of the answer after an early stop, we measured: it offsets only a small part of the thinking tokens saved. The two token counts are also clarified.

What changes in the paper. The definitions of OR and AD, the data and the construction of the frontiers stay as they are. The revision adds the error analysis, posterior OR as the main OR estimate, the resolution study, the corrected fixed-budget baseline, six further methods and the comparison with RCP. The findings of the paper are unchanged; what the revision adds is an explicit account of how precisely they are measured.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.