acceptodds
Under review as a conference paper at ICLR 2027

The Model Knows When to Stop: Training-Free Early Stopping for Long-Context Reading

Abstract

Language models often process long inputs sequentially in chunks, but continuing to read after sufficient evidence has been acquired wastes computation. Existing stopping mechanisms either learn sufficiency from internal activations or train an exit gate, while a simpler alternative asks the model whether it has read enough. We introduce Answer-Convergence Stopping (ACS), a training-free stopping rule that measures rather than asks. After each chunk, it probes the frozen model's current answer state and stops when that state is both confident and stable. The rule requires only output-side generation and token log probabilities, has no trained components, and uses one shared configuration across models and benchmarks. Because a stopping policy can save computation simply by stopping too early, we evaluate the stopping decision itself using evidence position where available. On the full LongBench-v2 with two frontier models, ACS is the only stopping policy that matches or exceeds full-reading accuracy. Furthermore, across 250 S-NIAH questions, the premature stopping rate for ACS across five models from two families ranges from 0% to 12%, compared to 8.4% to 45.6% for the verbalized gate. Taken together, ACS reveals that by properly utilizing the output signals of frozen models, we can achieve favorable behaviors like adaptive stopping without the need for additional training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language models, efficiency, context processing, early stopping, long-context, adaptive computation, training-free, calibration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HepQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper looks at when a model can stop reading a long document without hurting its answer. The idea is to stop when the model is confident and its answer has stayed stable as it reads more context. The method does not require training, and the experiments test how much computation it can save across different models and tasks.

Critical strengths

1.I like the basic idea. Being confident does not always mean that the model has read enough, so checking whether the answer stays stable makes sense to me. The method is also simple and does not require training another model, which makes it easy to try in an existing reading pipeline. 2.Some of the savings are substantial. For example, with Qwen on LongBench v2, the method saves about half the tokens while keeping similar accuracy. The authors also count the extra answer checks in the cost, which helps me judge whether the savings are useful in practice.

Critical weaknesses

1.The comparison with existing methods is the main thing missing for me. The paper discusses DCC and GRU-Mem, but does not compare with them experimentally. Most of the comparisons are between stopping rules within the same reading pipeline. I still want to know how ACS compares with these existing approaches, and what performance trade-off comes with avoiding training. 2. I am also not sure how much of the benefit comes from checking stability, and how much comes from simply reading longer before stopping. A useful comparison would be to let the simpler baselines spend the same number of tokens. ACS does not always do better than END either. With Kimi on LongBench v2, default ACS gets 51.9% accuracy and saves 3% of tokens, while END gets 52.5% accuracy and saves 46%. This leaves me unsure when I should choose ACS over the simpler rule.

Questions for authors

1.How does ACS compare with DCC or GRU-Mem in a comparable setting? I would like to see training and inference costs reported separately, so that the benefit of avoiding training is clear. 2.If the methods use the same token budget, does ACS still do better than confidence-only stopping or an END gate that requires several consecutive END decisions? This would help me understand what the stability check adds. 3.Work on confidence-based self-training has shown that confident mistakes can get reinforced. For ACS, I wonder whether the model can also stay confident and consistent about a wrong answer, then stop before reaching the information that would correct it. How often does ACS stop with a wrong answer that full reading would get right? Could you break this down by task, especially in cases where misleading information appears early and decisive evidence comes later, and show whether the stability check reduces these failures compared with confidence alone?

4 comments

We thank the reviewer for the constructive comments. We answer each point below.

Comparison with DCC and GRU-Mem (W1, Q1). Of the two methods the reviewer names, GRU-Mem has no public code or weights, so we ran DCC, with its official code, default settings and protocol, on the same Qwen3-14B and the same 500 RULER-HotpotQA documents as the paper (Table 10). Under this protocol, ACS matches DCC in overall accuracy without training, DCC uses fewer tokens per document, and ACS is significantly more accurate on the longer documents. The table gives accuracy by document length (100 documents each), then the training and inference costs separately.

ACSDCC (official protocol)
Accuracy at 8K0.630.83
Accuracy at 16K0.650.82
Accuracy at 32K0.650.57
Accuracy at 64K0.610.53
Accuracy at 128K0.500.27
Accuracy, all lengths0.6080.607
Trainingnone1.1 GPU-hours on one A100
Labelled documents for trainingnone360 (8K to 32K)
Inference, tokens per document37.6K20.6K
Access to Qwen3-14Boutput probabilitiesattention activations
Longest inputno limitthe context window (6 of the 100 documents at 128K did not fit)

At 8K and 16K neither method stops early, and the difference is between reading in one context (DCC) and reading in chunks (ACS). From 32K on both stop early, and ACS is the more accurate one at each length: over the 294 documents of 32K and above that both methods can read, it is 12.2 points above DCC (95% CI 6.1 to 18.4).

This gap could come from the training of the probe, because the official protocol of DCC covers training on documents of up to about 40K tokens, and our longest documents are beyond that. To make the comparison fairer, we therefore also trained the probe on all five lengths (600 labelled documents, 4.6 GPU-hours), keeping everything else as in the official code and following the DCC paper as closely as we could. The probe then stops early on fewer than 10 of the 494 documents it can read, so DCC saves no tokens and becomes full-context reading (0.781 at 48.3K tokens per document).

Taken together, the learned stopper depends on the lengths it was trained on: trained on the shorter documents it stops too early on the longer ones, and trained on all lengths it almost never stops. ACS needs no training and uses only the output probabilities of the model, so it can be applied at once to any model or API that returns them, at any document length. DCC has its own advantage: it reads the document in one context, which is more accurate on the short documents, and it uses fewer tokens per document. We will add this comparison to Section 4.

Same token budget (W2, Q2). ACS, the END gate and confidence-only stopping are all adaptive rules: each decides, document by document, how much to read. A comparison between them at the same tokens therefore compares one adaptive rule with another. To separate the idea from the amount of reading, the amount has to be given with no decision. So we read every document of LongBench-v2 (503 questions) up to a fixed share of its chunks (rounded up), with no stopping rule. The last fixed row is an oracle: its share is the average share that ACS reads.

ReadingQwen3.5: accuracyQwen3.5: tokensKimi K2.5: accuracyKimi K2.5: tokens
Fixed, 25% of every document0.43793.1K0.47383.3K
Fixed, 50% of every document0.483191.4K0.467171.0K
Fixed, 75% of every document0.533290.5K0.501259.3K
Full reading0.535383.0K0.517340.8K
Fixed, the average share of ACS (oracle, 51% and 52%)0.505197.0K0.483178.6K
ACS0.541193.8K0.535175.6K

Against fixed reading, ACS leads at the same tokens. The oracle row is told how much ACS reads in total (51% of the tokens with Qwen3.5 and 52% with Kimi K2.5), which a fixed rule could not know in advance, and reads that share of every document. It is still 3.6 points less accurate with Qwen3.5 and 5.2 points with Kimi K2.5. The right amount differs from one document to the next: ACS stops earlier than the oracle share on a third of the documents and reads further on 60% of them. Taken together, against the fixed reading controls, the adaptive compute of ACS gives both better accuracy and larger savings: it is more accurate than every control, including those that read up to twice its tokens. Against the other adaptive rules, trained and training-free, the comparison is in Q1 and in the paper: ACS matches the trained one without training, and among the training-free ones it stops before the evidence least often. We will add the fixed controls to Table 1.

ACS and the END gate with Kimi K2.5 (W2). The reviewer's example is correct, and it uses the default θ. This is one of three ways to set θ: tuned on task questions when they are available, set on synthetic needles when they are not, and a default that needs no data and gives up saving but not accuracy. The table gives the three settings and the END gate on LongBench-v2, with the tokens saved and the accuracy relative to full reading.

Stopping ruleNeedsKimi K2.5: savingKimi K2.5: accuracy against full readingQwen3.5: savingQwen3.5: accuracy against full reading
ACS, θ tuned on task questionslabelled questions47%+1.1 points53%0.0 points
ACS, θ set on synthetic needlesno labels49%+1.8 points54%-0.6 points
ACS, default θ = 0.995nothing3%+0.2 points50%+0.6 points
END gatenothing46%+0.8 points65%-2.8 points

With Kimi K2.5, whose confidence runs lower, the default stops less and moves toward full reading, so it gives up saving and not accuracy. Setting θ on synthetic needles needs no task labels (θ is the smallest value with no stop before the evidence, 0.92 for Kimi K2.5, Table 4), and ACS is then above the END gate in both accuracy and saving. With Qwen3.5 it is the END gate that pays, and it pays in accuracy: it saves the most and loses 2.8 points, while ACS is never more than 0.6 points below full reading. The revision will present the three settings in this way and add the two tuned ones to Table 1.

Wrong early stops (Q3). Yes, this can happen: ACS can stop on a confident and stable answer that is wrong while full reading gets it right. On LongBench-v2 it does so on 20 of the 503 questions with Qwen3.5 (4.0%) and on 12 with Kimi K2.5 (2.4%, θ set on needles). The table gives the number of such stops by task, for ACS and for confidence alone, which stops at the first confident answer.

LongBench-v2 taskQuestionsQwen3.5: ACSQwen3.5: confidence aloneKimi K2.5: ACSKimi K2.5: confidence alone
Code repository understanding505534
Long in-context learning815658
Long structured data understanding333400
Long-dialogue history understanding391302
Multi-document QA1251324
Single-document QA1755725
All50320 (4.0%)28 (5.6%)12 (2.4%)23 (4.6%)

These failures are few in every task, at most 5 questions, and they are not confined to one type of data. The stability check reduces them, from 28 to 20 with Qwen3.5 and from 23 to 12 with Kimi K2.5, and they are lower or equal in every task with both models. The case where misleading information comes early and the decisive evidence later is tested directly in the next block. We will add this table to the appendix.

Misleading information early, decisive evidence late (Q3). We built this case on the 250 S-NIAH questions of the paper: for each question, three needles of the same form for other keys are placed before the target needle, as in the multi-key task of RULER. With Qwen3-14B, ACS stops before the decisive evidence on 1 of the 250 questions (0.4%). In that one case it returns the number of a misleading needle, which is the failure the reviewer describes.

Stopping ruleTokensStops before the evidenceAccuracy
ACS39.1K1 (0.4%)0.984
Confidence alone (first confident answer)29.1K7 (2.8%)0.972
END gate23.8K71 (28.4%)0.716
END gate, 4 ENDs in a row38.0K21 (8.4%)0.904
END gate, 5 ENDs in a row40.9K16 (6.4%)0.924
Verbalized gate20.8K101 (40.4%)0.596
Full reading54.1K00.988

The stability check lowers the stops before the evidence from 7 to 1. The END gate is misled far more: it stops before the evidence on 28.4% of the questions, against 5.2% without the misleading needles (Table 2), and four or five ENDs in a row, which cost the tokens of ACS, still leave 8.4% and 6.4%. So when misleading information comes first, ACS keeps reading until the decisive evidence on all but one question, and the END gate often stops too early. We will add this experiment to Section 4.3.

When to choose ACS over the END gate (W2). Taken together, the results above answer this question. Both rules need no training, and they differ in what they read: the END gate asks the model whether it has read enough, and ACS measures the answer itself. The accuracy of ACS stays close to that of full reading with both models, where the END gate loses 2.8 points with one of them, and under misleading information ACS stops before the evidence on 0.4% of the questions, against 28.4% for the END gate (Q3). Its saving depends on θ, and the three settings above say how to set it with the data at hand. In general, ACS is therefore the more stable choice, also when the documents contain misleading information. The END gate is a useful alternative when the model returns no output probabilities, which ACS needs.

I thank the authors for the extensive new experiments.These analyses should have been part of the original submission.

For this reason I raise my score only modestly, from 2 to 3.Good luck!

Reviewer_sLEZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Answer-Convergence Stopping (ACS), a training-free stopping rule for sequential long-context reading. After each chunk, the model updates its notes and produces a provisional answer; reading stops when the answer is both confident and stable. Experiments measure accuracy, inference cost, and stopping behavior relative to known evidence locations.

Critical strengths
  1. ACS is simple, requires no training or internal activations, and can be added to a frozen reader with token-log-probability access.
  2. The evidence-position analysis distinguishes useful early stopping from premature guessing.
  3. The experiments account for probing overhead and show that stability improves over confidence-only stopping.
Critical weaknesses
  1. Comparison with existing methods is limited. The baselines are stopping rules applied to the same custom reader, rather than established systems such as GRU-Mem[1] or DCC[2]. Their training requirements explain the narrower comparison, but leave ACS’s accuracy–efficiency trade-off relative to closely related methods unclear.
  2. The efficiency claims are specific to the chunk-and-note pipeline. The full reading baseline repeatedly rewrites notes rather than directly processing the document. A direct-context baseline on inputs that fit the model’s window would clarify whether ACS offers a practical advantage beyond accelerating this particular pipeline.
  3. The strongest savings are not consistent under the shared configuration, and the timing protocol needs clarification. On LongBench-v2, the default saves 50% of tokens on Qwen3.5 but only 3% on Kimi; Kimi’s 49% saving uses a threshold selected on the evaluation set. It is also unclear whether reported times come from separate online runs or aggregation of recorded call durations.

[1] When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning

[2] Knowing When to Stop: Efficient Context Processing via Latent Sufficiency Signals

Questions for authors
  1. Can the authors compare ACS with an established learned stopping method, reporting training and inference costs separately?
  2. How does ACS compare with direct-context inference on documents within the model’s context window?
  3. How were elapsed times measured, and what fraction of examples become slower after including intermediate probes?
4 comments

We thank the reviewer for the constructive comments. We answer each point below.

Comparison with a learned stopping method (W1, Q1). Of the two methods the reviewer names, GRU-Mem has no public code or weights, so we ran DCC, with its official code, default settings and protocol, on the same Qwen3-14B and the same 500 RULER-HotpotQA documents as the paper (Table 10). Under this protocol, ACS matches DCC in overall accuracy without training, DCC uses fewer tokens per document, and ACS is significantly more accurate on the longer documents. The table gives accuracy by document length (100 documents each), then the training and inference costs separately.

ACSDCC (official protocol)
Accuracy at 8K0.630.83
Accuracy at 16K0.650.82
Accuracy at 32K0.650.57
Accuracy at 64K0.610.53
Accuracy at 128K0.500.27
Accuracy, all lengths0.6080.607
Trainingnone1.1 GPU-hours on one A100
Labelled documents for trainingnone360 (8K to 32K)
Inference, tokens per document37.6K20.6K
Access to Qwen3-14Boutput probabilitiesattention activations
Longest inputno limitthe context window (6 of the 100 documents at 128K did not fit)

At 8K and 16K neither method stops early, and the difference is between reading in one context (DCC) and reading in chunks (ACS). From 32K on both stop early, and ACS is the more accurate one at each length: over the 294 documents of 32K and above that both methods can read, it is 12.2 points above DCC (95% CI 6.1 to 18.4).

This gap could come from the training of the probe, because the official protocol of DCC covers training on documents of up to about 40K tokens, and our longest documents are beyond that. To make the comparison fairer, we therefore also trained the probe on all five lengths (600 labelled documents, 4.6 GPU-hours), keeping everything else as in the official code and following the DCC paper as closely as we could. The probe then stops early on fewer than 10 of the 494 documents it can read, so DCC saves no tokens and becomes full-context reading (0.781 at 48.3K tokens per document).

Taken together, the learned stopper depends on the lengths it was trained on: trained on the shorter documents it stops too early on the longer ones, and trained on all lengths it almost never stops. ACS needs no training and uses only the output probabilities of the model, so it can be applied at once to any model or API that returns them, at any document length. DCC has its own advantage: it reads the document in one context, which is more accurate on the short documents, and it uses fewer tokens per document. We will add this comparison to Section 4.

Direct-context inference (W2, Q2). We ran it on LongBench-v2 with both frontier models: the whole document, the question and the options in one call. With a context window of 256K tokens, one call can read the whole document for 399 of the 503 questions. The two tables give, on these 399 questions and by document size, the accuracy of one call and of ACS, the tokens ACS saves against full reading, and the largest single call, which sets the memory a call needs.

Qwen3.5:

Document sizeQuestionsAccuracy: one callAccuracy: ACSTokens saved by ACS against full readingLargest call: one callLargest call: ACS
Under 32K1110.6130.523-3.9%21K13K
32K to 64K710.6200.5778.2%46K15K
64K to 128K1160.5170.50025.7%96K15K
128K to 256K1010.6140.55435.1%179K16K

Kimi K2.5:

Document sizeQuestionsAccuracy: one callAccuracy: ACSTokens saved by ACS against full readingLargest call: one callLargest call: ACS
Under 32K1180.5510.517-3.2%19K11K
32K to 64K720.4720.50012.1%42K13K
64K to 128K1280.3830.49224.8%93K13K
128K to 256K810.5430.61731.1%174K14K

The saving of ACS comes from the longer documents: against full reading it grows from none under 32K tokens to 35% with Qwen3.5 and 31% with Kimi K2.5 at 128K to 256K. The memory gap grows the same way. The largest call of ACS stays near 15K tokens at every size, while one call holds the whole document, so one call needs about 1.6 times the memory of ACS on the shortest documents and about 11 times on the longest that fit (for scale, 2.6 GB against 29 GB of KV cache with Qwen3-14B). With Qwen3.5, one call is more accurate on the documents that fit, by about 5 points. This difference comes from reading in chunks and not from stopping early, since full reading in chunks has the same accuracy as ACS (0.529). With Kimi K2.5 the order is reversed from 32K on: one call falls to 0.383 on documents of 64K to 128K, against 0.492 for ACS, and over the 399 questions ACS is 4.5 points above one call (0.526 against 0.481).

Taken together, accuracy does not decide between the two: one call is ahead with Qwen3.5, and ACS is ahead with Kimi K2.5. So without training or labelled data, the choice between one call and reading in chunks with ACS depends on the document and on the memory available. For a short document and a model that reads long contexts well, one call is the better choice. For long documents, or when the memory of a call is limited (e.g., small GPUs), ACS is the better choice: its saving in tokens and in time (Q3) and its memory advantage all grow with the length of the document. We will report the one-call baseline in the paper.

Saving under the shared setting (W3). The reviewer is right that the saving under the shared θ differs between the two models (50% and 3% on LongBench-v2), and that the 49% of Kimi K2.5 uses a θ selected on the evaluation questions, which Table 1 marks as post hoc. The shared θ is one of three ways to set θ: tuned on task questions when they are available, set on synthetic needles when they are not, and a default that needs no data and gives up saving but not accuracy. The table gives the three settings on LongBench-v2, with the tokens saved and the accuracy relative to full reading.

Setting of θNeedsKimi K2.5: savingKimi K2.5: accuracy against full readingQwen3.5: savingQwen3.5: accuracy against full reading
Tuned on task questionslabelled questions47%+1.1 points53%0.0 points
Set on synthetic needlesno labels49%+1.8 points54%-0.6 points
Default, θ = 0.995nothing3%+0.2 points50%+0.6 points
  • Tuned on task questions, when labelled questions of the task are available. θ is chosen on one half of the questions and tested on the other half (mean over 2,000 random halves). It keeps the accuracy of full reading or more with both models, at about half of the tokens.
  • Set on synthetic needles, when there are no task labels. Needles can be generated for any model, and θ is the smallest value with no stop before the evidence (0.92 for Kimi K2.5 and 0.99 for Qwen3.5, Table 4). It gives nearly the same saving as task questions and stays within 0.6 points of full reading.
  • Default, when there is no data at all. It is the conservative setting: for a model whose confidence runs lower, ACS stops less and moves toward full reading, so it gives up saving and not accuracy. On S-NIAH it stops before the evidence on 0% of the questions with both models (Table 2).

The paper reports the default and the sweep over θ (Tables 1 and 4) without setting them out as options. The revision will present the three settings in this way, with when to use each, and add the two tuned ones to Table 1.

Timing protocol and slower examples (W3, Q3). The times in Table 1 are computed from the durations recorded in the full-reading run, and not from separate online runs: for each example, the recorded time is scaled by the share of chunks a policy reads. The revision will state this in the caption of Table 1.

For the second question we measured the time online. ACS and full reading were run one after the other on the same GPU (A100 80GB), on the 250 S-NIAH questions with Qwen3-14B, each issuing only its own calls. The table gives the mean time per document by length.

Document lengthChunksFull readingACSExamples where ACS is slower
8K22.7 s3.1 s50 of 50
16K34.7 s5.1 s50 of 50
32K610.7 s10.1 s26 of 50
64K1122.6 s18.2 s18 of 50
128K2147.8 s28.8 s9 of 50
All17.7 s13.1 s153 of 250

The time ACS saves grows with the length of the document: 5% at 32K, 20% at 64K and 40% at 128K, and 26% over all 250 documents. ACS is slower than full reading on 153 of the 250 examples, and 100 of them are the 8K and 16K documents. These have two and three chunks, and the rule needs three answer states, so it cannot stop before the end there and its probes add 0.4 s. On the longer documents it is slower on 53 of 150 examples.

ACS is therefore useful on long documents, where it saves time as well as tokens while the memory of a call stays small (W2), and it has nothing to save on documents of two or three chunks. We will add this measured table to the appendix.

Reviewer_sLEZOpenReview ↗

Thank you for your reply.

The authors have provided the additional experimental results I requested, so I will raise my rating accordingly.

Reviewer_TnrDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The motivation of this paper is: When reading a long document, the key evidence often appears early, so reading the remaining text wastes compute. The authors want to show that a model knows when it has enough information to stop reading. In their setup, the document is split into chunks. After each chunk, the model updates a 6,000-character running note and answers a probe query using only that note. Reading stops when the probe answer stays unchanged across 3 chunks with confidence.

Critical strengths
  1. The motivation is practical and intuitive for sequential reading tasks where relevant evidence appears early in a long document.
  2. Combining answer confidence with a short stability window is simple to implement and requires no model training.
  3. The paper evaluates six models across four benchmarks and provides clear ablations on window size and chunk ordering.
Critical weaknesses
  1. Answer stability over 3 chunks does not mean the model found the true evidence. In Section 3.1, the probe never sees the source text and only sees the query and the running note N_t. Suppose the model writes an early guess into N_1, and chunks 2 and 3 contain unrelated background. Then the note does not change (N_1 = N_2 = N_3), so the probe naturally outputs the same answer across 3 chunks. Thus, the stability rule only measures whether recent chunks changed the note, not whether the true evidence was found. This explains why Gemma-3-27B stops before reaching the needle on 12.0% of S-NIAH questions (Table 5) and 16.5% on held-out S-NIAH (Table 8).
  2. First-token probability >= 0.995 reflects model calibration habits rather than evidence sufficiency. Token probability depends heavily on post-training style rather than whether the model has enough facts. As noted in the paper, Kimi K2.5 has a median full-reading confidence of 0.873 versus 0.992 on Qwen3.5. The method has limitations on several models.
  3. The baseline setup forces chunking and note-taking instead of comparing against single-pass full-context reading. To show real compute savings, the paper should compare chunked early stopping against feeding the full document in one call.
Questions for authors

See weakness.

8 comments

We thank the reviewer for the constructive comments. We answer each point below.

Stability and evidence (W1). The reviewer is right that an unchanged answer does not show that the evidence was found, and this is why ACS does not stop on stability alone: the answer must also be given with high confidence on each of the three chunks. We measured the two conditions on S-NIAH, where the position of the evidence is known. The table gives the share of the 250 questions on which each condition is met before the evidence is read.

ModelAnswer unchanged for three chunksAnswer given with high confidenceBoth (ACS stops)
Qwen3.531.6%0%0%
Kimi K2.54.8%0%0%
Qwen3-14B25.2%2.0%1.6%

An early guess that stays unchanged for three chunks does occur, as the reviewer describes, but the reader rarely gives it with high confidence, so ACS does not stop on it. Gemma-3-27B, the model the reviewer cites, shows why the stability condition is needed as well. With this model early guesses are often given with high confidence, and the stability condition is the one that reduces them, as shown in Table 5: a stop on a confident answer alone comes before the evidence on 23.2% of the questions, and with the stability condition added on 12.0%. The two conditions therefore cover each other: confidence filters the unchanged guesses, and stability filters the confident ones. We will add the stability-alone result to Table 3.

Confidence threshold and calibration (W2). The confidence scale does differ between models, and ACS therefore offers three ways to set θ: tuned on task questions when they are available, set on synthetic needles when they are not, and a default that needs no data and gives up saving but not accuracy. The table gives the three settings on LongBench-v2, with the tokens saved and the accuracy relative to full reading.

Setting of θNeedsKimi K2.5: savingKimi K2.5: accuracy against full readingQwen3.5: savingQwen3.5: accuracy against full reading
Tuned on task questionslabelled questions47%+1.1 points53%0.0 points
Set on synthetic needlesno labels49%+1.8 points54%-0.6 points
Default, θ = 0.995nothing3%+0.2 points50%+0.6 points
  • Tuned on task questions, when labelled questions of the task are available. θ is chosen on one half of the questions and tested on the other half (mean over 2,000 random halves). It keeps the accuracy of full reading or more with both models, at about half of the tokens.
  • Set on synthetic needles, when there are no task labels. Needles can be generated for any model, and θ is the smallest value with no stop before the evidence (0.92 for Kimi K2.5 and 0.99 for Qwen3.5, Table 4). It gives nearly the same saving as task questions and stays within 0.6 points of full reading.
  • Default, when there is no data at all. It is the conservative setting: for a model whose confidence runs lower, ACS stops less and moves toward full reading, so it gives up saving and not accuracy. On S-NIAH it stops before the evidence on 0% of the questions with both models (Table 2).

The paper reports the default and the sweep over θ (Tables 1 and 4) without setting them out as options. The revision will present the three settings in this way, with when to use each, and add the two tuned ones to Table 1.

Single-pass full-context reading (W3). We ran it on LongBench-v2 with both frontier models: the whole document, the question and the options in one call. With a context window of 256K tokens, one call can read the whole document for 399 of the 503 questions. The two tables give, on these 399 questions and by document size, the accuracy of one call and of ACS, the tokens ACS saves against full reading, and the largest single call, which sets the memory a call needs.

Qwen3.5:

Document size (tokens)QuestionsAccuracy: one callAccuracy: ACSTokens saved by ACS against full readingLargest call: one callLargest call: ACS
Under 32K1110.6130.523-3.9%21K13K
32K to 64K710.6200.5778.2%46K15K
64K to 128K1160.5170.50025.7%96K15K
128K to 256K1010.6140.55435.1%179K16K

Kimi K2.5:

Document size (tokens)QuestionsAccuracy: one callAccuracy: ACSTokens saved by ACS against full readingLargest call: one callLargest call: ACS
Under 32K1180.5510.517-3.2%19K11K
32K to 64K720.4720.50012.1%42K13K
64K to 128K1280.3830.49224.8%93K13K
128K to 256K810.5430.61731.1%174K14K

The saving of ACS comes from the longer documents: against full reading it grows from none under 32K tokens to 35% with Qwen3.5 and 31% with Kimi K2.5 at 128K to 256K. The memory gap grows the same way. The largest call of ACS stays near 15K tokens at every size, while one call holds the whole document, so one call needs about 1.6 times the memory of ACS on the shortest documents and about 11 times on the longest that fit (for scale, 2.6 GB against 29 GB of KV cache with Qwen3-14B). With Qwen3.5, one call is more accurate on the documents that fit, by about 5 points. This difference comes from reading in chunks and not from stopping early, since full reading in chunks has the same accuracy as ACS (0.529). With Kimi K2.5 the order is reversed from 32K on: one call falls to 0.383 on documents of 64K to 128K, against 0.492 for ACS, and over the 399 questions ACS is 4.5 points above one call (0.526 against 0.481).

Taken together, accuracy does not decide between the two: one call is ahead with Qwen3.5, and ACS is ahead with Kimi K2.5. So without training or labelled data, the choice between one call and reading in chunks with ACS depends on the document and on the memory available. For a short document and a model that reads long contexts well, one call is the better choice. For long documents, or when the memory of a call is limited (e.g., small GPUs), ACS is the better choice: its saving and its memory advantage both grow with the length of the document. We will report the one-call baseline in the paper.

Thanks author for your reply.

W1: Your new results address my W1 concern.

W2: Since in multiple places (abstract and conclusion), one of the novelties is the method uses "one shared configuration across models and benchmarks". Based on your new results (needs to be tuned), it seems they go against your previous conclusion & renovation.

W3: I calculated the results you showed in W3. (by AUC and token saved weighted by num_of_questions)

The AUC and token saved for two models are: Qwen3.5: AUC 0.5338, token saved 16.73% Kimi K2.5: AUC 0.5262, token saved 15.51%

The results doesn't match Table 1.

Based on that, I won't raise my rating.

Comments on the paper

1 comment

We thank the AC and the reviewers for their time and constructive comments. Every point raised in the reviews is addressed in the response below, and the revised PDF reflects these responses, with changes highlighted in blue.

Stability and premature stopping (TnrD, HepQ). The reviews asked whether a stable answer shows that the evidence has been found. When misleading information is placed before the target evidence, ACS almost never stops too early, while the END gate often does. On LongBench-v2, the stability check reduces wrong early stops compared with confidence alone, with both models. (Section 4.3, Appendix D: Table 9, Appendix G: Table 18)

Threshold selection (TnrD, sLEZ, HepQ). The reviews raised the calibration of the confidence and the different savings across models. We set out three settings of the threshold. The shared tuning-free default carries the main results, and the two calibrated settings, one label-free on synthetic needles and one tuned on task questions, are optional and raise the saving at comparable accuracy. (Section 5, Appendix E: Table 11)

Baselines (sLEZ, HepQ). The reviews requested a comparison with learned stopping methods. We ran DCC with its official code and protocol. ACS matches its overall accuracy without any training, and it is more accurate on the longer documents. (Section 4.4: Table 4)

Token-budget comparison (HepQ). The review asked whether the benefit comes from the stability check or from reading longer. At the same number of tokens, ACS is more accurate than reading a fixed share of every document, with both models, so its benefit comes from deciding where to stop and not from reading longer. (Section 4.2, Appendix G: Table 19)

Full-context comparison (TnrD, sLEZ). The reviews requested a comparison with reading the whole document in one call. On the documents that fit the context window, we compare the two by document length, in accuracy, tokens and the size of the largest call (Appendix G, Tables 20 and 21).

Online efficiency (sLEZ). The review asked how the times were measured and how often ACS is slower. We measured the end-to-end time online. The time ACS saves grows with the length of the document, and the slower cases are short documents of two or three chunks. (Appendix G: Table 22)

We hope these revisions address the reviewers' concerns, and we thank the AC and all reviewers again for their time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.