acceptodds
Under review as a conference paper at ICLR 2027

Periscope: Extending Frozen Language Models Beyond Their Context Window

Abstract

A language model reads long text in one forward pass. The pass is quadratic in length, stops at the context window, and loses accuracy with length before reaching it. We ask whether the read can be factorized when it is a decision over a finite set: which document is relevant, which option is supported, which passage is the evidence. Periscope, a training-free inference method, arranges the chunks of a text on a grid with and asks a frozen model the same question about local spans of consecutive chunks and strided spans that sample the whole text, reading the log-odds of every answer at one token. Each answer takes its best local and its best strided score, and scoring every chunk by its two spans gives an evidence map at no further cost, whose peak is the chunk behind the answer. Every probe is about tokens for a text of tokens and chunk size , so a window of tokens reaches tokens at cost. The map replaces the long read. On LongBench v2, reading only the chunks the map ranks highest, 9k tokens, matches the same model's best window read across windows from 32k to 1M tokens, and on InfiniteBench, where the median context is 150k tokens, it leads the best window read by 5 points. The same map ranks BRIGHT's long-document corpora with the best NDCG@10 of six methods. Each call caches only one probe, so a 27B model reads 4.5M-token contexts on one 80GB GPU, where a single pass would need 296GB of cache. A long read then needs a GPU that holds the model, not one that holds the text.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, long context, inference scaling, training-free, retrieval, evidence localization

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_fpZrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Periscope, a training-free method for long-context question answering and retrieval with a frozen LLM. The NN chunks of a document are arranged on a K×KK \times K grid (K=⌈N⌉K = \lceil \sqrt{N} \rceil), and the model is asked the same question over KK local probes (consecutive chunks) and KK strided probes (one chunk from each row). Each answer is scored by the sum of its best local and best strided log-odds, and combining the two probe scores of each chunk gives an evidence map at no extra cost. The map is used either to answer directly or to select the top-KK chunks for one final read. Experiments use Qwen3.5-4B/27B and Gemma-4-26B-A4B on LongBench v2, InfiniteBench and BRIGHT.

Critical strengths
  1. Simple and clearly formulated. The probing scheme, composition rule (Eq. 3) and evidence map (Eq. 4) are easy to follow and require no training.

  2. Benefits at extreme lengths. Every probe is about sc\sqrt{sc} tokens, so peak KV cache stays small: the 27B reads 4.5M-token contexts on one 80GB GPU with 3.1 GB of cache per probe versus 296 GB for a single read (Sec. 4.6). It also helps when the evidence falls outside the window, e.g. +4.8 points over the best window read on InfiniteBench with the 4B (82.1 vs. 77.3, Table 2).

Critical weaknesses

1. Restrictive problem formulation, limited relevance, and weakly justified 2D probing. The paper largely reduces long-context QA to locating a small set of supporting chunks: each answer is scored by its single best local and single best strided span (Eq. 3), on the stated premise that one strong span is enough. This does not capture tasks whose evidence is distributed across the text, and the results reflect it: on LongBench v2 multi-document QA, the selected read trails the window read for both Qwen models, 33.6 vs. 40.0 (4B) and 44.8 vs. 53.6 (27B) (Table 6). It is therefore unclear to what extent the method addresses long-context reasoning rather than a restricted evidence-retrieval problem. The practical relevance is also limited for LLM agents, where a long context is typically kept as an external file or store that the model accesses on demand through tool calls or iterative retrieval (e.g., [1, 2, 3]); such access handles both extreme lengths and distributed evidence without a single bounded read. The paper neither discusses nor compares against this setting: its related work covers parallel and chained chunk reading (Chain of Agents, LongAgent) but not interactive, on-demand access. The 2D decomposition is also largely heuristic: the document has no inherent 2D structure, and a high Li(y)L_i(y) together with a high Sj(y)S_j(y) does not establish that their intersection, cell (i,j)(i,j), contains the evidence, since the two spans can be supported by different chunks. The paper offers no analysis of this assumption beyond the one-line premise.

2. Unconvincing accuracy–compute tradeoff and incomplete validation of the design. The probes process every token twice, and the compute crossover is about 119k tokens for the 4B and 278k for the 27B (Sec. 4.6). On LongBench v2 the selected read only matches the 4B's best window read (41.9 vs. 41.9) at 4.6 vs. 1.2 PFLOPs, and for the 27B it trails the 262k window read (52.7 vs. 55.9) at 30.2 vs. 10.8 PFLOPs (Table 1). The only retrieval baseline on LongBench v2 is single-shot embedding top-KK; stronger retrieval-based or adaptive test-time strategies at matched compute (larger read budgets, reranking, iterative retrieval) are missing. The ablations (Table 4: local-only vs. local + strided, Chain of Agents; Table 7: selected-chunk budgets) do not isolate the value of the 2D decomposition: there is no comparison with alternative probe layouts or aggregation rules (e.g., sum or soft aggregation instead of max). The evidence map is shown qualitatively (Fig. 4) and assessed indirectly through downstream accuracy and agreement with the whole read (Table 9), but never against ground-truth evidence locations. Overall, the experimental analysis feels incomplete relative to the paper's claims.

Minor:

  • The related work section is long relative to the technical contribution and could be substantially condensed.
  • The writing leans on stylized phrasing and metaphors that obscure rather than clarify, e.g., the last sentence of the abstract (a GPU that holds the model, not one that holds the text).
  • Fig. 2 is a rudimentary 3×33 \times 3 schematic and could convey the probing and map construction more informatively.

References

[1] K.-H. Lee et al. A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts. ICML 2024.

[2] H. Chen et al. Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading. arXiv 2023.

[3] H. Trivedi et al. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. ACL 2023.

Questions for authors
  1. (Clarification) How often does the peak of the map coincide with the annotated evidence location? Can you report localization accuracy against ground-truth evidence, and how often the maximizing local and strided spans for the predicted answer are supported by different chunks?

  2. (Suggestion) At the same probe budget, how does the K×KK \times K local/strided layout compare with alternatives (e.g., random or overlapping partitions, other grid shapes), and how does max-aggregation compare with sum or soft aggregation?

  3. (Suggestion) Can you compare against retrieval-based, iterative or agentic baselines matched in PFLOPs, e.g., embedding or BM25 retrieval with reranking or a larger read budget, iterative retrieval, or an agent that reads the document on demand (e.g., ReadAgent or MemWalker)? In what settings would Periscope be preferable to such on-demand access?

  4. (Clarification) Is there a natural extension to distributed evidence, such as selecting multiple peaks per answer, and would it close the multi-document QA gap in Table 6?

2 comments

We thank the reviewer for the constructive comments. We answer each point below.

Localization against ground-truth evidence (W1, W2, Q1). We measured it with Qwen3.5-4B on the 750 BABILong questions, which have one, two or three supporting facts at known positions in texts of 32k, 131k and 1M tokens. The last column counts the case the reviewer describes: a peak with no supporting fact whose local span and strided span are each supported by a different chunk.

Supporting facts per questionQuestionsTop-1: the peak is a supporting chunk (%)Top-K: supporting facts among the K chunks read (%)Every supporting fact is read (%)Peak is an unsupported intersection (%)
One25052.491.691.60.0
Two25013.240.016.43.6
Three2508.023.23.22.4

With one supporting fact the map localizes far above chance, where a random read of K chunks holds the fact in 2% to 14% of the cases, depending on the length. A peak whose two spans are supported by different chunks is rare, 3.6% and 2.4% of the questions with several facts.

Alternative layouts (W2, Q2). At the same probe budget the local and strided layout is what gives the map its chunk-level resolution. We compared it with an overlapping layout (local spans shifted by half a span) and with two random balanced partitions, with the same 2K probes and the same input tokens. LongBench v2 has no evidence annotations, so it cannot isolate the contribution of the layout cleanly. So we measured this on the same BABILong questions as above, where the positions of the evidence are known.

Probe layoutTop-1: the peak of the map is a supporting chunk (%)Chunks tied at the peakTop-K: supporting facts among the K chunks read (%)Accuracy of the selected read (%)
Local + strided (Periscope), 2K probes24.41.051.438.4
Two random balanced partitions, 2K probes15.61.948.036.5
Local + shifted local (overlapping), 2K probes6.79.741.436.0
Local only, K probes4.119.330.434.5

A local span scores one region of the text, so local spans alone can place the evidence among the K chunks read (top-K) but cannot say which chunk it is. A strided span takes one chunk from every region, so the two together point at a single chunk (top-1). With overlapping spans about ten chunks still tie at the peak, so the map names a region and not a chunk. Random partitions separate the chunks as well, but none of their spans is contiguous text, and they localize less well on both measures. The selected read needs the evidence among its K chunks and not at the peak, so its accuracy moves with top-K, and less than top-1 does.

Aggregation rule (W2, Q2). The maximum is above the sum and the soft maximum with all three backbones. The table recomputes the direct readout on LongBench v2 from the same probes under each rule.

BackboneMaximum (Periscope)Soft maximumSum
Qwen3.5-4B40.237.634.2
Qwen3.5-27B52.749.142.1
Gemma-4-26B41.039.230.6

A sum adds the scores of the spans that hold no evidence, and the maximum does not.

Retrieval and agentic baselines (W1, W2, Q3). We ran BM25, embedding retrieval, a trained reranker over every chunk and the reading agent ReadAgent, next to Chain of Agents. Each selector reads the same K chunks as Periscope select, and ReadAgent may reread up to K pages. The table gives each method on three axes, on LongBench v2 with Qwen3.5-4B: accuracy, compute (PFLOPs per question, as in Table 1) and memory (the largest call of a question, which the KV cache must hold).

MethodAccuracy (%)Compute (PFLOPs)Memory: largest call (tokens)Needs besides the LLM
Window read (131k tokens)41.91.2387.4knothing
Periscope select41.94.619.4knothing
ReadAgent38.43.2843.3kgenerated notes
Trained reranker + read of K chunks39.62.359.5ka trained reranker
BM25 + read of K chunks36.00.079.5kan index
Embedding retrieval + read of K chunks35.81.919.4ka trained embedder
Chain of Agents32.02.429.2kgenerated summaries

No method in the table is more accurate than Periscope select, and only the window read is level with it, with a call 9 times longer. On Llama-3.1-8B (InfiniteBench En.MC) we also ran two training-free long-context methods with their official code: ParallelComp scores 66.8% and EM-LLM 44.8%, against 76.9% for Periscope select.

Compute and when Periscope is preferable (W2, Q3). Periscope targets the Pareto frontier of memory and accuracy at every text length, and on long texts the frontier of memory, compute, speed and accuracy together. It reaches the accuracy of the best window read with 1.6 GB of KV cache per call against 4.3 GB (Table 1), unlocking long contexts for small GPUs. On short texts it pays compute for this, about twice that of a single read, since the probes process every token twice, as the reviewer notes. Beyond the crossover of Sec. 4.6 it needs less compute than a single read of the same text, and it is also faster, as the table shows. It gives the complete time per question of both, by text length, with Qwen3.5-4B on one A100.

Text length (tokens)16k32k65k131k262k524k996k
Single read (s)0.81.94.713.040.3138.0458.6
Periscope select, probes and final read (s)1.94.47.115.326.759.0110.1

So Periscope is preferable when you have a long text, a large backbone or a small GPU and want results comparable to a full read. On-demand access is a different step: the agent still has to find which passage to open, with a retriever or with generated notes, one call after another. Periscope can be that retriever: its map ranks the chunks of the text, and reading the chunks it picks is 6 points more accurate than reading those picked by BM25 or embedding retrieval.

Distributed evidence (W1, Q4). A deeper read is the natural extension. It recovers part of the gap but does not close it. The selected read already takes the K highest cells of the map, so several peaks are read, and the table gives the accuracy on the one-fact and two-fact questions of BABILong for reads of K, 2K and 4K chunks from the same map.

ReadOne fact (%)Two facts (%)Two facts: both among the chunks read (%)Tokens in the read
Best window read44.032.4
Periscope select, K chunks66.022.416.49.7k
Periscope select, 2K chunks65.229.230.419.4k
Periscope select, 4K chunks56.026.448.438.8k

A read of 2K chunks holds both facts almost twice as often and gains 6.8 points on two-fact questions, with the one-fact accuracy unchanged. A read of 4K chunks holds both facts more often still and loses 10 points on one-fact questions, because the read loses precision as it grows. The remaining gap points to how recognizable a piece of evidence is from its span, since a probe sees it with its neighbours alone and not with the full text, so a fact that does not decide the answer on its own is harder to recognize as evidence. The multi-document questions of Table 6 have this property by design, since LongBench v2 requires their answers to be drawn from several documents, and there a deeper read does not move the accuracy (33.6% from K/2 to 4K chunks). Multi-document QA is also the only domain of that table where the selected read is behind the window read with both backbones, and the agent the reviewer cites is behind there as well: ReadAgent scores 36.8%, against 40.0% for the window read. We recognize this as a current limitation of Periscope and will add it to the limitations section.

Minor points. We will condense the related work, rewrite the stylized sentences in plain terms, and improve Figure 2 to make it more informative.

Reviewer_9mdJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Periscope is a training-free method that uses local and strided probes of a frozen language model for finite-answer tasks beyond its context window. Aggregated scores support direct answering, document ranking, and chunk selection. Experiments on LongBench v2, InfiniteBench En.MC, and BRIGHT demonstrate reduced per-call KV-cache requirements, with accuracy–compute trade-offs varying across models and tasks.

Critical strengths
  1. The method addresses a practical limitation of long-context inference with a simple design. Its independent, bounded probes require neither model fine-tuning nor generated intermediate summaries.

  2. The experiments provide encouraging evidence that answer-space probing can select useful context.

Critical weaknesses
  1. The comparative evidence does not yet sufficiently establish a practical advantage over existing training-free long-context methods, nor does it show whether its strongest comparative gains generalize across backbones.

  2. The contribution beyond prior grid probing and the interpretation of the evidence map are insufficiently validated.

Questions for authors
  1. Baseline coverage: Tables 1–2 primarily compare against truncated window reads and basic chunk-selection methods, while Table 4(b) evaluates Chain of Agents only with Qwen3.5-4B on LongBench v2. Can the authors provide comparisons with representative training-free methods such as ParallelComp [1], REFORM [2], and EM-LLM [3], using a common backbone and clearly specified resource budgets? Why is embedding-based selection included in Table 1 but omitted from Table 2?

  2. End-to-end efficiency and practical advantage: In Table 1, Select matches the 4B model’s best window accuracy at approximately 3.8× the FLOPs, while trailing the 27B model’s best window by 3.2 percentage points at approximately 2.8× the FLOPs. Figure 3 reports probe-batch latency rather than the complete Select pipeline, and the KV figures describe individual calls. Can the authors report complete per-question latency, including the final reading pass?

  3. Statistical reliability and generalization: The InfiniteBench improvement is demonstrated only with Qwen3.5-4B on 229 questions; the 4.8-point gain corresponds to a net increase of 11 correct answers. Can the authors provide paired confidence intervals or significance analyses, and assess whether this gain and the advantage over Chain of Agents persist with Qwen3.5-27B or Gemma? For InfiniteBench, please also separate results according to whether inputs fit the strongest 262k window baseline, rather than only the 131k threshold used in Section 4.4.

  4. Contribution beyond GridProbe and ablation controls: Given that GridProbe already uses a square grid, row/column probes, and map-based selection, what gains arise specifically from Periscope’s answer-specific log-odds, abstention mechanism, and aggregation rule? Adding strided probes improves LongBench v2 Direct accuracy from 39.4% to 40.2%, but also increases computation. Can the authors isolate these contributions and compare strided probing with additional local or alternative grouped probes under matched computation, for both Direct and Select? Please also explain how the 150 BRIGHT queries in Table 4(a) were selected and whether the findings hold on the 350-query set used in Table 3.

  5. Evidence-map validity and scope: The strongest local and strided scores for the same answer may originate from different supporting chunks, so their intersection need not contain evidence. How reliably does the proposed map localize supporting chunks in such cases? The algebraic relationship between answer scores and map maxima, document-ranking performance, answer agreement, and qualitative examples do not directly establish localization accuracy. Can the authors provide quantitative evidence-localization or selected-evidence recall results, including cases requiring distributed evidence? Please clarify the assumptions under which maximum-based aggregation is appropriate, since a finite answer set does not imply that a single passage suffices to answer the question.

[1] Xiong et al. ParallelComp: Parallel Long-Context Compressor for Length Extrapolation. ICML 2025.

[2] Song et al. Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers. NeurIPS 2025.

[3] Fountas et al. Human-inspired Episodic Memory for Infinite Context LLMs. ICLR 2025.

4 comments

We thank the reviewer for the constructive comments. We answer each point below.

Baseline coverage (W1, Q1). We ran ParallelComp and EM-LLM with their official code and configuration on Llama-3.1-8B-Instruct, a backbone both papers report, on InfiniteBench En.MC. REFORM has no public code. The table gives each method on three axes: accuracy, compute (how many times the text is read, and the time per question on one A100) and memory (the largest call, and what is kept between calls).

MethodAccuracy (%)Reads of the textSeconds per questionLargest call (tokens)Kept between calls
Periscope select76.92.1319.7kthe 2K scores
Window read (head and tail of 131k tokens)71.60.829119.5knothing
ParallelComp66.81.02018.0kthe keys and values of 3 windows
EM-LLM44.81.07768.8kthe keys and values of every token

Periscope select is 10.0 points above ParallelComp and 32.5 points above EM-LLM on the same questions, and our runs reproduce the accuracy the authors report on this backbone.

On the second part of the question, embedding selection was missing from Table 2 and is now run there as well: 73.4% with Qwen3.5-4B against 82.1% for Periscope select. We will add these results to the revision.

End-to-end efficiency (Q2). The table reports the complete latency per question, with the final read included, for Qwen3.5-4B on one A100.

Text length (tokens)16k32k65k131k262k524k996k
Single read (s)0.81.94.713.040.3138.0458.6
Periscope select, probes and final read (s)1.94.47.115.326.759.0110.1

The final read adds 0.2 to 1.3 s to the probes, and Periscope select is faster than the single read above about 160k tokens. Compute has a crossover as well, at about 119k tokens (Sec. 4.6): on short texts Periscope pays about twice that of a single read, while on longer texts it needs less, about half the FLOPs of the 1M window read, which covers most texts whole (Table 1). Beyond both crossovers, Periscope reads a long text faster and with less compute than a single read.

Statistical reliability and generalization (Q3). We computed paired 95% confidence intervals, split InfiniteBench at the 262k window, and added Qwen3.5-27B and Llama-3.1-8B. The table gives Periscope select against the window read on InfiniteBench En.MC.

Backbone and booksQuestionsPeriscope select (%)Window read (%)Difference (95% CI)
Qwen3.5-4B, all books22982.177.3+4.8 (0.0 to +9.6)
Qwen3.5-4B, books that fit the 262k window18181.882.9-1.1 (-5.5 to +3.3)
Qwen3.5-4B, books that do not fit it4883.356.2+27.1 (+14.6 to +41.7)
Qwen3.5-27B, all books22992.193.0-0.9 (-3.9 to +2.2)
Llama-3.1-8B, all books (131k window)22976.971.6+5.2 (+0.4 to +10.5)

With Qwen3.5-4B the gain comes from the 48 questions on books that do not fit the window, and on the books that fit the two reads are level, which is what Periscope targets (the same accuracy with less memory). Qwen3.5-27B already answers most questions from its window, and the two reads give the same answer on 217 of the 229 questions. With Llama-3.1-8B Periscope select is 5.2 points above the window read.

As for Chain of Agents, the advantage persists with Qwen3.5-27B: 52.7% against 42.7% on LongBench v2, the same gap of about 10 points as with Qwen3.5-4B.

Contribution beyond GridProbe and its components (W2, Q4). Periscope differs from GridProbe in its scale and in its goals. GridProbe samples a fixed grid of 12×12 frames from a video. Periscope reads the whole text, so its grid is set by the text of each question and grows with it, as high as 45×45 chunks for 1M tokens and 95×95 for 4.5M, where the evidence can be one cell among thousands. The method differs as well. Periscope composes its scores per answer, where GridProbe multiplies one confidence per subset whatever the answer. Periscope also has a direct readout that needs no second call, where GridProbe always runs a second pass on the selected frames. Also, instead of only targeting accuracy-compute efficiency, Periscope targets the Pareto frontier of memory and accuracy at every text length, and on long texts the frontier of memory, compute, speed and accuracy all together. A smaller KV cache answers an everyday problem, a long read on a small GPU, and Periscope can be used for it immediately, since it is training-free and uses the frozen model as it is. Additionally, the three components the reviewer names each have a measurable role as well. Each row of the table changes one of them and keeps the same probes. The first two rows are on LongBench v2 with Qwen3.5-4B, and the last one on BABILong, where the positions of the supporting facts are known.

Changed componentMeasurePeriscopeChanged
No abstain optionselected read (%)41.938.2
Sum over spans in place of the maximumdirect readout (%)40.234.2
One score per span in place of one score per answerpeak of the map is a supporting chunk (%)24.419.6

Each of the three is needed: without the abstain option or the maximum, accuracy falls by 4 and 6 points, and without per-answer scores the map loses a fifth of its correct peaks.

Strided probes at matched computation (Q4). At matched computation the strided spans are what gives the map its chunk-level resolution. We compared them with additional local spans (shifted by half a span) and with two random balanced partitions, at the same 2K probes and input tokens, for both readouts, with Qwen3.5-4B. LongBench v2 has no evidence annotations, so it cannot isolate the contribution of the grouping cleanly. So we measured this on BABILong, where the positions of the supporting facts are known (750 questions, texts of 32k, 131k and 1M tokens).

Probe partitionDirect readout (%)Selected read (%)Top-1: the peak of the map is a supporting chunk (%)Chunks tied at the peakTop-K: supporting facts among the K chunks read (%)
Local + strided (Periscope), 2K probes34.138.424.41.051.4
Two random balanced partitions, 2K probes33.936.515.61.948.0
Local + shifted local, 2K probes34.436.06.79.741.4
Local only, K probes34.334.54.119.330.4

A local span scores one region of the text, so local spans alone can place the evidence among the K chunks read (top-K) but cannot say which chunk it is. A strided span takes one chunk from every region, so the two together point at a single chunk (top-1). With shifted spans about ten chunks still tie at the peak, so the map names a region and not a chunk. Random partitions separate the chunks as well, but none of their spans is contiguous text, and they localize less well on both measures. Each readout uses the map differently. The direct readout takes the best local and the best strided score of each answer and never picks a chunk, so it does not depend on this resolution and is level for the four partitions. The selected read needs the evidence among its K chunks and not at the peak, so its accuracy moves with top-K, and less than top-1 does.

The 150 BRIGHT queries (Q4). The 150 queries of Table 4(a) are a sample stratified across the domains. On the 350 queries of Table 3, local + strided spans score 0.561 NDCG@10 and local spans alone 0.480. So the findings hold.

Localization accuracy (W2, Q5). The strongest local and strided scores can come from different chunks, as the reviewer notes, and we measured how often this misleads the map. We used Qwen3.5-4B on the 750 BABILong questions, which have one, two or three supporting facts at known positions in texts of 32k, 131k and 1M tokens. The last column counts unsupported intersections: a peak with no supporting fact whose local span and strided span each hold one.

Supporting facts per questionQuestionsTop-1: the peak is a supporting chunk (%)Top-K: supporting facts among the K chunks read (%)Every supporting fact is read (%)Peak is an unsupported intersection (%)
One25052.491.691.60.0
Two25013.240.016.43.6
Three2508.023.23.22.4

With one supporting fact the map localizes far above chance where a random read of K chunks holds the fact in 2% to 14% of the cases, depending on the length. Unsupported intersections are rare, 3.6% and 2.4% of the questions with several facts.

Distributed evidence and the maximum (Q5). This table compares the two readouts with the best window read on BABILong, by the number of supporting facts the answer needs.

Supporting facts per questionQuestionsDirect readout (%)Selected read (%)Best window read (%)
One25055.666.044.0
Two25022.022.432.4
Three25024.826.823.2

When a single passage suffices (one fact), both readouts are above the window read. When the answer needs two passages together, both are about 10 points below it, and on three-fact questions every read is close to chance (16.7% for six options). The reason is that a probe sees a fact with its neighbours alone and not with the full text, so a fact that does not decide the answer on its own is harder to recognize as evidence.

The assumption behind the maximum is therefore that some span supports the answer on its own. Replacing it by a sum or a soft maximum over spans gives a lower score with all three backbones (34.2% and 37.6% against 40.2% with Qwen3.5-4B).

Thank you for the time and effort you have devoted to the additional experiments and clarifications. All of my questions and concerns have been carefully addressed. I have therefore decided to raise my score from 2 to 3.

Reviewer_5yHGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Periscope is a training-free method for finite-answer tasks on frozen language models that enables them to process texts beyond their context window. Periscope splits the input document into a K×K grid of chunks and runs 2K short independent probes on K local spans (consecutive chunks) and K strided spans (every K-th chunk) of length sc\sqrt{sc} tokens each. The scores from the probes are then composed into an evidence map of likely answer locations. Two readouts are proposed from the evidence map: Direct, which simply returns the answer corresponding to the max-pooled evidence map, and Select, which makes one additional QA call on the top-K ranked chunks. They validate Periscope on LongBench v2, InfiniteBench and BRIGHT using multiple open-weight models and report accuracy, latency and memory usage against window-read and embedding-based baselines. The main contribution of Periscope is its memory efficiency: each probe reads only sc\sqrt{sc} tokens and thus enables a 27B model to process 4.5M-token contexts on a single 80GB GPU.

Critical strengths
  1. The approach is elegant and clearly explained, especially through Figure 2, which makes it clear and easy to understand.
  2. The substantial reduction in per-call memory requirements allows very long documents to be processed on limited hardware, introducing practical value to the proposed methods. I would say this is the strongest contribution from my point of view.
Critical weaknesses

Weakness 1: Insufficient novelty and unjustified partitioning design The grid-based probing scheme closely follows GridProbe scheme, while the benefit of the local-and-strided partitioning for text is not sufficiently established. The ablation study in Table 4a compares the local-only probing with the local+strided probing. However, the strided probes in the local+strided probing also double the probing budget, making it not clear whether the gain is due to the grouping or to the additional computation. A matched-budget comparison with other partitions would be better to justify the choice of this strided-span adaptation and to clearly distinguish it from prior work.

Weakness 2: Evidence map localization is unverified and the claim is overstated The evidence map is constructed as M[i,j]=max⁡y≠yˉ[Li(y)+Sj(y)]M[i,j]=\max_{y\ne\bar y}[L_i(y)+S_j(y)], where a cell scores highly whenever its row and column independently score highly. For example, if there is evidence in chunks (1,1) and (2,2), then the cells (1,2) and (2,1) will also have high scores although they do not contain any evidence. Therefore, the high score of a cell in the evidence map does not necessarily correspond to a chunk with evidence. The paper does not provide any quantitative evaluation of the evidence map localization, i.e., whether the peak of the evidence map corresponds to a supporting chunk. Figure 4(c) shows a selected-read failure case, but it is not further analyzed whether the peak identifies an incorrect evidence chunk. The QA and document-retrieval results in Tables 1–3 and Table 6 demonstrate the usefulness of the evidence map for downstream tasks, but the results do not establish the chunk-level localization accuracy. In particular, the consistently poor performance on multi-document QA of -6 to -9 points (Table 6) is a natural consequence of the separability of the evidence map score, but it is not further analyzed in the paper. Presenting evidence localization as an explicit contribution without direct empirical validation is an overstatement.

Questions for authors

Question 1: Partitioning

Following Weakness 1, could the authors provide a matched-budget comparison against alternative groupings, such as shifted local partitions (same block structure offset by half a block width), random balanced partitions (chunks are randomly assigned to equally sized probe groups), etc, holding the number of probes and total input tokens approximately fixed? This would help distinguish the benefit of strided grouping from additional computation.

Question 2: Localization Accuracy

This question follows Weakness 2: Is there quantitative evidence for the peak of the map identifying a supporting chunk? It would be great to evaluate top-1 localization accuracy (whether the peak cell corresponds to a ground-truth evidence chunk) and evidence recall among the selected K chunks on a small diagnostic set (for example, LongBench v2 question ) where the evidence chunk can be identified by matching the correct answer against the source document. Specifically, results should include cases where the evidence spans multiple chunks and thus are most likely to mislead the map by forming unsupported intersections.

Question 3: Multi-document QA failures:

Why is there a 6-9% point deficit for multi-document QA in Table 6? In a small analysis of failed examples, were the chosen chunks missing required evidence, or did the reader fail even when sufficient evidence was provided? This will help to clarify the limitations of the map and the selected-read stage.

If the authors can answer the above questions with sufficient supporting analysis and evidence, I will consider reassessing the work.

2 comments

We thank the reviewer for the constructive comments. We answer each point below.

Novelty (W1). Periscope differs from GridProbe in its scale and in its goals. GridProbe samples a fixed grid of 12×12 frames from a video. Periscope reads the whole text, so its grid is set by the text of each question and grows with it, as high as 45×45 chunks for 1M tokens and 95×95 for 4.5M, where the evidence can be one cell among thousands. The method differs as well. Periscope composes its scores per answer, where GridProbe multiplies one confidence per subset whatever the answer. Periscope also has a direct readout that needs no second call, where GridProbe always runs a second pass on the selected frames. Also, instead of only targeting accuracy-compute efficiency, Periscope targets the Pareto frontier of memory and accuracy at every text length, and on long texts the frontier of memory, compute, speed and accuracy all together. It reaches the accuracy of the best window read with a fraction of its KV cache per call (Table 1), and beyond a certain length it needs less compute than a single read of the same text (Sec. 4.6) and is faster from about 160k tokens on, 1.5 times at 262k and about 4 times at 1M, as the following table shows. It gives the complete time per question of both, by text length, with Qwen3.5-4B on one A100.

Text length (tokens)16k32k65k131k262k524k996k
Single read (s)0.81.94.713.040.3138.0458.6
Periscope select, probes and final read (s)1.94.47.115.326.759.0110.1

Taken together, Periscope takes grid probing to a harder setting than GridProbe, with a far larger grid and broader goals, and the results above show that it reaches them.

Localization of the evidence and the partition (W1, W2, Q1, Q2). The strided spans are what gives the map its chunk-level resolution: at matched budget they lead every alternative on every localization measure. LongBench v2 has no evidence annotations (only 10 of its 503 correct options match the source verbatim), so it cannot isolate the contribution of the grouping cleanly. So we measured this instead on BABILong with Qwen3.5-4B, where the positions of the evidence are known: 750 questions with one, two or three supporting facts in texts of 32k, 131k and 1M tokens. The first table compares the partitions the reviewer names, with the same 2K probes and the same input tokens as Periscope: shifted local spans (offset by half a span) and two random balanced partitions.

Probe partitionTop-1: the peak of the map is a supporting chunk (%)Chunks tied at the peakTop-K: supporting facts among the K chunks read (%)Accuracy of the selected read (%)
Local + strided (Periscope), 2K probes24.41.051.438.4
Two random balanced partitions, 2K probes15.61.948.036.5
Local + shifted local, 2K probes6.79.741.436.0
Local only, K probes4.119.330.434.5

A local span scores one region of the text, so local spans alone can place the evidence among the K chunks read (top-K) but cannot say which chunk it is. A strided span takes one chunk from every region, so the two together point at a single chunk (top-1). With shifted spans about ten chunks still tie at the peak, so the map names a region and not a chunk. Random partitions separate the chunks as well, but none of their spans is contiguous text, and they localize less well on both measures. The selected read needs the evidence among its K chunks and not at the peak, so its accuracy moves with top-K, and less than top-1 does.

The second table gives the localization of Periscope by the number of supporting facts. The last column counts unsupported intersections: a peak with no supporting fact whose local span and strided span each hold one.

Supporting facts per questionQuestionsTop-1: the peak is a supporting chunk (%)Top-K: supporting facts among the K chunks read (%)Every supporting fact is read (%)Peak is an unsupported intersection (%)
One25052.491.691.60.0
Two25013.240.016.43.6
Three2508.023.23.22.4

With one supporting fact the map localizes far above chance: a random read of K chunks holds the fact in 2% to 14% of the cases, depending on the length. With evidence spread over two or three chunks the map finds part of it, and the unsupported intersections the reviewer describes are rare, so they are not what limits the map there (Q3).

Multi-document QA failures (Q3). The chosen chunks were missing required evidence. When the evidence is among them, the reader answers as often as the window read. LongBench v2 has no evidence annotations, so we analysed the failures on BABILong, on the 246 two-fact questions whose facts sit in different chunks. The table splits them by whether the chunks read held both facts, and compares Periscope select with the best window read on the same questions.

BABILong questionsQuestionsPeriscope select (%)Best window read (%)
Two facts, the chunks read held both3860.557.9
Two facts, the chunks read missed one20815.428.4
One fact, for comparison25066.044.0

Of the 191 two-fact questions that Periscope select fails, 176 are in the second row: a required fact was not among the chunks read. In the first row, where both facts were read, it is level with the window read. The third row shows the other side: when a single fact decides the answer, the selection finds it, and Periscope select is 22 points above the window read. The deficit is therefore a miss of the selection, not of the reader. It points to how recognizable a piece of evidence is from its span, since a probe sees it with its neighbours alone and not with the full text, so a fact that does not decide the answer on its own is harder to recognize as evidence. The multi-document questions of Table 6 have this property by design, since LongBench v2 requires their answers to be drawn from several documents. Multi-document QA is also the only domain of that table where the selected read is behind the window read with both backbones, and we attribute its deficit to the same miss. We will add these analyses to the paper.

Reviewer_9YfvOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Periscope lets a frozen LLM answer questions over texts far longer than its window by splitting the text into a K×K grid and making 2K independent short probes: one per row (contiguous span) and one per column (strided span covering the whole text). Each chunk’s score combines its row and column probes into an evidence map, which can be read out directly or used to select a short final read. On LongBench v2, InfiniteBench, and BRIGHT, it matches or beats window reads, with a large memory advantage at very long contexts. However, it costs more total compute than a single read at moderate lengths, and many gains are at parity or lack significance testing.

Critical strengths

Training-free and uses the same model, prompt, and readout as a standard read, so it is easy to adopt. Memory per call is bounded by √(sc), which lets a small GPU handle multi-million-token contexts. Ablations and agreement analyses (Table 4, Table 9, Figure 1b) show the map tracks the whole read’s decisions, not just lucky answers.

Critical weaknesses

Total compute is much higher (about 3–4× the single read for the 4B model at equal accuracy), and the paper’s framing underplays this.

Many claimed gains are ties, and key differences rest on small samples (e.g., n=33 in the >1M bin) with no confidence intervals.

The baselines miss the most natural competitor, an LLM pointwise chunk judge at equal cost, and the best-window baseline is chosen post hoc on the test set.

Questions for authors

As in weakness

5 comments

We thank the reviewer for the constructive comments. We answer each point below.

Compute and framing (W1). Periscope targets the Pareto frontier of memory and accuracy at every text length, and on long texts the frontier of memory, compute, speed and accuracy together. It reaches the accuracy of the best window read with 1.6 GB of KV cache per call against 4.3 GB (Table 1). On short texts it pays compute for this, about twice that of a single read, since it reads every token twice. However, beyond a certain length Periscope needs less compute than a single read of the same text (Sec. 4.6), and is even faster than standard inference, as the following table shows. It gives the complete time per question of both, by text length, with Qwen3.5-4B on one A100.

Text length (tokens)16k32k65k131k262k524k996k
Single read (s)0.81.94.713.040.3138.0458.6
Periscope select, probes and final read (s)1.94.47.115.326.759.0110.1

Table 1 shows both sides of this crossover on LongBench v2: Periscope uses 3.8 times the FLOPs of the 131k window read, which stops at 131k tokens, as the reviewer notes, and about half the FLOPs of the 1M window read, which covers most texts whole. The revision will present Periscope in these terms from the abstract on: the frontier of memory and accuracy at every text length, the compute this costs on short texts, and the lead in compute and speed on long ones, with when it is preferable to use.

Ties and confidence intervals (W2). We computed paired 95% confidence intervals. On LongBench v2 and on the InfiniteBench books that fit the window, the difference to the best window read is inside its interval, which is the target stated above: the same accuracy with less memory. On the books that do not fit the window it leads, and the interval is clear of zero on 48 questions. The table gives the three comparisons with Qwen3.5-4B.

BenchmarkQuestionsPeriscope select (%)Best window read (%)Difference (95% CI)
LongBench v250341.941.90.0 (-4.2 to +4.0)
InfiniteBench, books that fit the window18181.882.9-1.1 (-5.5 to +3.3)
InfiniteBench, books that do not fit it4883.356.2+27.1 (+14.6 to +41.7)

We will add confidence intervals to Tables 1 and 2.

Pointwise chunk judge at equal cost (W3). We added this baseline, and at equal cost it is not above the K local spans of Periscope: it is below them on the direct readout (35.8% against 40.0%) and level with them as a selector (38.8% against 39.6%). Equal cost means one read of the text: the judge probes every chunk alone, with the same prompt and the same readout, as the K local spans do. The table gives the rerun results for both readouts on LongBench v2 with Qwen3.5-4B.

ProbesReads of the textDirect readout (%)Selected read (%)
One probe per chunk (pointwise judge)one35.838.8
K local spansone40.039.6
K local + K strided spans (Periscope)two40.241.9

The judge also needs one call per chunk, 539 calls per question on average against 38 for Periscope, and the gap widens with the text: the calls of the judge grow with its length, those of Periscope with its square root (2,000 against 91 for a text of 1M tokens). We will add the judge to Table 4.

Best window chosen on the test set (W3). Table 1 reports every window from 32k to 1M tokens, and we compare against the best of them, which is the comparison least favourable to Periscope. Periscope select is at or above each of them (41.9% against 36.4% to 41.9%), so the conclusion does not depend on which window is fixed in advance.

Thank you for the additional experiments. While they clarify the efficiency trade-offs, I remain unconvinced that the row-column probing strategy offers sufficient methodological novelty beyond existing chunk-based approaches, particularly without a stronger compute-matched comparison isolating its benefits.

I maintain my original rating.

However, I think the general story is interesting, therefore, I'd leave the decision to AC.

We thank the reviewer for the follow-up and for finding the story interesting. Below we bring together the results that address these concerns.

Novelty over chunk-based approaches. Chunk-based approaches score each chunk on its own, by its similarity to the question or by one judge call per chunk, or they read the chunks in sequence and carry notes forward. Periscope differs from them in three ways, and we measured each one.

First, Periscope never scores a chunk alone. Each chunk is read twice, once in its local span (a row of the grid) with its neighbors, and once in its strided span (a column) that takes one chunk from every region of the text. So each chunk is judged in context, without reading the whole text in one call. With the same model and one read of the text each, scoring every chunk alone gives 35.8% on the direct readout, against 40.0% for local spans of K chunks (LongBench v2, our first reply, W3).

Second, Periscope scores answers, not similarity. A retriever ranks chunks by how close they are to the question, with a trained model or an index. Each probe scores every answer against an abstain option, and a chunk scores high only when both of its spans support the same answer. So Periscope can answer from the probe scores alone, with no read at all. This direct readout scores 40.2%, above embedding retrieval followed by a read of its chunks, at 35.8% (LongBench v2, Table 1 of the submission).

Third, the probes are independent. A chain reads the chunks in sequence and waits for each note, which limits how well it uses the GPU. The probes run as one parallel batch and can be spread over several GPUs, so they use far more parallel hardware than a chain can. Chain of Agents, run over the same spans, scores 32.0% against 41.9% and takes three times as long (LongBench v2, Table 4(b) of the submission).

Together, these properties are what Periscope adds to chunk-based approaches, and the two tables below show that they lead to better results.

Compute-matched comparison isolating the local and strided probes. We matched compute by fixing the probes. Every layout with 2K probes uses the same number and size of probes over the same chunks. So each reads the same input tokens, and only the grouping changes. The overlapping row is the chunk-based design, contiguous windows that overlap by half, at exactly our compute. We ran this on BABILong with Qwen3.5-4B, on 750 questions of 32k to 1M tokens, where the positions of the supporting facts are known.

Probe partitionDirect readout (%)Selected read (%)Top-1, peak of the map on a supporting chunk (%)Chunks tied at the peakTop-K, supporting facts among the K chunks read (%)
Local and strided (Periscope), 2K probes34.138.424.41.051.4
Two random balanced partitions, 2K probes33.936.515.61.948.0
Overlapping contiguous chunks (local and shifted local), 2K probes34.436.06.79.741.4
Local only, K probes34.334.54.119.330.4

At the same compute, the local and strided probes put the peak of the map on a supporting chunk more than three times as often as overlapping contiguous chunks (+17.7 points, 95% CI +14.5 to +20.9). They also read more of the evidence (+10.0, 95% CI +7.6 to +12.5). Since compute is fixed and only the grouping changes, this isolates the benefit of the local and strided probes.

Existing chunk-based approaches. We ran this on LongBench v2 with Qwen3.5-4B, on the same 503 questions for every row, and we matched the read. Each retriever reads the same K chunks as Periscope select. ReadAgent keeps a short gist of every page and decides itself how many pages to reread, up to the same K. It rereads 5.6 pages on average.

MethodAccuracy (%)Compute (PFLOPs)Memory, largest call (tokens)Needs besides the LLM
Window read, 131k-token window41.91.2387.4knothing
Window read, 1M-token window39.28.4up to 1Mnothing
Periscope select41.94.619.4knothing
ReadAgent38.43.2843.3kgenerated notes
Trained reranker + read of K chunks39.62.359.5ka trained reranker
BM25 + read of K chunks36.00.079.5kan index
Embedding retrieval + read of K chunks35.81.919.4ka trained embedder
Chain of Agents32.02.429.2kgenerated summaries

Since the read is the same, these approaches differ only in how they choose what to read. Choosing with the local and strided probes gives the highest accuracy. Periscope matches the 131k window read without a trained model, an index or generated text.

So at equal compute the local and strided probes locate the evidence best, and at the same read they give the highest accuracy. We will add these results to the revised PDF. We hope this resolves the remaining points, and we are happy to clarify anything further.

Comments on the paper

2 comments

Dear Reviewers,

Kind reminder that the deadline of the discussion period is Nov. 18 so please please read the reviewers' response and take a minute to write an acknowledgment and confirm your final evaluation, as soon as possible.

Thanks a lot for your help.

We thank the AC and the reviewers for their time and constructive comments. Every point of the four reviews is answered in the reply under it, and the revised PDF reflects the replies (changes in blue).

Compute and memory (9Yfv, 5yHG, 9mdJ, fpZr). The shared concern of the reviews was the cost of the probes. We argue that this cost buys memory. Periscope lies on the frontier of memory and accuracy at every text length, and it pays for this with extra compute. But that happens only on short texts, where any read is cheap. On long texts, however, its probes need less compute and less time than a single read, so Periscope lies there on the frontier of memory, compute, speed and accuracy together, as our experiments show. (Section 4.6, Appendix A.2)

Novelty (9Yfv, 5yHG, 9mdJ). The reviews raised the novelty over GridProbe. We believe Periscope is a distinct method, as it differs from GridProbe in modality, problem, scoring, method components, readouts and scale. It targets a more useful and prominent problem, reading long texts on small GPUs, and achieves useful practical results on it. A compute-matched comparison of probe layouts and ablations of each part of the method show the value of each design choice. (Sections 2, 3 and 5)

Baselines (9Yfv, 9mdJ, fpZr). The reviews raised some missing baselines. Periscope outperforms them overall (BM25, a trained reranker, a chunk judge, ReadAgent, Chain of Agents, ParallelComp and EM-LLM). (Sections 4.3 and 4.4, Appendix A.4 and A.6)

Localization and statistics (5yHG, 9mdJ, 9Yfv). The reviews asked for evidence that the map localizes well. On BABILong, where the positions of the evidence are known, the map localizes the evidence well above the baselines, and the main comparisons now report confidence intervals, as requested. (Section 4.5, Appendix A.5)

Scope and presentation (5yHG, 9mdJ, fpZr). The reviews raised concerns about questions whose evidence is spread across several passages, and suggested improvements to the writing. We first showed empirically that a false peak, where the spans of two different facts cross, is rare. However, the selected read can still miss one of the facts, and we stated this case as a limitation of the paper. We also revised the writing, including the related work and Figure 2. (Sections 2, 3, 4.5 and 6)

We hope these results address the reviewers' concerns, and we thank the AC and all reviewers again for their time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.