acceptodds
Under review as a conference paper at ICLR 2027

CALIBER: Context-Aware LLM Intelligent Boundary Estimation and Routing

Abstract

Large language models (LLMs) support increasingly long context windows, yet the amount of context they can reliably use often falls well below their nominal limits and varies substantially across models. Existing long-context evaluations primarily measure synthetic retrieval capacity or aggregate task performance, providing limited guidance for selecting an operational context budget. We propose Context-Aware LLM Inference via Boundary Estimation and Routing (CALIBER), a framework that empirically estimates a model-specific effective context threshold and uses it to organize long-context inference. CALIBER measures answer correctness, faithfulness, and consistency risk across context lengths and identifies a conservative boundary based on sustained quality degradation and consistency criteria. It then hierarchically indexes documents into threshold-bounded subcontexts, routes queries to relevant indices, processes selected subcontexts in parallel, and aggregates their partial responses. Across 14 LLMs, CALIBER reveals substantial nominal-to-effective context gaps and, on three frontier models, significantly improves average EQ compared to baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Long-Context LLMs, Context Calibration, Retrieval-Augmented Generation, Hierarchical Indexing, Agentic Orchestration, Faithfulness Evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_E8TCOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work argues that longer contexts do not necessarily improve LLM performance on RAG tasks and proposes CALIBER for training-free context management. Specifically, during offline calibration, the authors construct EQ as the product of answer correctness (Pans), contextual support (Fsup), and consistency across multiple sampled responses (Cresp). They then estimate an effective context boundary, τeff, using sustained quality degradation and a consistency threshold. During online inference, this boundary guides the organization of hierarchical indices and subcontexts. Vector retrieval and LLM-based routing are followed by parallel generation and recursive aggregation into a final answer.

Critical strengths

S1 The research problem is relevant to practical deployment. The paper examines the gap between a model’s advertised context window and the amount of context it can reliably use, and attempts to translate quality measurements into an actionable context budget. This problem is directly relevant to configuring systems for long-document question answering and retrieval-augmented generation.

S2 The method requires no fine-tuning and offers a relatively complete system design. CALIBER uses existing models for offline calibration and applies budget constraints throughout source-subcontext organization, parallel inference, and recursive aggregation, without updating the target model’s weights. The framework offers a practical approach to implementing context management with existing models.

Critical weaknesses

W1 The relevance of the evaluated models and the need for the method on stronger models require further validation. My main concern is that the observed degradation with increasing context length may be confined to weaker models. In particular, the main experimental results cover only Gemini 2.5 Pro, GPT-4o, and Claude Sonnet 4.5. The existing length-bucket curves therefore do not establish that longer contexts generally weaken the capabilities of stronger models.

W2 The first stage depends heavily on the calibration tasks, with insufficient evidence of generalization and broad applicability. τeff explicitly depends on the model, calibration workload, and evaluation protocol, while correctness evaluation during calibration requires reference answers.

W3 Systematic comparisons with other context management methods are lacking. The main baselines in Table 2 are limited to Dense RAG and Self-RAG. RULER primarily provides an alternative budget in Table 3 and does not cover different context management strategies. The authors have not sufficiently demonstrated advantages over approaches based on grouped long-context reading, context compression, hierarchical summarization, or adaptive budgeting.

Questions for authors

Q1 On more recent and stronger long-context models, does consistent quality degradation still occur when only context length is varied while the question and relevant evidence are held fixed? Does CALIBER still offer clear benefits over direct long-context inference on these models?

Q2 Could the authors calibrate τeff only on source tasks, keep it fixed, and test its transfer to unseen tasks or domains? Could they also compare quality and cost against other context management strategies using the same document corpus and comparable computational budgets?

5 comments

[W1, Q1] Is long-context degradation still a problem?

Thank you for requesting this controlled evaluation. Long-context limitations remain relevant even for strong, recent models, although their severity varies across models and input conditions. We evaluated 120 items from SQuAD, HotpotQA, and FanOutQA, keeping questions and relevant evidence fixed while extending contexts to 8–128K with distractor passages, unrelated text, or neutral filler. Degradation was predefined as a loss of at least 5 points with a paired-bootstrap 95% confidence interval excluding zero.

Even among recent API models, Haiku 4.5 met this criterion in 2/6 tested experimental conditions and GPT-5.2 in 1/6, with maximum losses of 16.9 and 5.7 points. The other seven API models did not meet the criterion. Degradation was more frequent for Qwen3-4B (8/9 configurations, maximum loss of 27.5 points) and Llama-3.1-8B (5/9, 18.1 points). These findings show that stronger models can still exhibit length sensitivity and motivate calibrating the operating budget to each model.

Table R.4. CALIBER vs. Truncated Direct on Over-Window Documents (FanOutQA)

ModelDirectCALIBER (128K)Δ\Delta (pts) [95% CI]
Claude Haiku 4.542.252.6+10.42 [+4.46, +16.55]
Claude Sonnet 4.646.658.0+11.36 [+6.76, +16.20]
Claude Sonnet 5.545.755.4+9.79 [+3.84, +15.47]
GPT-5.243.047.8+4.83 [−4.83, +14.63]
Gemini 3.1 Pro28.131.8+3.73 [−3.54, +11.01]
GPT-5.538.938.6−0.37 (CI unavailable)
GPT-5.440.238.0−2.26 (CI unavailable)
Pooled (7 models)——+8.01 [+5.29, +10.82]

Benefits beyond the available window. Models that remain robust within their window still face documents exceeding their single-call capacity. On such held-out FanOutQA documents, CALIBER outperforms truncated Direct by 8.01 points [95% CI: 5.29, 10.82]. This demonstrates the benefit of bounded processing and aggregation for over-window inputs. Together, these results support CALIBER’s relevance to strong models through both model-dependent context calibration and processing of documents beyond a single-call window.

[W2, Q2] What does τeff\tau_{\mathrm{eff}} predict, and does it transfer?

Thank you for raising this question. CALIBER estimates a model- and workload-dependent operating budget from observed response quality. We examine whether this budget captures behavior beyond its calibration data and remains useful without target-specific tuning.

Predictive relevance and transfer. On an independent controlled-length evaluation, models with longer calibrated boundaries exhibit smaller performance declines from 8K to 32K, with a strong rank association across five models (Spearman ρ=0.90\rho=0.90). This suggests that τeff\tau_{\mathrm{eff}} captures meaningful differences in context sensitivity. On unseen LongBench v2, applying the fixed boundary without retuning yields positive mean gains over a 128K budget for both Haiku (+2.11 points, 95% CI [−3.52, +8.45]) and GPT-5.2 (+2.21 points, [−4.41, +8.82]). These results provide encouraging evidence for transfer, although the confidence intervals do not establish significant improvements.

Calibration stability and label dependence. Leaving out one calibration source shifts the boundary by at most one bucket in 37/55 cases (67%), indicating stability across source compositions. The reference-free Fsup×CrespF_{\mathrm{sup}}\times C_{\mathrm{resp}} variant also reproduces the full Pans×Fsup×CrespP_{\mathrm{ans}}\times F_{\mathrm{sup}}\times C_{\mathrm{resp}} boundary in 9/13 models, suggesting feasibility without reference answers. These findings support reusing calibrated budgets across related workloads with reduced dependence on labeled data, while recommending recalibration for substantial task or domain shifts.

[W3, Q2] Comparison to alternative context-management strategies

We evaluated CALIBER against adaptive retrieval, grouped reading, context compression, and hierarchical retrieval methods on matched items and models.

Table R.5. CALIBER improvements (points, 95% CI).

BaselineΔ [95% CI]
Adaptive-kk (Llama-3.1-8B)+16.2 [+10.3,+22.0]
Adaptive-kk (Qwen3-4B)+12.1 [+7.1,+17.1]
LongRAG+13.91 [+8.85,+18.80]
DOS-RAG+8.03 [+5.16,+10.98]

All five comparisons show statistically significant gains, including adaptive retrieval with Adaptive-kk.

Table R.6. Additional strategy comparison (Qwen2.5-14B-Instruct).

Strategy / MethodScoreTotal Input
CALIBER49.2150.3K
Grouped reading / LLM×MapReduce42.8372.6K
Compression / LLMLingua-2 22K45.9346.8K
Hierarchical retrieval / MiA-RAG top-1047.0860.8K

Tables R.5–R.6 show CALIBER's quality gains across adaptive retrieval, grouped reading, compression, and hierarchical retrieval, with lower total input-token usage in Table R.6.

Quality and computational trade-offs. CALIBER achieves the highest score (49.2), outperforming grouped reading (LLM×MapReduce, +6.4 points), context compression (LLMLingua-2 22K, +3.3), and hierarchical retrieval (MiA-RAG top-10, +2.2). Despite its multi-stage architecture, CALIBER processes only 150.3K total input tokens, compared with 372.6K, 346.8K, and 860.8K for these alternatives, corresponding to reductions of 59.7%, 56.7%, and 82.5%, respectively.

These results suggest that CALIBER's targeted routing and bounded subcontext processing can retain relevant distributed evidence more effectively than the evaluated compression and grouped-reading approaches, while avoiding repeated processing of large portions of the source corpus. Unlike approaches that rely primarily on compressed representations or broad batch processing, CALIBER generates partial answers directly from selected raw evidence and aggregates them within calibrated context budgets.

Importantly, the reported total input includes upstream processing as well as generation, revealing costs that generation-only comparisons may overlook. While these results demonstrate favorable quality–token trade-offs in the evaluated setting, token counts do not directly measure FLOPs, latency, or monetary cost.

Dear Reviewer E8TC,

We know discussion periods get busy for everyone, and we appreciate the time you've already put into this review. With the window now closing, we wanted to follow up: we posted a response to W1–W3 and Q1–Q2, including the new controlled-length experiment isolating degradation from over-window coverage, the τ_eff transfer and stability results on unseen data, and the extended comparison against Adaptive-k, compression-based, and batch-reading baselines.

We'd really value hearing whether this addresses your concerns, particularly around model strength and systematic baseline coverage, or whether there's anything you'd like us to dig into further. We're glad to keep discussing for as long as is useful before the deadline.

Thank you for pushing us to tighten these comparisons — it's made the evaluation substantially stronger.

Reviewer_kaxUOpenReview ↗
Rating
2(+1)
Confidence
4(+1)
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

Summary CALIBER estimates a model- and workload-dependent effective context boundary using answer correctness, contextual support, and response consistency, then applies this boundary to hierarchical context organization, routing, parallel inference, and aggregation. The paper calibrates 14 models and reports average EQ improvements of 19.2% over dense RAG and 8.6% over Self-RAG across three models and five benchmarks. My main concern is that these system-level gains do not sufficiently establish the independent value of the proposed boundary estimator.

Critical strengths
  1. Practical relevance and broad model coverage. Choosing a usable context budget is an important deployment problem, and the calibration study covers both proprietary and open-weight models with substantially different effective boundaries. This breadth is a useful aspect of the empirical study.

  2. Useful diagnostic analyses. The paper includes component ablations, evaluator checks, and detector-sensitivity analyses, helping clarify which parts of the pipeline contribute and where boundary estimates may vary.

Critical weaknesses

1 — Major: The boundary estimator's advantage over simpler alternatives is insufficiently established. The gap between advertised and effective context lengths has already been documented in prior work such as RULER. The contribution therefore depends on demonstrating the value of the proposed estimator, rather than the existence of this gap. A decisive test would hold the backbone and orchestration pipeline fixed and compare CALIBER's estimator with budgets selected by validation accuracy, a near-optimal-accuracy tolerance rule, and a fixed-budget sweep. The supplementary correctness-signal ablation does not replace this comparison: it retains a shared consistency criterion and examines selected boundaries rather than downstream performance. Thus, the necessity of the additional calibration machinery remains insufficiently supported.

2 — Major: The organizational contribution needs sharper positioning and separation from boundary estimation. Hierarchical organization and selective access have precedents: MiA-RAG constructs a global representation through hierarchical summarization to guide retrieval and generation. G-Memory uses a three-tier memory graph with query-conditioned traversal. HiAgent groups trajectories by subgoal, summarizes completed groups, and retrieves details when needed. These methods address different settings and are not equivalent to CALIBER, but they motivate a clearer account of what is novel in its organization mechanism. Moreover, CALIBER's orchestration can operate with alternative budgets, as its own RULER-budget ablation demonstrates. The paper should more clearly distinguish the contribution of the organization pipeline from that of the specific estimator. Agent-memory methods are relevant conceptual comparisons, not necessarily direct QA baselines.

3 — Secondary: The main benchmark comparison lacks key alternatives. The main end-to-end results compare only dense RAG and Self-RAG. Adaptive-k is a particularly relevant alternative: it selects retrieval size from the query–passage similarity distribution without fine-tuning or iterative prompting. Comparing against such a lightweight approach would more directly test whether model-side calibration offers additional value beyond simpler adaptive context selection. The appendix includes LongRAG and DOS RAG adaptations, but only in three purposively selected cases with successful archived CALIBER outputs, without compute matching. These examples do not substitute for systematic benchmark-level comparisons. Controlled comparisons should also report input/output tokens, model calls, and latency.

4 — Secondary: The reported ablation gains are modest and lack uncertainty estimates. In the FanOutQA ablation, replacing the RULER-derived budget with the proposed boundary improves Loose Accuracy from 0.739 to 0.752 for Claude Haiku 4.5 and from 0.709 to 0.723 for GPT-4o—absolute gains of 1.3 and 1.4 percentage points, respectively, or relative improvements of approximately 1.8% and 2.0% over the RULER-budget versions. The table provides no uncertainty estimates, so the statistical reliability of these modest differences remains unclear. Meanwhile, removing orchestration reduces GPT-4o's score from 0.723 to 0.674, suggesting that the surrounding inference pipeline contributes substantially to the overall benefit. This reinforces the attribution concern in Weakness 1, rather than establishing that the estimator has no value.

Questions for authors

none

5 comments

[W1] Comparison with accuracy-based budget selection

Thank you for requesting a direct comparison with simpler budget-selection rules. With the inference pipeline fixed, Validation-based selection yields test regrets of 1.66 points on FanOutQA, matching the fixed 128K budget, and 6.34 points on LongBench v2. These regrets represent performance gaps relative to the oracle, which selects the optimal context budget using test-set performance.

Why consider more than accuracy? Accuracy-based selection overlooks contextual support and response consistency. Using the original calibration data, CALIBER's τeff\tau_{\mathrm{eff}} achieves higher faithfulness in all 13 models and higher overall quality (EQ=P×F×CEQ=P\times F\times C) in 8 of 13 models, despite an average accuracy reduction of 0.08.Compared with the 10% accuracy-tolerance rule, τeff\tau_{\mathrm{eff}} improves mean EQ from 0.297 to 0.364 (+23%), outperforming the rule in six models: Gemini 2.5 Pro (+59%), Phi-3-Mini (+33%), GPT-4o (+27%), Claude Sonnet 4.5 (+13%), GPT-5.2 (+13%), and Claude Haiku 4.5 (+8%).

Furthermore, comparing shorter (8–24K) and longer (≥80\geq80K) contexts, 8 of 18 model–dataset pairs exhibit statistically significant consistency degradation without significant accuracy changes. For example, Claude Sonnet 4.5/NarrativeQA shows ΔP=+0.037\Delta P=+0.037 (95% CI: [−0.034,+0.110][-0.034,+0.110]), but ΔC=−0.130\Delta C=-0.130 (95% CI: [−0.169,−0.094][-0.169,-0.094]). These results highlight why effective context boundaries should consider accuracy, faithfulness, and consistency jointly.

These observations show that accuracy alone can leave changes in response stability uncharacterized. CALIBER incorporates correctness, contextual support, and consistency through EQ=PansFsupCrespEQ=P_{\mathrm{ans}}F_{\mathrm{sup}}C_{\mathrm{resp}}, allowing boundary estimation to respond to these complementary signals. This motivates a reliability-oriented operating budget rather than one selected solely to maximize validation accuracy.

[W2, W4] Separating organizational gains from boundary estimation

Thank you for highlighting the need to distinguish these contributions. CALIBER’s organization determines how evidence is processed and combined, while its estimator determines the context budget for each call. Our contribution lies in using an empirically calibrated, model-specific context boundary to guide the organization and processing of distributed evidence. Specifically, CALIBER (1) uses summaries to route queries to relevant source subcontexts, (2) generates partial answers directly from the raw source content within the calibrated budget, and (3) aggregates these answers under the same budget constraint. This design connects boundary estimation to the inference pipeline without requiring additional training. This differs from MiA-RAG’s summary-conditioned retrieval and generation and from G-Memory/HiAgent’s organization of agent histories.

Controlled comparisons. We examine organization at a fixed per-call budget and boundary selection with the inference pipeline held fixed.

Table R.2. Loose accuracy difference in points with 95% confidence intervals. Boundary comparisons use 210 held-out FanOutQA items, pooled across Haiku (τeff=56\tau_{\mathrm{eff}}=56K) and GPT-5.2 (τeff=112\tau_{\mathrm{eff}}=112K).

ContributionComparisonΔ\Delta [95% CI]
OrganizationCALIBER − DOS-RAG, 16K+8.03 [+5.16,+10.98]
OrganizationCALIBER − front truncation, 56K+12.86 [+9.82,+15.91]
Boundary selectionτeff\tau_{\mathrm{eff}} vs 8K, same pipeline+4.71 [+2.38,+7.11]
Boundary selectionτeff\tau_{\mathrm{eff}} vs. 16K, same pipeline+4.14 [+1.84,+6.48]

The organizational comparisons support the value of processing and combining distributed evidence beyond setting a context limit alone. The fixed-pipeline comparisons show that overly short budgets can also limit performance. Conversely, Mistral-NeMo-12B performs better at 8K than at 56K, indicating that simply increasing the budget is not uniformly beneficial. Together, these comparisons provide separate evidence for the value of the organizational pipeline and for adapting the per-call budget to the model. The former improves how distributed evidence is processed, while the latter improves performance over the tested fixed-budget alternatives within the same pipeline.

Relation to MiA-RAG. On the evaluated FanOutQA subset with contexts of at least 250K tokens, we observe cases where MiA-RAG retrieves all gold values but its generator selects the wrong entity or returns an incomplete list. CALIBER’s stronger answers in these cases suggest that evidence coverage alone is insufficient and support the role of subcontext-level processing followed by aggregation. We treat these observations as supplementary evidence about the organizational mechanism and will revise the paper to distinguish this contribution from boundary estimation more explicitly.

[W3] Adaptive-k: evidence coverage versus computational cost

Thank you for suggesting Adaptive-kk. We applied its official similarity-gap selection to the same 150 FanOutQA items using matched readers, prompts, and seeds. Adaptive-kk selects a query-dependent number of 512-token chunks, whereas CALIBER routes queries to relevant subcontexts and processes them within a per-call budget. Adaptive-kk selects 6–7 chunks on approximately two-thirds of these queries, which may omit evidence needed for questions requiring aggregation across multiple locations.

Table R.3. Routed CALIBER minus Adaptive-kk, in score points with 95% confidence intervals. τ\tau denotes CALIBER’s per-call budget. Generation input counts are summed across generation calls per query.

Modelτ=8\tau=8Kτ=16\tau=16KGeneration input/query at 8K, CALIBER vs Adaptive-kk
Llama-3.1-8B+16.2 [+10.3,+22.0]+19.5 [+13.7,+25.3]68.5K vs 6.0K
Qwen3-4B+12.1 [+7.1,+17.1]+19.3 [+14.3,+24.5]67.9K vs 5.3K

CALIBER achieves higher answer quality in all four configurations, with confidence intervals excluding zero. For Llama on documents of at least 250K tokens, the improvement at an 8K per-call budget reaches 24.2 points [13.0, 36.1]. These results support the value of processing evidence across multiple bounded subcontexts for aggregation-heavy questions.

Computational trade-off. CALIBER uses more generation tokens than Adaptive-kk(approximately 68K vs. 5–6K per query at an 8K budget), achieving statistically significant quality gains of 12–16 points. However, generation tokens alone do not capture the full processing cost. In a separate evaluation on 45 FanOutQA questions with documents exceeding 250K tokens (Qwen2.5-14B), Adaptive-kk processes 406.8K total input tokens, including document embeddings, compared with CALIBER's 150.3K—approximately 2.7× more input tokens. Despite 16 generation calls versus one, CALIBER's reported end-to-end latency is comparable (44.8s vs. 47.9s), including Adaptive-kk's embedding time. These results highlight the importance of accounting for preprocessing and generation separately; token counts are not equivalent to FLOPs, and latency comparisons depend on whether indexing is amortized across queries.

Dear Reviewer kaxU,

We hope this finds you well, and we fully understand how busy the discussion period can be. We wanted to check in as the window is closing: we posted a detailed response addressing W1–W3 (the accuracy-only budget comparisons, the organization-vs-estimator attribution with the new MiA-RAG and held-out ablations, and the Adaptive-k coverage/cost comparison), and we wanted to make sure it reached the points you raised.

We'd be very grateful to know whether any concerns remain, or whether anything in our response needs further clarification or additional evidence. We're fully open to continuing the discussion in whatever depth would be useful before the period closes.

Thank you again for the detailed and constructive review — it meaningfully improved the paper.

Reviewer_4JoBOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This work proposed CALIBER, a framework that empirically estimates a model-specific effective context threshold (τ_eff ) and uses it to organize long-context inference. It identifies a conservative boundary, hierarchically indexes documents into threshold-bounded subcontexts, routes queries to relevant indices, processes selected subcontexts in parallel, and aggregates their partial responses.

Critical strengths
  • The proposed framework can empirically estimate a model-specific effective context threshold, which can be used to guide boundary-aware inference with long contexts.
  • The evaluation is comprehensive, covering a wide range of backbone models and several benchmark datasets.
  • The experimental analyses provide meaningful support for the paper’s main claims and are complemented by further discussion in the paper.
Critical weaknesses
  • My main concern is that the framework’s two-stage routing and evaluation design may introduce additional computational costs, reduce efficiency, and make the results more difficult to reproduce. Could the authors discuss these trade-offs more explicitly and clarify the framework’s intended scope of application?
  • The hyperparameters in the proposed framework also increase the diffiulty, yielding 81 configurations per model.
Questions for authors
  • How to decide the hyperparameters in this paper and what if researchers want to work on a new dataset?
  • What are the factors that might influcence reproducibilty of the proposed framework?
2 comments

[W1] Routing cost and efficiency

Thank you for raising this concern. We agree that CALIBER’s additional stages require a clear discussion of computational trade-offs. Calibration and corpus indexing are performed offline and reused across queries. So, the correctness, support, and consistency evaluations are not repeated for each incoming query.

Online cost and trade-off. Across all six models, the LLM router uses 130–290 tokens per query, under 1% of total input. Routing without a filtering threshold and reading only the top 10 pages uses 43–44% of the input of the no-routing configuration while achieving 0.97 gold-page recall. Across 900 paired comparisons, the mean quality difference is −1.05 points 95% CI: [−1.96, −0.16]. Thus, routing reduces input-token usage by approximately 56% at an average quality decrease of 1.051.05 points.

Scope of application. CALIBER primarily targets a fixed backbone model, repeated queries over a stable corpus, and inputs exceeding τeff\tau_{\text{eff}} with evidence distributed across multiple locations. These settings allow the offline costs to be amortized. When the full input fits within τeff\tau_{\text{eff}}, a single generation call is sufficient. Simpler pipelines may therefore be preferable for short-context or low-volume applications, while substantial model or workload changes may require recalibration. We will clarify these trade-offs and deployment conditions in the revision.

[W2, Q1] The 81 configurations and how hyperparameters are chosen

Thank you for raising this concern. The 81 configurations constitute a sensitivity analysis (343^4 combinations), rather than a hyperparameter search required for each model or dataset. Our main experiments use one shared configuration, δdrop=0.15\delta_{\mathrm{drop}}=0.15, w=3w=3, ρrec=0.90\rho_{\mathrm{rec}}=0.90, and θcons=0.40\theta_{\mathrm{cons}}=0.40, across all 14 models and benchmarks, without per-model tuning.

Sensitivity to hyperparameters. Table 4 in the paper reports how the estimated boundary changes when the detector parameters are varied around the default. Among 959 comparisons with defined boundaries, 689 (71.85%71.85\%) remain within one bucket of the default, with a mean absolute shift of 1.56 buckets. Thus, many nearby configurations yield similar boundaries, although sensitivity varies across models and some exhibit larger shifts. These results characterize boundary stability rather than establishing downstream optimality.

Application to new datasets. We distinguish the shared detector settings from the data-dependent boundary τeff\tau_{\mathrm{eff}}. The drop and recovery criteria measure changes relative to the model’s preceding quality level, allowing the same detection rule to accommodate different quality scales. For a new dataset, we recommend retaining the default detector settings and estimating τeff\tau_{\mathrm{eff}} using representative calibration examples, without repeating the 81-configuration sweep. As our workload-composition analysis shows (Appendix E.4), the boundary can depend on the calibration workload; substantial domain or task shifts therefore call for recalibration rather than assuming that a previously estimated boundary transfers unchanged.

[Q2] Reproducibility factors

Thank you for this question. Reproducibility depends on model and evaluator versions, prompts and decoding settings, calibration splits, and the handling of sparse buckets and missing scores. We additionally assessed three implementation choices across the 13 models with a defined default boundary, measuring boundary shifts as index differences on the ordered 16-bucket grid. Table R.1. Sensitivity of the estimated context boundary to implementation choices across 13 models with a defined default boundary. Shifts are measured as index differences on the ordered 16-bucket grid. For fold splits, we report the median model-level mean shift and the number of models with a mean shift within 0.5 buckets.

FactorChangeBoundary stability
Minimum bucket sample size3 → 1 / 5 / 1013/13 (all three settings)
Missing-judge-score handlingimpute → exclude11/13
Fold splitmean shift over 50 foldsmedian 0.38 buckets; 10/13 within 0.5 buckets

The estimated boundaries are unchanged under the tested minimum-sample requirements and generally show limited variation across calibration splits. Missing-score handling affects two models, highlighting the importance of specifying this rule rather than treating it as an inconsequential implementation detail.

To support reproducibility, we will document these choices and provide code, configurations, fixed data splits and random seeds, available model/version identifiers, and per-query call logs. These materials will make the calibration and inference procedures explicit and allow the reported boundary estimates to be traced to their underlying observations.

Dear Reviewer 4JoB,

Thank you again for your thoughtful review — we understand this is a demanding time for reviewers. As the discussion period is nearly over, we wanted to check whether our response reached your concerns: we addressed the routing cost/efficiency tradeoffs and scope of application (W1), clarified that the 81 configurations are a sensitivity grid rather than a tuning search (W2, Q1), and reported the reproducibility factors across bucket size, missing-score handling, and fold splits (Q2).

If any concerns remain, or if you'd like us to expand on anything before the window closes, we're very happy to continue the conversation in as much depth as helpful.

We appreciate the constructive engagement throughout this process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.