acceptodds
Under review as a conference paper at ICLR 2027

Perplexity-Guided Trajectory Repair Shows No Advantage over Length-Matched Controls

Abstract

Recovery from a failed agent trajectory can restart the task or keep a prefix and regenerate its suffix. We test whether stored-token perplexity helps choose where to regenerate, beyond positional controls, in ReAct-style QA agents. In a pre- specified 250-question HotpotQA study, two-step backtracking from the highest- perplexity step achieves 9.73% repair success versus 11.21% for restart (95% in- terval −4.13 to +0.88 points). A diagnosis/replay follow-up, a frozen two-model, three-dataset replication and a deadline study find no clear advantage in any pre- specified Holm family. A length-matched swap, which gives each failed question the uncertainty-chosen origin of an equal-length partner, holds origin distributions exactly equal by construction and is also null, including two confirmatory cells with enough discordant pairs to resolve a large effect: −0.17 points (p = 1.000) at 32B and +1.48 (p = 0.397) at 72B. All experiments pass replay, scoring and allowance audits. These results limit claims for this policy in offline QA; they establish neither equivalence nor an advantage under equal total compute.

Primary area:
foundation or frontier models, including LLMs
Keywords:
llm agents, trajectory repair, uncertainty estimation, negative results, multi-hop question answering, react, experimental design, reproducibility

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_SuYHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper tests whether a simple uncertainty rule helps a failed ReAct-style QA agent choose where to regenerate its trajectory. The rule backtracks two steps from the step with the highest perplexity. Across a main HotpotQA study, replications on two models and three datasets, and budget and deadline variants, the rule never beats restart or positional controls. The key methodological contribution is a length-matched swap control: failed trajectories of equal length exchange their uncertainty-chosen origins, so both arms have exactly the same origin distribution. Two confirmatory swap studies, powered on discordant pairs, are also null: −0.17pp at 32B and +1.48pp at 72B.

Critical strengths
  1. The rigor is exemplary. Protocols are frozen before execution, and cohorts and exclusions are documented. Inference uses paired question-level bootstrap intervals and exact sign-flip tests with Holm correction. Replay, scoring and budgets are fully audited, and a reproduction supplement is provided. Negative results reported this carefully are rare in the agent literature.
  2. The swap control is clever. Both arms share an identical set of origins, so it separates what the uncertainty signal adds from mere positional preference more cleanly than a position control fitted on development data. It could be reused to evaluate other origin selectors.
  3. The discussion of statistical resolution is also valuable. The authors point out that an exact sign-flip test over m non-zero blocks cannot report a p-value below 2/2^m, so several earlier cells could never have reached significance. They respond by powering the confirmatory cells on discordant pairs rather than on failures.
Critical weaknesses
  1. The tested selector is weak, which limits significance. The rule (highest-perplexity step minus two) is the simplest possible one. It is motivated only by the authors’ own uncontrolled, unpublished observations, not by any claim in the literature, so refuting it tells readers little. The authors also say the conclusions do not extend to entropy, action disagreement, verbalized confidence, trained diagnosers or search-based methods.
  2. There is no headroom analysis. The paper never shows that origin choice matters in this setting. It reports neither an oracle upper bound (the best origin chosen in hindsight) nor how much success varies across origins. Without these, the null cannot separate “perplexity is uninformative” from “origin choice is unimportant in this task.”
  3. Floor effects and budget limits may drive the result. All policies succeed on about 10% of attempts. The treatment’s effective origin is zero in 62% of rows, so it often turns into a restart. In addition, 168 of 339 treatment attempts stop at a token or step cap without producing an answer. This suggests the recovery budget is too tight. Results under looser budgets are needed; the uncapped runtime study covers only 25 questions.
  4. Verification is incomplete. The internal audits had not reported by submission. The 32B confirmatory cell finished only 100 of its 131 informative pairs, which deviates from its protocol. The 72B cell should be presented as the primary confirmatory result.
Questions for authors
  1. Budget sensitivity: Does the swap control give a different result under a 2× or looser recovery budget?
  2. Equivalence testing: Would you consider an equivalence test against a prespecified smallest effect size of interest? What effect size would you regard as practically meaningful?
  3. Alternative signals: The swap framework applies directly to other signals such as entropy or action disagreement. Why not test at least one alternative signal within it?
5 comments
AuthorsOpenReview ↗

Response to Reviewer SuYH - Part 1

We thank Reviewer SuYH for a careful and constructive review and for three concrete questions. The review notes that the arms of the length-matched swap share one origin set, and that few discordant pairs cap the sign-flip p-value. In response, the revision adds new analyses of the existing runs and makes the complete 72B cell the primary confirmatory result. This reply also adds an agreement analysis for two other uncertainty signals. The new analyses were specified after all outcomes were known and are exploratory. The exception is the allowance success rates, which come from the prespecified diagnosis follow-up. P-values are unadjusted unless marked Holm, and paper pointers refer to the revised PDF by section and appendix name.

W1. Simplicity of the selector

We agree that the rule is the simplest one and that a null on it has limited reach. The rule came from our earlier uncontrolled runs. FLARE (Jiang et al., 2023, EMNLP) and DRAGIN (Su et al., 2024, ACL) use low token confidence to trigger retrieval and then regenerate the affected text within one generation, and the revision now cites both in Related Work. The use of the rule on a finished failed agent trajectory and the two-step backoff are ours.

The claim covers this rule, the two open-weight model families tested and offline QA. Table R5 under Q3 shows how often two other signals pick the same origin. In the revision, the future-work paragraph names the swap control on other signals, such as entropy and action disagreement.

AuthorsOpenReview ↗

Response to Reviewer SuYH - Part 2

W2. Whether origin choice matters

We agree that this analysis was missing. The revision adds two analyses, in the Analysis and Discussion paragraph "Does origin choice matter in this setting?" and in the appendix named in Table R1. The first is a cross-fitted choice among tested origins. For each failed question with at least two tested origins, we chose the origin with the highest mean success on two seeds and scored it on the third. The 95% bootstrap intervals resample questions, or whole pairs in the swap cells.

Table R1. Cross-fitted headroom. Source: appendix "Exploratory Headroom, Precision and Allowance Analyses".

CellQuestionsChosen (%)ComparatorComparator (%)Difference, points [95% interval]
Main11313.27Restart11.21+2.06 [−1.18, +5.90]
Main11313.27Treatment9.73+3.54 [0.00, +7.67]
Diagnosis 1×9113.92Restart11.36+2.56 [−0.37, +6.23]
Diagnosis 1×9113.92Treatment10.62+3.30 [−1.10, +8.06]
72B swap24814.78Treatment12.90+1.88 [−0.13, +4.03]
72B swap24814.78Swap11.42+3.36 [+0.94, +5.91]
32B swap20014.00Treatment11.50+2.50 [0.00, +5.33]
32B swap20014.00Swap11.67+2.33 [+0.17, +4.67]

Chosen origins gain 1.88 to 3.54 points over the policy. The two intervals that lie fully above zero are both against the swap arm, and two more have a lower limit of 0.00. These are cross-fitted references limited to tested origins and chosen from two noisy seeds, not bounds on a hindsight oracle. A deployable selector would lack the other seeds.

The second analysis tests whether origins are consistent across seeds. A positive statistic means that an origin succeeding on one seed tends to succeed on another seed of the same question. The null permutes origin labels within question and seed, 2,000 times. Holm covers the four all-origin tests, and the retained-prefix test drops origin zero.

Table R2. Origin consistency, one-sided permutation p. Source: appendix "Exploratory Headroom, Precision and Allowance Analyses" for the p-values. The counts of questions with any variation were computed for this reply from the packaged trial exports and are not in the paper or the current supplement.

CellQuestions with any variationp, all originsHolm pRetained-prefix questions with any variationp, retained-prefix origins
Main41 of 1130.0040.01618 of 600.109
Diagnosis 1×27 of 910.0070.0228 of 220.077
72B swap72 of 2480.0290.05921 of 620.475
32B swap59 of 2000.0370.05925 of 980.978

The statistic needs variation on at least two seeds. This holds for 19, 12, 35 and 17 questions across all origins but for only 8, 2, 11 and 5 retained-prefix questions, so the retained-prefix tests are too small to tell whether consistency differs among origins that keep a prefix. Origin choice appears to carry some signal across seeds that perplexity did not turn into a gain over restart, which partly separates your two readings.

Edited on Nov 12 after partial internal audit reports: two p-values in Table R2 and the description of the cross-fitted reference.

AuthorsOpenReview ↗

Response to Reviewer SuYH - Part 3

W3 and Q1. Floors, tight caps and the swap at 2×

We partly agree, and we correct one fact about the runtime study below. The completed swap cells ran only at 1×. To answer this, we have started one new cell, a 72B length-matched swap on fresh HotpotQA questions with twice the token allowance and 16 new steps instead of 8, under a protocol frozen before launch. It keeps the same primary exact sign-flip test and adds an equivalence test with a 3-point margin fixed before launch (corrected after launch, Part 4). Tokens and steps change together, so the cell cannot separate the two limits.

The prespecified diagnosis follow-up ran restart and uncertainty at three allowances on the same 118 failures, with 354 seed attempts per cell. The cap reading below is exploratory.

Table R3. Success and cap stops by allowance. Source: appendix "Exploratory Headroom, Precision and Allowance Analyses" for success and uncertainty any-cap stops, and the "Recovery allowance" paragraph for the uncertainty token-cap stops at 1× and 2×. The other columns were computed for this reply from the packaged trial exports and are not in the paper or the current supplement.

AllowanceSuccess, uncertainty (%)Success, restart (%)Token-cap stops, uncertaintyToken-cap stops, restartStep-limit stops, uncertaintyAny-cap stops, uncertainty (%)Any-cap stops, restart (%)
0.5×2.822.82335/354336/3540/35494.6394.92
1×9.8910.45114/354111/35473/35452.8253.67
2×11.0212.432/3542/354139/35439.8338.14

The token cap binds at 1× but rarely at 2×, where the eight-step limit becomes the binding cap. Uncertainty minus restart is −0.56 points at 1×, 95% interval [−3.67, +2.54], and −1.41 points at 2×, [−4.80, +1.41], with Holm p 1.000 for both. These contrasts are in the appendix "Diagnosis Follow-up: Complete Results and Costs".

At 1× in the 72B cell, 493 of 744 treatment attempts (66.26%) and 512 of 744 swap attempts (68.82%) stopped at a cap, compared with 49.56% (168 of 339) of main-study treatment attempts. The swap cells may therefore be more constrained by the allowance, and the new cell tests this directly. The 62% origin-zero rate, 210 of 339 rows, means that treatment and restart share all three executions on 70 of 113 main questions. On the other 43 questions, treatment minus restart is −3.88 points, 95% interval [−10.85, +2.33], as reported in the appendix "Distinct Intervention, Resources and Metric Sensitivity".

The runtime study removed only the total generated-token cap. It kept the eight-new-step limit and the 512-token per-step limit. Of its 225 attempts on 25 questions, 79 stopped at the step limit and 0 at a token limit. It is therefore a looser token budget, and the step limit still binds.

In the revision, the "Recovery allowance" paragraph and the appendix "Exploratory Headroom, Precision and Allowance Analyses" report the success rates and the uncertainty and swap-cell cap rates. The other Table R3 columns come from the analysis output. The Budget Scope subsection states that the runtime experiment retains the step limits, and future work names the swap under looser token and step limits.

W4. Verification and ranking of the confirmatory evidence

We agree, and the revision follows the suggestion. The complete 72B cell is now primary. It ran all 124 informative pairs, and 51 are discordant, with 30 positive and 21 negative. Uncertainty succeeded on 12.90% and the swapped origin on 11.42%, a difference of +1.48 points, 95% interval [−1.48, +4.44], p=0.397 from the exact pair-block sign-flip test. The completed design has power 0.96 at a discordant-pair win probability of 0.75 and 0.84 at 0.70. These are powers of a sign-only test on the 51 discordant pairs. The weighted sign-flip test used for p has lower power, 0.89 and 0.69, with the observed block magnitudes held fixed. The packaged analysis of the cell reports zero replay, scoring or allowance mismatches over 1,488 trial rows.

The 32B cell is a declared deviation. It stopped at its frozen time limit with 100 of 131 informative pairs run and 31 unrun. Its difference is −0.17 points, 95% interval [−3.17, +2.84], p=1.000. The internal auditors are two co-authors, and the revision says so. Partial audit reports arrived during the discussion period, and the revision corrects what they found. It still calls internal verification incomplete.

In the revision, the Abstract and Conclusion report the 72B cell before the 32B cell, and the section "Replication across Models and Datasets" names it our primary confirmatory result. The Abstract and that section call the 32B cell a declared deviation, the Conclusion calls it truncated, and the appendix "Confirmatory Length-Matched Swap" reports it as a deviation.

Edited on Nov 12 after partial internal audit reports: the power description, the audit status and the equivalence test.

Reviewer_y4CcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether step-level perplexity can identify effective regeneration points for repairing failed ReAct-style question-answering trajectories. The proposed policy selects the step with the highest perplexity, computed from the log probabilities of sampled tokens, and backtracks two trajectory positions before resuming generation. The experimental results show no clear advantage over baselines such as restarting from scratch.

Critical strengths

The paper studies whether step-level perplexity can identify effective repair points, a valuable research question. Its findings challenge the assumption that high uncertainty necessarily signals a useful intervention point.

Critical weaknesses
  • The paper offers limited methodological novelty, primarily evaluating a simple heuristic that combines perplexity-based step selection with fixed backtracking.

  • I strongly suspect that this paper was generated entirely by AI, as it uses a lot of defensive language, which makes the paper read unnaturally. For example, many paragraphs end with negative statements or caveats:

    • Lines 57–58: Neither uncertainty nor agreement with an error annotation directly establishes the value of intervening at that step.
    • Line 67: not a new claim about prefix reuse or causal attribution.
    • Lines 72–73: an apparent benefit from an uncontrolled internal baseline, not a published claim, and exactly the kind of evidence a control can remove.
    • Lines 79–80: Neither the main study nor replication establishes an advantage for this policy in its prespecified comparison family.
    • Lines 87–88: its valid JSON and absent replication fallbacks establish parser coverage, not diagnosis quality.

Many other passages exhibit the same pattern but are not listed here.

Questions for authors

Although AI can help with research, I believe the authors should take responsibility for all content in the paper, including any AI-generated material and the reported experimental results. Please carefully polish the paper and verify the experimental code and results.

1 comment
AuthorsOpenReview ↗

Response to Reviewer y4Cc

We thank Reviewer y4Cc for the review and for raising authorship and verification directly. The review notes the research question, whether step-level perplexity marks useful repair points, and the finding that high uncertainty need not mark a useful point to intervene. We answer the novelty concern with the length-matched swap control, and the authorship concern with the quoted passages and the checks we ran.

W1. Novelty

We agree that the rule is simple. The study adds a length-matched swap control. Failures of equal length exchange their chosen origins, so both arms repair from identical origin distributions and only the match between origin and question differs. In the complete 72B cell, the question's own origin succeeded on 12.90% of attempts, compared with 11.42% for the swapped origin. The difference was +1.48 points, with a 95% interval of [−1.48, +4.44] and p=0.397 over 124 pairs.

In the revision, the Abstract and Conclusion report the 72B cell before the 32B cell, and the Replication section names it the primary confirmatory result. The Conclusion now calls the 32B cell truncated.

W2 and Q1. Authorship, responsibility and checks

The style criticism is fair, and the passages you quote show the pattern. The work used extensive AI assistance, and the submission form and the AI Use Statement disclose it, including research ideation, implementation, interpretation and drafting. The paper was not written entirely by AI. The authors wrote parts of the manuscript themselves, read and edited the AI-drafted text, and checked the results as described below. We take full responsibility for all content of the submission, including the reported results and any error the AI assistance introduced.

Two of the quoted passages state the limit of a claim, so we kept them and moved them inside their paragraphs. Table R1 lists all five.

Table R1. The five quoted passages in the revision.

Review linesPassage as quotedRevisionWhere it sits now
57–58"Neither uncertainty nor agreement with an error annotation directly establishes the value of intervening at that step."Kept, moved ahead of the two closing examplesIntroduction, paragraph on error attribution and recovery
67"not a new claim about prefix reuse or causal attribution."Removed—
72–73"an apparent benefit from an uncontrolled internal baseline, not a published claim, and exactly the kind of evidence a control can remove."Reworded as its own sentence on where the evidence came fromIntroduction, "Why this selector", last sentence
79–80"Neither the main study nor replication establishes an advantage for this policy in its prespecified comparison family."Reworded as a result: no clear advantage in either prespecified familyIntroduction, contribution 1
87–88"its valid JSON and absent replication fallbacks establish parser coverage, not diagnosis quality."Kept, moved inside the itemIntroduction, contribution 2

We also edited other passages for directness, and we kept statements that limit a claim on purpose.

The AI Use Statement describes four checks by the authors. We ran the reproduction supplement from its own extraction and confirmed that the packaged studies return the reported primary test and trial counts. We read the core analysis code, which pairs failures by length, asserts equal origin multisets in the two arms and evaluates the exact sign-flip null. We traced reported quantities to trial-level records rather than to the generated macros. Finally, we compared the rendered figures with their numbers and found three figures whose legends hid their labels.

The authors also directed the AI assistance, so these checks are not independent replication. They covered the recent audited experiments, and the statement marks the historical appendix and the pre-September results as unverified. In addition, two co-authors act as internal auditors. Both reviews were outstanding at submission. Partial reports arrived during the discussion period, and the revision corrects what they found while still reporting the audit as incomplete.

In the revision, the AI Use Statement also discloses AI help with the exploratory analyses and with this revision. The revision labels those analyses as exploratory and keeps the earlier results unchanged.

Edited on Nov 12 after partial internal audit reports: the audit status.

Reviewer_49rEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper evaluates whether token-level perplexity can guide trajectory repair in failed ReAct-style QA agents. Across multiple models, datasets, controls, budgets, and a length-matched swap study, the proposed highest-perplexity-plus-two-step-backtracking policy shows no clear advantage over restart or positional controls. The results provide a carefully audited negative finding, but are limited to this fixed policy and offline QA settings.

Critical strengths
  1. Relatively comprehensive experimental evaluation. The paper goes beyond a simple perplexity-versus-restart comparison by including positional controls and a length-matched swap design, as well as evaluations across multiple models, datasets, and recovery settings. This breadth provides useful evidence about the behavior of the specific policy studied.

  2. Transparent auditing and reproducibility reporting. The authors report replay, scoring, and budget audits together with detailed experimental configurations and local reproduction results. Although these checks do not resolve the study’s statistical limitations, they make the reported experiments relatively transparent and easier to inspect.

Critical weaknesses
  1. Insufficient statistical resolution for the central negative claim. In the length-matched swap study, 83 of 102 pairs are tied, leaving only 19 informative pairs; the paper also acknowledges that several cells could not reach conventional significance levels under the observed resolution. Consequently, the experiments establish that no clear advantage was detected, but do not convincingly rule out practically meaningful effects.

  2. Limited novelty and external validity. The study evaluates one fixed perplexity heuristic in offline QA, while the diagnosis baseline is untrained and the runtime experiments do not reflect realistic deployment conditions such as prefix caching. Thus, the negative findings have limited implications beyond this particular policy and experimental setting.

Questions for authors

See Weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 49rE - Part 1

We thank Reviewer 49rE for the careful review. The review notes that the positional controls and the length-matched swap go beyond a perplexity-versus-restart comparison, and it raises two concerns, the statistical resolution of the central negative claim and its novelty and external validity. This reply corrects which swap cells the tie count describes, reports the power and equivalence margins of the confirmatory cells, and sets out what was run and what remains future work. Pointers refer to sections of the revised PDF.

Response to Weaknesses and Questions

W1. Resolution of the swap study

We add one fact and agree with the conclusion. The 83 tied pairs out of 102 belong to the four earlier swap cells. The two later confirmatory cells on Qwen HotpotQA paired failures so that origins differ where possible, and repaired only those informative pairs. They have 51 discordant pairs at 72B and 40 at 32B. The 72B cell is complete and is the primary confirmatory result. The 32B cell is a declared deviation, because 31 of its 131 informative pairs were unrun when the frozen time limit ended the run. However, these cells still tie on 73 of 124 and 60 of 100 run pairs, so the reviewer's point largely stands.

Table R1. Every length-matched swap cell (source: Appendices I.7–I.9).

ModelDatasetPairsTiedDiscordant (+/−)Floor 2/2^mΔ (points)95% intervalp (Holm)Status
Qwen2.5-32BHotpotQA23194 (4/0)0.125+3.62[+0.72, +7.25]0.125 (0.250)Complete
Qwen2.5-32B2WikiMultiHopQA18126 (2/4)0.031−3.70[−11.11, +3.70]0.500 (0.500)Complete
Mistral-NemoHotpotQA33294 (3/1)0.125+1.01[−2.02, +4.04]0.750 (1.000)Complete
Mistral-Nemo2WikiMultiHopQA28235 (2/3)0.062−1.79[−6.55, +1.79]0.688 (1.000)Complete
Qwen2.5-32B, confirmatoryHotpotQA100 of 1316040 (22/18)2/2^40−0.17[−3.17, +2.84]1.000Declared deviation
Qwen2.5-72B, confirmatoryHotpotQA124 of 1247351 (30/21)2/2^51+1.48[−1.48, +4.44]0.397Complete

Δ is uncertainty minus swapped origin. The earlier cells form two prespecified two-test Holm families, one per model. The confirmatory cells have one exact two-sided sign-flip test at alpha 0.05, and their pair counts are the informative pairs that ran. The pairs that shared an origin, 93 at 32B and 99 at 72B, tie by construction and were counted and left unrepaired.

Pairing for differing origins moved both floors far below 0.05. The cells were sized on discordant pairs, with the 32B cohort fixed for power 0.80 at a discordant-pair win probability of 0.75. The 72B cell repeats that design and reports achieved power. The paper's power figures are for a sign-only test on the discordant pairs. The sign-flip test used for p also weighs each pair's success difference and has lower power: with the observed magnitudes held fixed, 0.89 and 0.69 at 72B and 0.78 and 0.57 at 32B.

Table R2. Power and equivalence margins of the confirmatory cells (source: the two confirmatory appendices and the exploratory appendix).

CellDiscordant pairsSign-only power at win probability 0.75At 0.70Margin above which two one-sided tests declare equivalenceSwap success (base rate)
72B510.960.844.03 points11.42%
32B400.900.702.83 points11.67%

The margins were computed after the outcomes were known, without a prespecified margin, so they are descriptive.

This power covers large effects. At a win probability of 0.60, the sign-only test has power 0.30 at 72B and 0.21 at 32B, and the weighted test 0.21 and 0.16. We computed the figures at 0.60 for this reply. The observed 72B split, 30 of 51, sits near that win probability. The 4.03-point margin is large compared with the 11.42% base rate, so effects a reader may consider practically meaningful remain possible. With exact positional balance, the tested policy shows no advantage of that large size, and the study does not establish equivalence. Mistral-Nemo and 2WikiMultiHopQA keep the earlier cells' resolution, since both confirmatory cells are Qwen HotpotQA.

In the revision, the Abstract and Conclusion report the complete 72B cell before the 32B cell, and "Replication across Models and Datasets" names it the primary confirmatory result. The Abstract and that section label the 32B cell a declared deviation, and the Conclusion calls it truncated. "Analysis and Discussion" adds "What the nulls exclude", and the appendix "Exploratory Headroom, Precision and Allowance Analyses" reports these margins as exploratory.

Edited on Nov 12 after partial internal audit reports: the power description and the wording of the margins.

AuthorsOpenReview ↗

Response to Reviewer 49rE - Part 2

W2. Scope beyond this policy and setting

We agree. The study tests one fixed rule, the highest-perplexity step backed up two steps, in offline multi-hop QA. The replication tested Qwen2.5-32B and Mistral-Nemo on HotpotQA, MuSiQue and 2WikiMultiHopQA with positional controls. The earlier swap covered both models on HotpotQA and 2WikiMultiHopQA, and MuSiQue has no swap cell. Qwen2.5-72B appears only in the HotpotQA confirmatory swap. The measured-time study used 25 failed Qwen2.5-32B HotpotQA questions, with prefix caching disabled and the step limits kept. The diagnosis baseline is untrained.

We left three things unrun, a trained diagnosis comparator, a runtime study with prefix caching, and the swap with entropy, action disagreement or any other signal. One new cell is running. It is a 72B swap at twice the token allowance and 16 new steps, under a protocol frozen before launch, with a prespecified equivalence margin of 3 points.

On the stored failed traces, we recomputed the origin that maximum token entropy and the probability complement would choose. Table R3 compares it with the perplexity origin. This analysis is exploratory, specified after the outcomes and absent from the paper.

Table R3. Agreement with the perplexity origin on failed traces (exploratory).

StudySignalSame originBoth restartSame where perplexity keeps a prefixSuccess on scored subset, signal vs perplexity (questions)
MainEntropy80/1135723/4328.9% vs 13.3% (15)
MainComplement83/1136023/4329.2% vs 4.2% (8)
72BEntropy294/44624054/15511.1% vs 12.0% (39)
72BComplement291/44623754/1558.9% vs 8.9% (41)

Most agreement is a shared restart. The other two signals choose the perplexity origin on 65% to 73% of failures. Where perplexity keeps a prefix, they choose its origin on 23 of 43 and 54 of 155 failures, or 35% to 53%. The scored subsets cover only differing origins that other arms had already run on every seed, so they are small and selected. The swap on another signal would often test different rewind points, so this question stays open.

In the revision, the future work in "What these limits imply for future work" names the swap on other signals, looser token and step limits, and a retry hint generated from the failed attempt. The scope statement in Limitations is unchanged. We will post the result of the looser-limit 72B run in this thread when it finishes.

Reviewer_TwjdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tests whether perplexity can tell a question-answering agent where to retry after a failed attempt. The agent finds the step where its own output was least predictable, backs up two steps, and tries again. On HotpotQA, this repaired 9.73% of failed attempts, compared with 11.21% for starting over. Using a question’s own rewind point also gave no clear advantage over borrowing one from another attempt of the same length.

Critical strengths
  • Interesting idea. I found the rewind-point swap useful. If two attempts have the same number of steps, they can exchange rewind points without changing how often the agent retries from each position. This tests whether perplexity picks a useful step for the particular question.

  • On 70 of the 113 failed HotpotQA questions, the perplexity-guided rule chose the first step as the retry point, just as a full restart would. Reporting this helps readers see that only remaining 43 questions tested whether keeping earlier steps helped.

Critical weaknesses
  • Evaluation is limited. The controlled experiments give the agent only a generic retry hint, and it does not see the failed steps after the rewind point. Every policy repairs only about 10% of failures (Table 1), so it is unclear whether perplexity picks poor rewind points or whether no rewind point would help much in this setup.

  • The paper tests only one fixed rule, the highest-perplexity step backed up two steps, and the backup alone turns 62% of retries into full restarts. Other uncertainty measures, such as entropy, appear only in the older uncontrolled runs (Section 4.1), so it is unclear whether the null result reflects uncertainty signals in general or only this one rule.

Questions for authors
  1. If the agent generated a short hint from its failed attempt without seeing the reference answer, would its own rewind point work better than a borrowed one when both retries use that same hint?

  2. On questions where the methods picked different rewind points, did some steps consistently give better answers across retries? This would help clarify whether perplexity missed a better place to retry or whether the choice of step made little difference.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Twjd - Part 1

We thank Reviewer Twjd for the careful review and the two concrete questions.

The review notes that the rewind-point swap tests whether perplexity picks a useful step for the particular question, and that 70 of the 113 failed questions rewound to the first step. In reply, we added a cross-fitted choice among tested rewind points, a test of whether rewind points are consistent across seeds, and a count of how often two other uncertainty signals pick the same rewind point as perplexity. The hint the agent writes from its failed attempt was not run, and the revision names it as future work.

Response to Weaknesses and Questions

W1 and Q2. The roughly 10% floor and consistent rewind points

We partly agree. The setup limits all policies, and a better choice among tested rewind points suggests a gain of a few points. The analyses below are exploratory and were specified after all outcomes were known. Their p-values are unadjusted unless marked Holm.

All conditions receive the same generic retry hint. The agent sees the question, the kept steps and that hint, and the failed steps after the rewind point stay out of its prompt. The method section "Interventions and Information Access" states both.

To estimate what a better choice would add, we used the three seeds on which the failed questions were retried. For a question and a held-out seed, we chose the rewind point with the highest success on the other two seeds and scored it on the held-out seed. This is a cross-fitted reference limited to tested rewind points, not a bound on a true oracle.

Table R1. Cross-fitted choice among tested rewind points, 95% bootstrap intervals (source: Appendix G).

CellQuestionsChosen (%)ComparatorComparator (%)Difference, points [95% interval]
Main11313.27restart11.21+2.06 [−1.18, +5.90]
Main11313.27rule9.73+3.54 [0.00, +7.67]
Diagnosis 1×9113.92restart11.36+2.56 [−0.37, +6.23]
Diagnosis 1×9113.92rule10.62+3.30 [−1.10, +8.06]
72B swap24814.78rule12.90+1.88 [−0.13, +4.03]
72B swap24814.78swap11.42+3.36 [+0.94, +5.91]
32B swap20014.00rule11.50+2.50 [0.00, +5.33]
32B swap20014.00swap11.67+2.33 [+0.17, +4.67]

Main is the main study, Diagnosis 1× the diagnosis follow-up at the 1× allowance, and 72B and 32B swap the confirmatory swaps. "Rule" is the perplexity rule, and "swap" a borrowed rewind point.

The chosen rewind point exceeds the rule by 1.88 to 3.54 points. The intervals include or touch zero in six of the eight comparisons, and the two that exclude zero compare with the swap arm. These results suggest that a better choice among tested rewind points would still leave most failures unrepaired. They cannot tell how much an untested rewind point or a richer hint would add.

We also tested whether a rewind point that succeeds on one seed of a question tends to succeed on another seed. The null permutes rewind-point labels within question and seed, with 2,000 permutations.

Table R2. Consistency of rewind points across seeds, one-sided permutation p, and questions with any variation (in brackets, on at least two seeds). Source: same appendix, except the any-variation counts, computed for this reply.

CellAll rewind points: questionspHolm pRewind points that keep a prefix: questionsp
Main41 of 113 (19)0.0040.01618 of 60 (8)0.109
Diagnosis 1×27 of 91 (12)0.0070.0228 of 22 (2)0.077
72B swap72 of 248 (35)0.0290.05921 of 62 (11)0.475
32B swap59 of 200 (17)0.0370.05925 of 98 (5)0.978

The statistic draws only on questions with variation on at least two seeds, which are the counts in brackets. Over all rewind points, the test suggests that rewind points are consistent across seeds. However, it cannot say whether this comes mostly from restarting or from keeping a prefix. Among rewind points that keep a prefix, the tests rest on 8, 2, 11 and 5 questions, too few to tell whether consistency differs among kept rewind points.

In the revision, the appendix "Exploratory Headroom, Precision and Allowance Analyses" reports both analyses, and the Analysis and Discussion paragraph "Does origin choice matter in this setting?" summarizes them. The method section states the generic hint and the hidden failed steps, and the Limitations section names a generated hint as future work.

Edited on Nov 12 after partial internal audit reports: two p-values in Table R2 and the description of the cross-fitted reference.

AuthorsOpenReview ↗

Response to Reviewer Twjd - Part 2

W2. Other uncertainty signals and the fixed rule

We agree. The swap tested one rule, the highest-perplexity step backed up two steps, and the conclusion holds only for that rule.

The two-step backup turns many nominal targeted retries into restarts. In the main study, the rule restarts from the first step in 210 of 339 retries (61.95%). Without the backup, perplexity restarts in 15.04% of retries.

The failed attempts store token probabilities, so we recomputed the rewind point that maximum token entropy and the probability complement would choose under the same rule. This analysis is exploratory and descriptive only, and it was specified after all outcomes were known.

Table R3. Restarts under the rule, and agreement of other signals with the perplexity rewind point, 95% Wilson intervals (sources: Results paragraph "How often backtracking becomes restart" for the first two rows; the rest computed for this reply from the stored failed attempts and trial exports, not in the paper or the current supplement).

QuantityMain study72B swap
Rule restarts from the first step, retries210/339 (61.95%)—
Same, without the backup, retries15.04%—
Rule restarts from the first step, failed attempts70/113 (61.95%)291/446 (65.25%)
Entropy picks the perplexity rewind point80/113 (70.80%) [61.84, 78.39]294/446 (65.92%) [61.40, 70.17]
Of these, both restart from the first step57240
Probability complement picks the perplexity rewind point83/113 (73.45%) [64.64, 80.72]291/446 (65.25%) [60.71, 69.52]
Of these, both restart from the first step60237
Perplexity keeps a prefix, failed attempts43/113155/446
Entropy picks the same kept rewind point23/4354/155
Probability complement picks the same kept rewind point23/4354/155

A dash marks a retry-level figure that the paper gives for the main study only.

Most of the agreement in Table R3 is shared restarts. Of the 80 main-study attempts where entropy matches perplexity, 57 restart from the first step under both signals, and the probability complement and the 72B swap show the same pattern. Where perplexity keeps a prefix, entropy and the probability complement pick the same rewind point on 23 of 43 main-study attempts and 54 of 155 at 72B. The other signals therefore often pick a different point where a prefix is kept, and whether uncertainty in general helps remains open. The swap with entropy, action disagreement or any other signal was not run.

Some rewind points that the other signals would choose had already been run by other arms on all seeds. On those questions, the entropy points succeed on 28.89% compared with 13.33% for the perplexity points (15 questions), and the probability complement points on 29.17% compared with 4.17% (8 questions). At 72B the rates are 11.11% compared with 11.97% (39 questions) and 8.94% compared with 8.94% (41 questions). These subsets are small and selected, so they give no estimate of a policy on those signals.

In the revision, the Limitations paragraph "What these limits imply for future work" names the swap on other signals, such as entropy and action disagreement, and looser token and step limits. The agreement counts in Table R3 and the success rates above are reported in this reply only.

AuthorsOpenReview ↗

Response to Reviewer Twjd - Part 3

Q1. A hint the agent writes from its failed attempt

We did not run this experiment, so we have no result for it. One new run fits the discussion period, and we gave it to a 72B swap at twice the token allowance and 16 new steps. That run tests whether the tight allowance holds all policies near 10%. The hint experiment would be a good test of whether the rewind point matters when the retry can use what the failed attempt revealed.

In the revision, the same Limitations paragraph names this as future work, as a retry hint generated from the failed attempt, without the reference answer and shared by both arms.

The 72B run with the looser limits is running now, and we will post its result in this thread when it finishes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.