acceptodds
Under review as a conference paper at ICLR 2027

ORDDAR: OBSERVATION- DRIVEN REASONING FOR DISTORTION- RESILIENT DECISION, ACTION, AND COGNITIVE RECOVERY

Abstract

AI agents increasingly perform long-term reasoning, planning, tool use, memory integration, and autonomous decision making, yet erroneous intermediate states can propagate and cause inconsistent decisions and unreliable outputs. Existing reasoning approaches mainly rely on iterative planning, self-reflection, augmented memory, or verification, but rarely localize and selectively repair faulty reasoning. We present ORDDAR (Observation-Driven Reasoning for Distortion-Resilient Decision, Action, and Cognitive Recovery), a reasoning framework that models reasoning as cognitive state transitions, detects localized distortions, retrieves related reasoning from prior experiences, and repairs only the affected states. ORDDAR therefore performs recovery at the local reasoning-transition level rather than regenerating the complete trajectory. Experiments across mathematical, commonsense, multi-hop, and clinical reasoning benchmarks demonstrate improved reasoning quality, recovery ability, and interpretability over multiple evaluated reasoning baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Trustworthy AI, Retrieval-Augmented Reasoning, Large Language Models, Cognitive Recovery, Reasoning Distortion Detection, Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_neAnOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ORDDAR, which treats an agentic reasoning trajectory as a sequence of cognitive states and attempts to repair only the transitions that are locally corrupted. Consecutive states are embedded and combined with a second "cognitive-anchor" distance to form a joint distortion score; transitions above a trajectory-adaptive mean threshold are flagged, each flagged transition is encoded as a signature of semantic shift and gained and lost anchors, and a signature is used to retrieve a recovery complement from a memory of validated trajectories. Only the flagged state is replaced and the rest of the trajectory is preserved. Evaluation uses 100 instances each from GSM8K, HotpotQA, StrategyQA, and a private clinical QA set built from hospital cases, with Qwen2.5-7B-Instruct against CoT, Self-Refine, Reflexion, Tree of Thoughts, and ProCo, plus encoder-choice, memory-component, and weight ablations.

Critical strengths

Localizing a fault and repairing only that state, rather than regenerating the trajectory, is a reasonable and well-motivated alternative to the regenerate-everything pattern that dominates self-correction work, and the paired encoder and memory-component ablations are the right kind of check to run on such a pipeline. The recovery rate on GSM8K, where removing the distortion memory costs seven points, indicates that memory is doing real work on at least one benchmark.

Critical weaknesses

The evaluation cannot support the claims. One hundred instances per benchmark at a single seed is far too small to separate the reported margins from noise, only one backbone is tested despite the detector depending entirely on the LLM's own reasoning, and the baselines are 2023-era prompting methods with missing cells in the main table. More fundamentally, the detector that performs the actual work reaches a joint F1 of only 0.62 to 0.74 against manually labelled distortions, so a substantial share of flagged transitions are false positives, and the claim that the framework localizes corruption is not supported at that level of detector reliability. Second, the headline clinical number is neither verifiable nor apparently like-for-like. CKDQA is private and unreleased, the recovery complements are described as incorporating locally supplied KDIGO knowledge, and the baselines have no access to that knowledge base, so the jump from 25 to 67 percent for baselines up to 94.38 percent should be expected regardless of the method. The gain is also smallest exactly where the framing predicts it should be largest, on the multi-hop benchmark where error propagation is most damaging, where the improvement over chain-of-thought is four points. Finally, the cognitive-anchor distribution, the gained and lost anchor sets, and the recovery operator are never formally defined, so the method cannot be reproduced or evaluated on its own terms.

Questions for authors

Can CKDQA be released, or can a public clinical substitute be provided so the main result can be checked? How often does recovery actually fire, on which benchmarks, and what fraction of the gain comes from retrieval hits rather than from the heuristic? Please give formal definitions of the cognitive-anchor distribution, the gained and lost anchor sets, and the recovery operator. What happens on full test splits with multiple seeds, and across several backbones? How do the results compare with recent self-correction and self-verification work and with retrieval-augmented baselines, given that the method is a retrieval system over a trajectory memory? Given a detector F1 of roughly 0.65, is there an ablation showing that the benefit survives when the detector is restricted to high-precision flags? Finally, how should readers interpret the CKDQA result given that the recovery complements embed KDIGO knowledge that the baselines do not receive?

2 comments

We sincerely thank Reviewer 3 for the detailed and constructive feedback. We have revised the manuscript and conducted additional analyses to address the concerns regarding methodological clarity, recovery reliability, evaluation scope, and clinical knowledge.

  1. CKDQA Dataset Availability and Reproducibility: We have created a public GitHub repository to support reproducibility, containing the implementation code, evaluation scripts, and supporting documentation.

GitHub Repository: https://github.com/deblinacode/CKDQA

The release of clinical question–answer pairs and CKD-stage annotations remains subject to institutional ethics approval, hospital data-sharing authorization, and applicable privacy requirements. We will provide an appropriate controlled-access mechanism or a public clinical substitute if unrestricted release is not permitted.

  1. Distortion Detection and Recovery Analysis: We conducted a supplementary analysis on 20 CKDQA test trajectories containing 255 transitions. At a threshold of 0.30, 223 transitions were flagged, and 203 yielded compatible validated memory retrievals (91.03% retrieval availability among flagged transitions). We emphasize that retrieval availability does not establish independently verified successful correction. We also report threshold sensitivity analysis and acknowledge that false-positive degradation requires further controlled evaluation.

  2. Formal Methodological Definitions: We have clarified the 12-dimensional cognitive-anchor distribution using pretrained MPNet embeddings, formally defined the gained and lost anchor sets, and specified the localized recovery operator. The formulation explains how only the selected reasoning state is modified while unaffected states are preserved.

  3. Evaluation Across Seeds and Models: We extended the evaluation using Meta-Llama-3.1-8B-Instruct across three random seeds (42, 123, and 2024). We also increased the GSM8K evaluation subset from 100 to 160 instances and examined three alternative sentence encoders. We acknowledge that complete test-split evaluation and broader backbone comparisons remain future work.

  4. Related Work and RAG Distinction: We expanded the Related Work discussion to include GLoRe, Stepwise Correction, REDE, ARS, and verifier-based approaches. We also clarified the objective difference between RAG and ORDDAR: RAG retrieves contextual knowledge to augment generation, whereas ORDDAR uses Shift–Gain–Loss signatures to localize potentially distorted reasoning transitions and retrieve intermediate bridge states for selective recovery.

  5. Detector Reliability: We report joint distortion detection F1-scores of 0.654, 0.664, 0.622, and 0.743 across GSM8K, HotpotQA, StrategyQA, and Clinical-CKD, respectively. We clarify that detection identifies candidate transitions rather than automatically triggering correction. A dedicated high-precision-only detector ablation has not been conducted, and we do not make a quantitative claim about its effect.

  6. CKDQA and KDIGO Knowledge: We conducted a prompt-level ablation, obtaining 94.38% accuracy with explicit KDIGO threshold instructions and 93.00% without them. Component-wise ablations further examine the contributions of the Cognitive Mirror, Distortion Module, and Recovery Module. Nevertheless, we acknowledge that clinical knowledge may also be embedded in validated recovery complements, and its contribution cannot be completely separated from memory-guided recovery in the current experiments.

We sincerely appreciate the reviewer’s feedback, which helped us improve the methodological presentation, strengthen the experimental analysis, and provide a more transparent account of the framework's limitations.

Additional Mathematical Clarification and Algorithm 1: Cognitive Mirror Construction, Trace-Guided Retrieval, and Localized Recovery

To clarify the mathematical formulation and practical execution of ORDDAR, we provide the formal definitions, algorithmic workflow, and an illustrative CKDQA before-and-after recovery example.

  1. Cognitive-Anchor Distribution

For a reasoning state s_t, the normalized weight of cognitive anchor j is calculated as:

psi_t_j = exp(sim(E(s_t), E(a_j))) / sum_{l=1}^{12} exp(sim(E(s_t), E(a_l)))

where E(.) denotes the pretrained MPNet encoder, a_j represents the textual description of the j-th cognitive anchor, and sim(.) denotes cosine similarity.

The framework uses 12 predefined cognitive anchors: goal, observation, reasoning, logic, calculation, comparison, validation, memory, constraint, decision, action, and conclusion.

  1. Gained and Lost Anchor Sets

The gained and lost anchor sets between two consecutive reasoning states are defined as:

A_gain(t) = {a_j : psi_(t+1)_j > psi_t_j}

A_loss(t) = {a_j : psi_(t+1)_j < psi_t_j}

These sets identify the cognitive anchors whose normalized weights increase or decrease between consecutive reasoning states. Anchors with unchanged weights are excluded from both sets. Together with semantic displacement, these sets constitute the Shift-Gain-Loss (SGL) distortion signature.

  1. Cognitive Mirror Representation

Cognitive_Mirror = {(k_i, c_i) : i = 1, ..., N}

where k_i represents the SGL distortion signature and c_i represents its associated validated intermediate bridge state, which serves as the recovery complement.

  1. Algorithm 1: Cognitive Mirror Construction, Trace-Guided Retrieval, and Localized Recovery

Require: Validated trajectories D, query q, trajectory-adaptive thresholding rule, and retrieval weights alpha_1, alpha_2, alpha_3.

Ensure: Final prediction y_hat.

Memory Construction

  1. Initialize Cognitive Mirror M = empty set.

  2. For each validated trajectory R_j = {s_1, ..., s_T} in D:

  3. For each consecutive reasoning-state pair:

    W_t = {s_t, s_(t+1)}.

    Compute the joint distortion score D_t.

  4. If D_t > tau_Rj:

    a. Construct the SGL distortion signature:

    k_t = {psi_shift, A_gain, A_loss}.

    b. Associate the validated bridge state c_t with the distortion signature k_t.

    c. Store (k_t, c_t) in M.

  5. End the inner and outer loops.

Runtime Detection and Recovery

  1. Generate the reasoning trajectory R = {s_1, ..., s_T} from query q.

  2. For each consecutive reasoning-state pair:

    W_t = {s_t, s_(t+1)}.

    Compute the joint distortion score D_t.

  3. If D_t > tau_R:

    a. Construct the SGL distortion signature:

    k_t = {psi_shift, A_gain, A_loss}.

    b. Compute the weighted similarity:

    S(k_t, k_i) = alpha_1 * S_shift + alpha_2 * S_gain + alpha_3 * S_loss.

    c. Retrieve the complement c_star associated with the most compatible distortion signature.

  4. If a compatible, validated trace-complement pair exists:

    a. Apply complement-guided recovery:

    s_hat_t = R(s_t, c_star).

    b. Replace only the affected reasoning state:

    R_hat = {s_1, ..., s_hat_t, ..., s_T}.

    c. Preserve all unaffected reasoning states.

  5. Otherwise, preserve the original reasoning state without modification.

  6. End the runtime loop.

Illustrative Before-and-After Recovery (CKDQA)

  1. Before recovery:

Serum creatinine = 3.6 mg/dL.

eGFR = 16.6 mL/min/1.73m2.

Initial CKD stage prediction = G3.

  1. Distortion detection and retrieval:

Detected distortion score = 0.874.

Retrieved complement similarity = 0.940.

  1. After localized recovery: The affected reasoning state is revised using the retrieved validated bridge state. Corrected CKD stage prediction = G4. All unaffected reasoning states remain unchanged. Validated Memory Update for Future Queries

  2. If the recovery is validated and memory updating is permitted:

M = M union {(k_t, c_star)}.

  1. Generate the final prediction:

y_hat = P(R_hat).

  1. Return y_hat.

End Algorithm 1.

  1. Localized Recovery Operator The recovery operation can be expressed using plain-text notation as follows: If a compatible and validated recovery complement exists:

Recovered_State(t) = Recovery_Operator(Original_State(t), Best_Complement)

Otherwise: Recovered_State(t) = Original_State(t) Thus, only the identified reasoning state is modified, while unaffected reasoning states are preserved. If no compatible, validated complement is available, the original state is retained. The pretrained MPNet encoder remains frozen, and no additional trainable recovery network is introduced. The retrieval weights (0.6, 0.2, 0.2) are selected through sensitivity analysis. The algorithm describes trajectory-level distortion detection followed by selective state-level recovery. The CKDQA example illustrates the operation of the recovery mechanism; it does not imply a formal guarantee that every retrieved complement produces a logically correct correction.

Reviewer_N7YiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes ORDDAR, an observation-driven reasoning framework designed to identify and repair localized distortions in multi-step reasoning trajectories. Instead of regenerating an entire trajectory, ORDDAR detects anomalous cognitive state transitions using semantic displacement and cognitive anchors, represents the detected distortion through a Shift–Gain–Loss signature, retrieves a corresponding recovery complement from a Cognitive Mirror memory, and applies a localized recovery operator.

The authors evaluate ORDDAR using Qwen2.5-7B-Instruct on GSM8K, HotpotQA, StrategyQA, and a private clinical question-answering dataset (CKDQA). The reported results show improvements over several reasoning and self-correction baselines, while component ablations suggest that distortion detection, memory retrieval, and localized recovery each contribute to the overall performance.

Critical strengths

The paper proposes identifying and repairing faulty reasoning transitions rather than regenerating entire reasoning trajectories. This is a meaningful direction for improving reasoning reliability, particularly for long reasoning processes where restarting from scratch can be computationally expensive. The distinction between distortion localization and subsequent recovery is conceptually appealing.

The combination of the Shift–Gain–Loss distortion signature and the Cognitive Mirror memory provides a structured mechanism for associating reasoning failures with potentially reusable recovery information. This formulation is promising because it could enable corrections to be transferred across similar failure patterns rather than treating each failure independently.

The paper evaluates the proposed framework across mathematical reasoning, multi-hop question answering, commonsense reasoning, and clinical QA. The component ablations also provide preliminary evidence that different parts of the framework contribute to its performance. Although the experimental scale is limited, this evaluation offers an initial indication of the framework's applicability across different reasoning tasks.

Critical weaknesses

Several essential components of ORDDAR are not adequately defined, including the construction of cognitive anchors, the content and generation of recovery complements, and the implementation of the recovery operator. These components are central to the proposed framework, yet the paper does not provide sufficient algorithmic details or concrete examples to establish how they operate across different tasks. Moreover, the stated optimization objective is not clearly connected to any training procedure or parameter updates. These omissions make it difficult to reproduce the method and assess which components are responsible for its reported improvements.

The evaluation uses only 100 sampled examples per benchmark, apparently with a single random seed. Given the relatively small sample size, the reported improvements may be sensitive to sampling variation, particularly when differences between methods are modest. The absence of confidence intervals, repeated trials, or statistical significance tests makes it difficult to determine whether the improvements are robust. Evaluations on complete benchmark splits or multiple independent samples would provide substantially stronger evidence.

The experimental comparison does not adequately establish whether ORDDAR outperforms competitive reasoning-correction methods under appropriate settings. In particular, applying greedy decoding uniformly may disadvantage search-based or iterative reasoning baselines such as Tree of Thoughts and Reflexion, whose effectiveness often depends on their sampling and exploration configurations. Some baseline results are also missing. Furthermore, direct comparisons with more closely related step-level detection, denoising, and verification-based correction methods are absent. Consequently, it remains unclear whether the reported gains result from the proposed localized recovery mechanism or from differences in experimental configurations.

The distortion detection mechanism relies on a relatively simple threshold defined using trajectory-level displacement statistics. Its theoretical or empirical justification is limited, and the reported detection precision suggests a substantial number of false positives. Since erroneous detection can trigger unnecessary corrections, the reliability of this component directly affects the validity of the overall framework. Furthermore, the construction and annotation protocol of the ground-truth distortion labels are insufficiently documented. The paper also lacks a systematic analysis of whether detected distortions are successfully repaired and whether false-positive interventions damage otherwise correct reasoning.

The paper does not adequately specify how the Cognitive Mirror is initialized, how many validated trajectories or recovery complements it contains, or how its contents are separated from evaluation examples. Since the framework relies on retrieving stored recovery information, its performance may depend heavily on memory coverage and task-specific knowledge. Without controlled experiments varying memory size, evaluating previously unseen tasks, or testing cross-task transfer, it is difficult to establish whether ORDDAR learns reusable correction patterns or primarily benefits from task-specific stored examples. This concern is particularly relevant to the private CKDQA dataset, where recovery complements may encode domain-specific clinical guidelines.

Questions for authors

How are cognitive anchors extracted for different reasoning tasks? What is the exact representation of a recovery complement, and how does the recovery operator modify a corrupted reasoning state? Please provide concrete algorithms and before-and-after examples. Additionally, which components, if any, are optimized using the stated training objective?

How were the decoding budgets, sampling strategies, and computational costs matched across baselines, particularly ToT and Reflexion? Could the authors compare ORDDAR against more directly related step-level detection and correction approaches, such as REDE, ARS, or verifier-based methods, under comparable settings?

How were the ground-truth distortion labels obtained and validated? What proportion of detected distortions leads to successful corrections, and how frequently do false-positive detections degrade previously correct trajectories? Additionally, is the trajectory-level distortion threshold computed using the entire trajectory, and if so, how is this compatible with online localized recovery?

How is the Cognitive Mirror constructed, and what are its memory size, data sources, and update rules? How is overlap between memory construction data and evaluation data prevented? Could the authors report performance under different memory sizes and evaluate recovery on unseen tasks without task-specific memory initialization?

In the CKDQA experiments, do recovery complements explicitly encode clinical guidelines, such as KDIGO staging rules? If so, how much of the improvement comes from the proposed recovery mechanism rather than access to additional domain-specific knowledge? Could the authors provide a controlled comparison using the same guidelines for the baseline methods?

4 comments

We sincerely thank the Reviewer for the detailed and constructive feedback. We have revised the manuscript to improve methodological clarity, provide additional experimental analyses, and explicitly acknowledge the limitations of the current evaluation. Our responses to the major concerns are summarized below.

  1. Methodological clarification: In Section 3.1, we have clarified the extraction of 12 shared cognitive anchors using pretrained MPNet embeddings, cosine similarity, and softmax normalization to obtain a 12-dimensional cognitive-anchor distribution. In Section 3.3, we have detailed the Cognitive Mirror representation, where each memory entry consists of a Shift–Gain–Loss distortion signature and its corresponding intermediate bridge state (recovery complement). In Section 3.5, we have further explained the localized recovery mechanism, including weighted similarity-based complement retrieval, modification of the identified distorted state, and preservation of unaffected reasoning states. We have also expanded Algorithm 1 and included a before-and-after CKDQA example illustrating the recovery process. In Section 3.6, we have clarified the role of the recovery formulation and optimization procedure, explaining that MPNet remains frozen, no additional trainable recovery network is introduced, and the retrieval weights (0.6, 0.2, 0.2) are selected through sensitivity analysis.

  2. Baseline fairness and related work: We clarified the common generation settings, including the underlying model, tokenizer, chat template, greedy decoding, and maximum generation length. We acknowledge that identical per-generation settings do not imply identical total inference costs for iterative methods. We also expanded the Related Work discussion to include GLoRe, Stepwise Correction, REDE, ARS, and verifier-based approaches. We have also clarified the distinction between conventional Retrieval-Augmented Generation (RAG) and ORDDAR in terms of their objectives. While RAG retrieves relevant external or stored knowledge and incorporates it into the generation process to improve response relevance and factual grounding, ORDDAR focuses on identifying potentially distorted intermediate reasoning transitions through Shift–Gain–Loss signatures and retrieving compatible, validated bridge states from the Cognitive Mirror. The recovery operator selectively modifies the identified distorted state while preserving unaffected reasoning states. Thus, RAG primarily aims at knowledge augmentation during generation, whereas ORDDAR aims at transition-aware distortion localization and selective recovery of intermediate reasoning states.

  3. Distortion detection and recovery reliability: We have clarified the construction of ground-truth distortion labels using benchmark-specific references and controlled perturbations, with clinical transitions assessed against doctor-verified ground truth. We also clarified that the trajectory-adaptive threshold is computed after trajectory generation, followed by transition-level examination and selective recovery. Our supplementary CKDQA analysis reports compatible retrieval availability, rather than treating every retrieval as a verified successful correction. We acknowledge that independent verification of correction success and false-positive degradation requires further controlled evaluation.

  4. Cognitive Mirror construction and generalization: We clarified the memory representation, initialization, and retrieval procedure. Each benchmark maintains an independently initialized Cognitive Mirror. For GSM8K, the implementation uses 50 seed trajectories sampled from the training split. We distinguish the number of seed trajectories from the total number of stored memory entries. We have not conducted a controlled memory-capacity scaling study or cross-task transfer evaluation; these limitations are now explicitly acknowledged.

  5. CKDQA and KDIGO knowledge: We conducted a prompt-level ablation using identical CKDQA test instances, obtaining 94.38% accuracy with explicit KDIGO threshold instructions and 93.00% without them. Component-wise ablations further examine the contributions of the Cognitive Mirror, Distortion Module, and Recovery Module. Nevertheless, we acknowledge that clinical knowledge may also be embedded in validated recovery complements, and its contribution cannot be completely separated from memory-guided recovery in the current experiments.

We appreciate the reviewer’s comments, which helped us improve the methodological presentation, strengthen the experimental analysis, and provide a more transparent discussion of the framework’s limitations.

Additional Mathematical Clarification and Algorithm 1: Cognitive Mirror Construction, Trace-Guided Retrieval, and Localized Recovery

To clarify the mathematical formulation and practical execution of ORDDAR, we provide the formal definitions, algorithmic workflow, and an illustrative CKDQA before-and-after recovery example.

  1. Cognitive-Anchor Distribution

For a reasoning state s_t, the normalized weight of cognitive anchor j is calculated as:

psi_t_j = exp(sim(E(s_t), E(a_j))) / sum_{l=1}^{12} exp(sim(E(s_t), E(a_l)))

where E(.) denotes the pretrained MPNet encoder, a_j represents the textual description of the j-th cognitive anchor, and sim(.) denotes cosine similarity.

The framework uses 12 predefined cognitive anchors: goal, observation, reasoning, logic, calculation, comparison, validation, memory, constraint, decision, action, and conclusion.

  1. Gained and Lost Anchor Sets

The gained and lost anchor sets between two consecutive reasoning states are defined as:

A_gain(t) = {a_j : psi_(t+1)_j > psi_t_j}

A_loss(t) = {a_j : psi_(t+1)_j < psi_t_j}

These sets identify the cognitive anchors whose normalized weights increase or decrease between consecutive reasoning states. Anchors with unchanged weights are excluded from both sets. Together with semantic displacement, these sets constitute the Shift-Gain-Loss (SGL) distortion signature.

  1. Cognitive Mirror Representation

Cognitive_Mirror = {(k_i, c_i) : i = 1, ..., N}

where k_i represents the SGL distortion signature and c_i represents its associated validated intermediate bridge state, which serves as the recovery complement.

  1. Algorithm 1: Cognitive Mirror Construction, Trace-Guided Retrieval, and Localized Recovery

Require: Validated trajectories D, query q, trajectory-adaptive thresholding rule, and retrieval weights alpha_1, alpha_2, alpha_3.

Ensure: Final prediction y_hat.

Memory Construction

  1. Initialize Cognitive Mirror M = empty set.

  2. For each validated trajectory R_j = {s_1, ..., s_T} in D:

  3. For each consecutive reasoning-state pair:

    W_t = {s_t, s_(t+1)}.

    Compute the joint distortion score D_t.

  4. If D_t > tau_Rj:

    a. Construct the SGL distortion signature:

    k_t = {psi_shift, A_gain, A_loss}.

    b. Associate the validated bridge state c_t with the distortion signature k_t.

    c. Store (k_t, c_t) in M.

  5. End the inner and outer loops.

Runtime Detection and Recovery

  1. Generate the reasoning trajectory R = {s_1, ..., s_T} from query q.

  2. For each consecutive reasoning-state pair:

    W_t = {s_t, s_(t+1)}.

    Compute the joint distortion score D_t.

  3. If D_t > tau_R:

    a. Construct the SGL distortion signature:

    k_t = {psi_shift, A_gain, A_loss}.

    b. Compute the weighted similarity:

    S(k_t, k_i) = alpha_1 * S_shift + alpha_2 * S_gain + alpha_3 * S_loss.

    c. Retrieve the complement c_star associated with the most compatible distortion signature.

  4. If a compatible, validated trace-complement pair exists:

    a. Apply complement-guided recovery:

    s_hat_t = R(s_t, c_star).

    b. Replace only the affected reasoning state:

    R_hat = {s_1, ..., s_hat_t, ..., s_T}.

    c. Preserve all unaffected reasoning states.

  5. Otherwise, preserve the original reasoning state without modification.

  6. End the runtime loop.

Illustrative Before-and-After Recovery (CKDQA)

  1. Before recovery:

Serum creatinine = 3.6 mg/dL.

eGFR = 16.6 mL/min/1.73m2.

Initial CKD stage prediction = G3.

  1. Distortion detection and retrieval:

Detected distortion score = 0.874.

Retrieved complement similarity = 0.940.

  1. After localized recovery: The affected reasoning state is revised using the retrieved validated bridge state. Corrected CKD stage prediction = G4. All unaffected reasoning states remain unchanged. Validated Memory Update for Future Queries

  2. If the recovery is validated and memory updating is permitted:

M = M union {(k_t, c_star)}.

  1. Generate the final prediction:

y_hat = P(R_hat).

  1. Return y_hat.

End Algorithm 1.

  1. Localized Recovery Operator The recovery operation can be expressed using plain-text notation as follows: If a compatible and validated recovery complement exists:

Recovered_State(t) = Recovery_Operator(Original_State(t), Best_Complement)

Otherwise: Recovered_State(t) = Original_State(t) Thus, only the identified reasoning state is modified, while unaffected reasoning states are preserved. If no compatible, validated complement is available, the original state is retained. The pretrained MPNet encoder remains frozen, and no additional trainable recovery network is introduced. The retrieval weights (0.6, 0.2, 0.2) are selected through sensitivity analysis. The algorithm describes trajectory-level distortion detection followed by selective state-level recovery. The CKDQA example illustrates the operation of the recovery mechanism; it does not imply a formal guarantee that every retrieved complement produces a logically correct correction.

Thank you for the response and additional clarifications. However, my main concerns remain largely unresolved.

  1. The core recovery mechanism is still insufficiently specified. Writing the recovery operation as R(s_t, c*) or Recovery_Operator(Original_State, Best_Complement) does not explain how it is actually implemented. Similarly, Algorithm 1 assumes that a validated bridge state is available without explaining how this state is generated and validated. These are essential components of the proposed method rather than minor implementation details. The CKDQA example only reports the initial and corrected predictions, without showing the original reasoning state, the retrieved complement, or the recovered reasoning state.

  2. The additional mathematical definitions raise further concerns about distortion detection and recovery. Changes in semantic similarity and cognitive-anchor distributions do not necessarily indicate reasoning errors, as such changes can naturally occur during correct reasoning. Moreover, replacing an intermediate reasoning state while preserving all subsequent states may introduce logical inconsistencies when later steps depend on the original state. The response does not provide sufficient evidence that these issues are effectively addressed.

  3. Most of my experimental concerns remain unresolved. The response does not provide additional results addressing the limited evaluation size, statistical robustness, matched inference budgets, or comparisons with closely related correction methods. The authors also acknowledge that correction success, false-positive degradation, and cross-task memory generalization have not been independently evaluated.

While I appreciate the authors' efforts to clarify the presentation and acknowledge these limitations, the response does not sufficiently establish the reproducibility or effectiveness of the core proposed mechanism. Therefore, I maintain my original rating.

We sincerely thank the reviewer for taking the time to carefully examine our revised responses and for providing such detailed and constructive feedback. We genuinely appreciate the reviewer's continued engagement with our work and the effort invested in identifying the remaining methodological and experimental concerns. These comments are valuable in helping us better understand the expectations for a rigorous and reproducible evaluation of the proposed framework.

  1. Recovery mechanism and bridge-state construction:

The proposed recovery mechanism is based on retrieving a compatible validated complement from the Cognitive Mirror using the distortion signature of the affected reasoning transition. The signature captures semantic displacement and changes in cognitive-anchor activation, while the retrieved complement provides a reference for localized recovery. The recovery operation is therefore not an unconstrained replacement of the entire reasoning trajectory. It operates on the identified affected state, while preserving the original trajectory when no compatible complement is retrieved. The validated bridge states are associated with the corresponding distortion signatures during Cognitive Mirror construction. We acknowledge that the current presentation could provide a more explicit step-by-step implementation trace, particularly showing the original state, retrieved complement, and recovered state in the CKDQA example.

  1. Distortion detection and downstream consistency:

Our framework does not treat every semantic change or anchor-distribution shift as a confirmed reasoning error. These measurements are used jointly as trajectory-adaptive indicators for identifying potentially distorted transitions. The retrieval compatibility mechanism further constrains which complement can be considered for recovery. Importantly, the framework preserves the original reasoning trajectory when a compatible complement is unavailable. We agree that downstream logical consistency is an important consideration. The current formulation provides localized recovery rather than a formal guarantee of global logical consistency across every subsequent reasoning step. This distinction is important when interpreting the scope of our claims.

  1. Experimental evaluation:

The revised responses provide additional analyses, including multi-seed evaluation, an expanded GSM8K evaluation, alternative sentence-encoder comparisons, component ablations, and a KDIGO-related ablation. These additions address several aspects of the original evaluation concerns, although they do not establish comprehensive statistical robustness or fully matched inference budgets across all correction baselines. Similarly, the reported CKDQA retrieval availability measures the availability of compatible retrievals among detected candidates, rather than independently verified correction success. We therefore distinguish retrieval availability from actual correction effectiveness and recognize that broader independent correction validation and cross-task memory generalization remain important evaluation directions.

We respectfully emphasize that the proposed contribution is a trajectory-guided, retrieval-based localized recovery framework, rather than a general guarantee that every detected semantic deviation represents an error or that every retrieved complement ensures globally consistent reasoning. We hope this clarification helps establish the intended methodological scope and the evidence supporting the current submission.

Reviewer_mHE8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ORDDAR to detect and correct local errors in multi-step reasoning. ORDDAR represents the reasoning process as a sequence of transiting states, where state distances are used to detect erroneous transitions. To correct the errors, relevant information is retrieved from memory to revise the states. Experiments show that ORDDAR improves task performance. The results suggest that local error correction can be an alternative to regenerating complete reasoning sequences.

Critical strengths
  • The paper addresses an important problem of error propagation in multi-step reasoning by proposing a unified framework for detecting and repairing localized reasoning errors. The integration of lightweight distortion detection with memory-based recovery is a potentially efficient alternative to regenerating entire reasoning trajectories.
Critical weaknesses
  • There are related studies [1][2] that are closely related to the paper yet not discussed. These previous papers studied localized reasoning error correction, which is central to the method proposed in this paper. This would hinder the novelty of the paper.

  • The experimental evaluation is limited to 100 test instances per dataset and a single model (Qwen2.5-7B-Instruct). Moreover, the proposed method incorporate local KDIGO knowledge, so it is difficult to distinguish the benefits of ORDDAR from that of the KDIGO knowledge.

  • The soundness of the proposed recovery mechanism are not well justified. As semantic changes do not necessarily indicate reasoning errors, the trajectory threshold in Eqs. (8)–(9) may flag valid transitions. This issue seems to be not easy to be overcomed.

[1] Havrilla, A., Raparthy, S., Nalmpantis, C., Dwivedi-Yu, J., Zhuravinskyi, M., Hambro, E., & Raileanu, R. (2024). Glore: When, where, and how to improve llm reasoning via global and local refinements. arXiv preprint arXiv:2402.10963.
[2] Wu, Z., Zeng, Q., Zhang, Z., Tan, Z., Shen, C., & Jiang, M. (2025, July). Enhancing mathematical reasoning in llms by stepwise correction. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 21602-21623).

Questions for authors
  • Given the low detection precision (mostly about 0.5 in Table 4 of experiments), does this mean that using semantic changes to identify reasoning errors is not effective?
  • Coud you given some theoretical guarantees about the correctness of the correction from the proposed method (e.g., it is more likely to be logically correct and does not introduce new errors)?
3 comments

We sincerely thank Reviewer for the positive assessment of the importance of localized reasoning correction and for highlighting the potential of ORDDAR as an alternative to regenerating entire reasoning trajectories. We appreciate the constructive comments, which have helped us identify areas requiring further clarification and experimental analysis. We have incorporated GLoRe (Havrilla et al., 2024) and Stepwise Correction (Wu et al., 2025) into the Related Work section. We have also clarified how ORDDAR differs in its use of Shift–Gain–Loss signatures and Cognitive Mirror-based localized recovery. To broaden the evaluation, we extended the experimental study beyond the initial Qwen2.5-7B-Instruct evaluation by incorporating Llama-based evaluations across three random seeds (42, 123, and 2024), and report the mean performance across these runs. We also extended the GSM8K evaluation from 100 to 160 randomly selected instances to examine performance across different evaluation sample sizes. Furthermore, we conducted a prompt-level ablation on CKDQA using the same test instances under identical experimental conditions, comparing configurations with and without explicit KDIGO threshold instructions. The corresponding accuracies were 94.38% and 93.00%, respectively, representing a difference of 1.38 percentage points. These additional evaluations and their implications are discussed in the Results section. We agree that semantic changes between consecutive reasoning states do not necessarily indicate reasoning errors, as valid reasoning may involve substantial semantic shifts when incorporating new observations or evidence. In ORDDAR, the trajectory-adaptive threshold in Eqs. (8)–(9) is intended to identify potentially distorted transitions based on their relative displacement within a reasoning trajectory, rather than to establish definitive reasoning errors. Moreover, ORDDAR jointly considers semantic displacement and cognitive-anchor changes to characterize transition-level distortions. The detected transitions are subsequently examined through Cognitive Mirror-based complement retrieval, and the original reasoning trajectory is preserved when no compatible recovery complement is available. Thus, the proposed mechanism provides a localized distortion-identification and recovery strategy rather than treating every semantic change as an error.

Response Q1: We thank the reviewer for this observation. The precision values in Table 4 should be interpreted in the context of ORDDAR's objective: identifying potentially distorted transitions for subsequent localized recovery, rather than classifying every semantic change as a reasoning error. ORDDAR does not rely on semantic displacement alone; it combines semantic changes with cognitive-anchor variations to characterize potential distortions. Although the detector achieves moderate precision, its recall indicates that it captures a substantial proportion of the annotated distorted transitions. More importantly, the detected transitions are passed to the Cognitive Mirror-based recovery mechanism, which retrieves compatible recovery complements instead of automatically modifying every flagged transition. These results support the usefulness of combining semantic and cognitive-anchor information for distortion localization and recovery.

Response Q2: We thank the reviewer for this important question. ORDDAR incorporates a localized recovery mechanism with an explicit trajectory-preservation safeguard. Specifically, the recovery operator modifies only the identified distorted reasoning state using a compatible recovery complement retrieved from the Cognitive Mirror, while all unaffected reasoning states remain unchanged. Furthermore, if no compatible distortion–complement pair is found, the original reasoning trajectory is preserved without applying an unsupported correction. Memory updates are also restricted to validated recovery patterns. Therefore, ORDDAR is designed to prevent correction-induced modifications to unaffected reasoning states and to avoid applying arbitrary recovery operations. This design provides a structural safeguard for localized recovery: only the identified state is considered for modification, unaffected reasoning states remain unchanged, and recovery is applied only when a compatible validated complement is retrieved. Thus, ORDDAR constrains the scope of recovery and avoids indiscriminate rewriting of the complete reasoning trajectory.

Thanks for the responses and revisioin.

These additions address some of the concerns I raised. After reading the authors' responses and the comments from other reviewers, I remain concerned about the novelty of the approach, specifically in relation to existing methods for "localized reasoning correction", and the limited scope of the experimental evaluation. Furthermore, I share the concerns of the other two reviewers regarding the lack of detail in the description of the core recovery mechanism, and questions about the reliability of the distortion detection. Given these considerations, I would like to maintain my original rating.

Thank you for your feedback. We appreciate your acknowledgment that our responses and revisions addressed some of your concerns. We hope our clarifications provide a better understanding of the motivation, methodology, and contributions of ORDDAR. Thank you for your time and thoughtful consideration.

Comments on the paper

2 comments

Dear Reviewers,

Please review the authors' rebuttal and participate in the discussion to address any remaining questions before the period closes on Nov 18 AoE.

Thank you!

AC

Dear Reviewers,

As rebuttal ddl is approaching, please engage in discussion. Thank you!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.