acceptodds
Under review as a conference paper at ICLR 2027

Distilling Reasoning Chain and Chain-Local Experiences for Few-Shot Multimodal In-Context Learning

Abstract

Multimodal in-context learning (ICL) offers a promising avenue for adapting Large Vision-Language Models (LVLMs) to specialized downstream vision-language tasks with only a few in-context demonstrations (ICDs). However, existing few-shot multimodal ICL methods either rely on superficial image-question-answer imitation or induce ICD-individual reasoning guidance, limiting their ability to capture shared task-level reasoning structures. Moreover, these methods are largely inherited from pure language scenarios, overlooking perceptual bottlenecks of models in multimodal tasks and failing to provide sufficient perceptual support for reliable step-wise execution. In this paper, we propose Distill-ICL, a training-free framework that distills few-shot ICDs into two complementary forms of guidance: a Normative Reasoning Chain and Chain-Local Experiences. Specifically, we introduce ICDs-MCTS, an ICD-set-oriented variant of Monte Carlo Tree Search that collectively expands and evaluates reasoning paths over the entire demonstration set, inducing a shared reasoning chain that specifies the sequence of reasoning steps for the target task. To support accurate visual perception when executing these steps, Distill-ICL further performs controlled rollouts along the induced chain and distills local guidance at critical nodes: Inter-ICD Pivotal Experiences capture answer-discriminative visual attributes to guide critical decisions, while Intra-ICD Fragile Experiences reinforce visual attributes prone to misinterpretation to help mitigate step-local perceptual errors. Extensive experiments on five specialized multimodal benchmarks show that Distill-ICL consistently outperforms existing training-free ICL methods across various LVLMs, while exhibiting favorable computational efficiency and improved robustness to ICD selection and ordering perturbations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
In-Context Learning, Multimodal, Tree-Search, Test-Time Scaling, Training-Free

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_ZTtwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Distill-ICL, a training-free framework for few-shot multimodal in-context learning. The method first employs an ICD-set-oriented MCTS to identify a shared task-level reasoning chain and then distills chain-local experiences from successful and failed rollouts to provide step-specific visual guidance. Experiments across multiple benchmarks and LVLM backbones demonstrate consistent improvements over existing methods.

Critical strengths
  1. The paper identifies two meaningful limitations of existing multimodal ICL methods: the lack of shared task-level reasoning guidance and insufficient support for reliable visual perception during reasoning. The proposed combination of a normative reasoning chain and chain-local experiences provide a reasonable approach to addressing these limitations.
  2. By distilling reasoning guidance during context construction, the proposed method achieves relatively low online inference costs compared with other reasoning-augmented ICL methods, making it potentially useful when the distilled context can be reused across multiple queries.
Critical weaknesses
  1. The method’s novelty is limited. The proposed framework primarily combines and extends existing techniques, including MCTS-based reasoning search, contrastive analysis of reasoning trajectories, and experience summarization. In particular, the ICD-set-oriented MCTS mainly differs from standard MCTS by aggregating candidate actions across demonstrations and introducing a variance-penalized reward. The improvement over standard MCTS is relatively modest (62.5% vs. 61.6% in Table 2). Similarly, identifying critical reasoning steps by contrasting successful and failed rollouts appears to be a relatively straightforward extension of existing experience-based reasoning methods. Therefore, the methodological novelty of the proposed components remains somewhat limited.
  2. Insufficient comparisons with simpler baselines. Although the paper compares the proposed method with several reasoning-based baselines, it remains unclear whether the relatively complex MCTS search and experience distillation procedures are necessary. For example, a simpler approach could directly prompt the LVLM to summarize a shared reasoning procedure and identify discriminative or error-prone visual attributes from the entire ICD set, without performing MCTS or multiple controlled rollouts. Furthermore, the effectiveness of the JSD-based critical-node identification could be better justified by comparing it with random-node selection or experience extraction from all reasoning steps. Compute-matched comparisons with these simpler alternatives would help establish whether the improvements arise from the proposed mechanisms rather than additional computation and LLM-based summarization.
Questions for authors

See weakness

2 comments
AuthorsOpenReview ↗

Author Response to Reviewer ZTtw (Part 1/2)

We sincerely thank you for your effort in reviewing our paper, and for your valuable comments and insightful feedback. We also appreciate your recognition of “reasonable approach” and “low inference cost”. Below, we address each question individually.

(W1) Clarification of Methodological Novelty

We sincerely thank the reviewer for this thoughtful comment. We appreciate this concern and realize that our original presentation may not have clearly conveyed the key idea behind our design.

Our key insight is to first establish a shared task-level reasoning procedure and then identify critical and difficult-to-interpret visual cues along this shared structure. To our knowledge, this is a new perspective in multimodal reasoning-based ICL. A shared reasoning procedure alone cannot guarantee reliable visual interpretation, while experiences derived from independent trajectories lack a common structure for effective comparison and reuse. Our contribution lies in connecting these two aspects to make fuller use of limited demonstrations.

Our two components serve distinct but closely connected roles in realizing this idea:

(1) ICDs-MCTS: Beyond reasoning accuracy to a shared foundation for experience distillation. Unlike instance-specific search, our ICDs-MCTS searches for a shared Normative Reasoning Chain across the entire ICD set through the proposed collective expansion and universality-aware rewarding. More importantly, this shared chain provides a common reference for aligning and comparing step-wise reasoning behaviors across ICDs and rollouts. Thus, its value goes beyond the standalone accuracy improvement (62.5% vs. 61.6%), as it provides the foundation for subsequent experience distillation. Without this shared structure, experiences from different reasoning trajectories would be more difficult to compare and organize into reusable guidance.

(2) Chain-Local Experience Distillation: Beyond shared reasoning to reliable visual execution. While the Normative Reasoning Chain specifies what reasoning steps to follow, it does not guarantee their correct execution. We therefore identify two complementary types of critical nodes along this shared chain: Inter-ICD Pivotal Nodes, which capture answer-discriminative visual cues, and Intra-ICD Fragile Nodes, which reveal error-prone visual interpretations. These two types of experiences provide targeted guidance for both visual discrimination and reliable step-wise execution. Their effectiveness is supported by the improvement from 62.5% to 65.4% (copy from Tab. 2) when incorporating Chain-Local Experiences.

Our design emphasizes simplicity and effectiveness rather than algorithmic complexity. Rather than making each component unnecessarily complex, we use established techniques to realize a unified idea: using a shared reasoning chain as the foundation for distilling complementary, execution-level visual guidance. We believe the methodological contribution lies in this coherent integration of two interdependent components, enabling more effective use of limited demonstrations in multimodal ICL.

We sincerely appreciate the reviewer for helping us better clarify the motivation and contribution of our work.

AuthorsOpenReview ↗

Author Response to Reviewer ZTtw (Part 2/2)

(W2) Comparison with Simpler Baselines

Thanks for this valuable suggestion. We conduct an additional ablation study to evaluate the necessity of our proposed two components.

Experimental Setup. We use three benchmarks following Tab. 2. We evaluate two aspects: (I) the necessity of structured reasoning distillation, and (II) the necessity of critical-node identification. For the former, we introduce the Direct Prompting baseline, which directly prompts the LVLM to summarize a shared reasoning chain, with or without discriminative and error-prone visual attributes. For the latter, we compare our JSD-based critical-node identification with two alternatives: Random-Node Selection, which randomly selects the same number of nodes, and All-Node Extraction, which extracts experiences from every reasoning step. To ensure fairness, all critical-node selection variants are based on the same Normative Reasoning Chain R∗R^*.

Table R2: Comparison with simpler alternatives using Qwen3-VL-32B.

MethodsSMMILEEndoBenchFinMMEAvg.
Baseline
Few-shot CoT63.950.066.360.1
Block I: Structured Distillation vs. Direct Prompting
Direct Prompting (Chain Only)63.250.065.759.6
Ours (Chain Only)66.353.268.062.5
Direct Prompting (Chain + Experiences)63.950.466.560.3
Ours69.556.370.565.4
Block II: Node Selection
Random-Node Selection67.153.767.762.8
All-Node Extraction67.654.468.063.3
Ours (JSD-based)69.556.370.565.4

Key Observations. (1) Both Direct Prompting variants achieve performance comparable to Few-shot CoT (59.6%/60.3% vs. 60.1%), while substantially underperforming our corresponding methods (62.5%/65.4%). This suggests that effectively distilling shared reasoning chains and chain-local experiences from limited demonstrations is non-trivial and cannot be readily achieved through direct prompting alone, showing the importance of our structured search and experience distillation mechanisms.

(2) Based on the same chain, our JSD-based approach outperforms both Random-Node Selection (65.4% vs. 62.8%) and All-Node Extraction (65.4% vs. 63.3%). Extracting experiences from all steps may introduce redundant guidance that obscures truly pivotal or error-prone steps, underscoring the importance of targeted experience distillation.

We recognize that these additional ablations are valuable for further validating the necessity of the proposed two components. We have therefore incorporated them into the revised manuscript as the third part of the ablation studies in Sec. 4.3.

Reviewer_L1bhOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

Distill-ICL is a training-free framework for adapting VLM behavior from a small demonstration set. It first searches for a shared sequence of reasoning actions using ICDs-MCTS. Candidate actions are proposed with individual demonstrations, canonicalized, and evaluated by execution across the whole demonstration set. A mean-minus-standard-deviation reward is intended to favor broadly useful prefixes.

The second stage runs multiple executions along the selected chain. It identifies inter-demonstration “pivotal” nodes by comparing successful conclusion distributions for examples with different answers, and intra-demonstration “fragile” nodes by comparing successful and unsuccessful rollouts of the same example. The model then abstracts visual attributes into localized guidance. At inference, the query is answered using the original demonstrations, the shared chain, and these experiences.

Across five specialized multimodal benchmarks and five open or proprietary backbones, the paper reports gains over standard ICL and reasoning-augmented alternatives, along with component ablations, latency, perturbation robustness, and cross-model guidance transfer.

Critical strengths

1. The shared reasoning chain is an interesting way to use the demonstration set.

I particularly like the idea of learning a common reasoning procedure from the demonstrations as a group. Independent rationales can capture how to solve individual examples without making the underlying task structure clear. Searching for a shared chain gives the model a reusable procedure to follow. The pivotal and fragile experiences complement that procedure by highlighting answer-discriminative visual cues and steps where execution is unreliable. Together, these components offer a plausible way to make few-shot multimodal reasoning more robust.

2. The experiments support the usefulness of both the chain and the local experiences.

The improvements are consistent across the evaluated backbones and specialized tasks, and the ablations show that the shared chain and the two experience types each contribute. I also appreciate the support-selection and ordering experiments, since robustness to the choice of demonstrations is important in a few-shot setting. Taken together, the results provide a reasonable empirical case for the overall framework, rather than relying on a single model or benchmark.

3. The distilled guidance is potentially useful beyond a single inference.

The method separates context construction from online answering, so the cost of deriving the chain and experiences can be shared across queries using the same demonstrations. This makes the approach attractive when adapting a model to a recurring task. The cross-model transfer experiment also suggests that the resulting guidance captures some reusable task information. These findings strengthen the practical motivation for distilling demonstrations into a compact procedure.

Critical weaknesses

1. The evaluation is limited to specialized benchmarks.

The experiments focus on medical imaging, financial analysis, industrial anomalies, and plant pathology. These are reasonable settings for the proposed method, but they leave open whether the shared-chain approach also helps on general VQA. Demonstrations from a narrowly defined task may naturally share a procedure; everyday visual questions can involve more varied perceptual and commonsense reasoning.

I would like to see an evaluation on one or two standard benchmarks used in broader multimodal ICL studies. VQAv2 would test general image question answering, while OK-VQA or A-OKVQA would test questions involving broader world or commonsense knowledge. MMMU would provide a complementary test across a wider range of expert disciplines and college-level reasoning tasks. These evaluations would help establish whether the gains extend beyond the current specialized task groups and clarify when a shared chain is most useful. [1, 2, 3, 4, 7]

2. Similarity-based demonstration selection is missing from the comparisons.

The current baselines cover standard few-shot prompting and reasoning-augmented approaches, but do not establish how Distill-ICL compares with stronger demonstration retrieval. RICES and MMICES provide relevant similarity-based demonstration-selection baselines. A useful comparison would keep the backbone and number of demonstrations fixed and evaluate retrieved examples both without rationales and with the same rationale-generation procedure used for the relevant prompting baseline.

This would help distinguish the benefit of a shared chain and local experiences from the benefit of selecting more relevant demonstrations. Retrieval and distillation could also be complementary, so applying Distill-ICL to the retrieved support set would be informative. [5, 6]

References

[1] Goyal et al. Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering. CVPR, 2017. Introduces VQAv2.

[2] Marino et al. OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge. CVPR, 2019.

[3] Schwenk et al. A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge. 2022.

[4] Baldassini et al. What Makes Multimodal In-Context Learning Work?. CVPR Workshops, 2024.

[5] Alayrac et al. Flamingo: a Visual Language Model for Few-Shot Learning. NeurIPS, 2022. See Appendix A.2 for RICES.

[6] Chen et al. Can Multimodal Large Language Models Truly Perform Multimodal In-Context Learning?. WACV, 2025. Introduces MMICES.

[7] Yue et al. MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI. CVPR, 2024.

Questions for authors
  1. Does the method retain its advantage on general VQA and commonsense benchmarks such as VQAv2, OK-VQA, or A-OKVQA? Would it also help across the broader expert disciplines covered by MMMU? Which question types benefit most from a shared reasoning chain?
  2. How does Distill-ICL compare with RICES or MMICES under the same backbone and demonstration budget, both without rationales and with matched rationale generation? Would retrieving the support set before applying Distill-ICL further improve performance?
4 comments
AuthorsOpenReview ↗

Author Response to Reviewer L1bh (Part 1/2)

We sincerely thank you for your effort in reviewing our paper, and for your valuable comments and insightful feedback. We also appreciate your recognition of “the shared reasoning-chain design is interesting”, “comprehensive experimental validation”, and “the distilled guidance has practical value”. Below, we address each question individually.

(W1 & Q1) Evaluation on General Benchmarks

We thank the reviewer for this valuable suggestion. To further evaluate the generalizability of Distill-ICL, we conduct additional experiments on three general multimodal benchmarks: VQAv2, OK-VQA, and MMMU, and follow their official evaluation protocol.

Experimental Setup. To construct ICD–query groups, we adopt benchmark-specific grouping strategies based on their original annotations and data splits. Specifically, for VQAv2, we group samples sharing the same question, and sample ICDs and queries within each group. For OK-VQA, we randomly construct ICD–query groups within the same predefined knowledge category. For MMMU, we group samples by subject, using the official dev split as ICDs and the validation split as queries.

Table R1: Accuracy comparison (%) on three general multimodal benchmarks using Qwen3-VL-32B.

MethodsVQAv2OK-VQAMMMUAvg.
Zero-shot72.356.175.267.9
Few-shot ICL73.856.775.668.7
Few-shot CoT74.556.077.069.2
ThoughtICR75.657.277.870.2
Distill-ICL (Ours)78.257.079.871.7

Key Observations. In Tab. R1, our Distill-ICL achieves the highest average accuracy across three general benchmarks. Specifically, it surpasses the best baseline, ThoughtICR, on VQAv2 (+2.6%) and MMMU (+2.0%), demonstrating its effectiveness beyond specialized domains. In contrast, the improvement on OK-VQA is relatively limited, with performance slightly below ThoughtICR (-0.2%). Analyzing the reasons, we consider that Distill-ICL is particularly effective for tasks requiring structured reasoning and accurate interpretation of visual content in images, while its advantages are less pronounced for tasks that heavily rely on external knowledge. Furthermore, we observe that ICDs within the same OK-VQA knowledge category often require substantially different factual knowledge, thereby limiting the effectiveness of the shared reasoning chain. These findings further clarify the applicability and limitations of our Distill-ICL.

We have incorporated the results and analysis into Appendix C.4 of the revised manuscript.

AuthorsOpenReview ↗

Author Response to Reviewer L1bh (Part 2/2)

(W2 & Q2) Comparison with Similarity-based Demonstration Selection

Thanks for this valuable suggestion. We first clarify that our original few-shot setting focuses on verifying the model’s ability to distill effective guidance from a small set of randomly provided demonstrations. Thus, the similarity-based selection strategies cannot be directly applied in this setting. Nevertheless, we agree that it is valuable to “distinguish the benefits of our method from the benefit of selecting more relevant demonstrations”. We therefore extend our evaluation to a retrieval-enabled setting and conduct additional experiments with two selection strategies, RICES and MMICES.

Experimental Setup. In our original protocol (denoted as "Random"), ICD–query groups are randomly constructed from samples of the same task type. For RICES and MMICES, we instead pool eligible ICD samples of each task type into a shared candidate pool, from which demonstrations are retrieved for each query. We exclude SMMILE because its limited number of same-task samples prevents the construction of a sufficient candidate pool for retrieval. We keep the query set, number of ICDs per query, and LVLM backbone identical across all methods.

Table R2: Comparison with demonstration selection strategies using Qwen3-VL-32B.

SelectionMethodEndoBenchFinMMEMMADPlantVillageVQAAvg.
RandomFew-shot ICL46.865.060.734.351.7
RandomFew-shot CoT50.066.365.438.555.1
RandomDistill-ICL56.370.572.844.761.1
RICESFew-shot ICL49.166.461.337.253.5
RICESFew-shot CoT51.567.165.840.956.3
RICESDistill-ICL57.071.073.245.761.7
MMICESFew-shot ICL49.467.161.236.853.6
MMICESFew-shot CoT51.667.666.040.656.5
MMICESDistill-ICL57.471.773.545.362.0

Key Observations. (1) Compared with Random, both RICES and MMICES consistently improve all three inference methods, confirming the benefits of selecting more relevant demonstrations.

(2) Across all four benchmarks, Distill-ICL with random demonstrations consistently outperforms “RICES/MMICES + Few-shot CoT”, even though both approaches incorporate rationales. This suggests that, in specialized downstream tasks, distilling effective reasoning guidance and chain-local experiences is more beneficial than simply selecting more similar demonstrations.

(3) Combining Distill-ICL with RICES/MMICES further improves its performance, demonstrating the complementarity of retrieval and distillation.

We have incorporated these additional experiments and analyses into the Appendix C.5 of the revised manuscript.

Thanks for the authors' detailed rebuttal. My concerns have been fully addressed (generalizability on general benchmarks, stable improvement combined with similarity-based demonstration selection. This is a very interesting work. I will raise my score to 4.

AuthorsOpenReview ↗

Author Response to Reviewer L1bh

We are pleased that the additional experiments and clarifications have addressed your concerns, and we are very grateful for your constructive comments and suggestions.

Reviewer_fiWJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

For few-shot multimodal ICL on specialized VQA tasks, this paper proposes Distill-ICL, a training-free framework which distills few-shot ICDs into two complementary forms of guidance: a Normative Reasoning Chain and Chain-Local Experiences. Given only a few labelled in-context demonstrations (ICDs), ICDs-MCTS runs a Monte Carlo tree search that gets the sequence of reasoning steps for the target task; controlled rollouts are performed along the induced chain; local guidance at critical nodes are distilled. The authors evaluate Distill-ICL and baselines on multiple domain benchmarks with multiple backbones (including closed- and open-souce VLMs).

Critical strengths
  • The problem setting seems to be well motivated, and the proposed method is practical.
  • I see consistent gains in abundant evaluation results. There are ablation study, perturbation study, cross-model transfer of the distilled guidance, and other statistics provided in main paper and appendix. I especially like the ablation where both components are evaluated against reasonable alternatives, proving they are useful.
Critical weaknesses
  • Authors claim that a shared task-level trajectory is more effective than selecting or aggregating instance-specific trajectories, but the weaker variants they use originally underperform ThoughtICR. Also, I think I can't tell to what extent the whole system's advantage over ThoughtICR owes to the collective search, as opposed to the chain-local experiences built on it.
  • I didn't find some specific details of the evaluation protocol; correct me if I'm wrong. For example, the number of ICDs (M) for each benchmark, the number of queries for each episode, how many queries share one ICD set, how invalid or not parseable outputs are dealt with, the values of λ (in Eq. 3) and c (in Eq. 5), etc.
  • If Eqs. 7–8 compute JS divergence between conclusion distributions, while step-wise conclusions seem to be free text, then I don't understand how they are discretized. Also, what're the meaning of thresholds γ_p = 0.3 and γ_f = 0.25 then?
Questions for authors
  • Please show the Zero-shot / Few-shot ICL / ThoughtICR prompts; confirm that all methods use the same answer-format instruction and extraction rule.
  • I like your figures, but will you slightly modify the details of them? For instance, in Figure 1, the Normative Chain block on its top-right side has a strange outline; in Figure 3, the bottoms of the bars somehow cover the x axis.
  • Is "gropu" in FinMME dataset description a typo? Are you using Gemini-3.1-Flash or Gemini 3.1 Flash-Lite?
4 comments
AuthorsOpenReview ↗

Author Response to Reviewer fiWJ (Part 1/2)

We sincerely thank you for your effort in reviewing our paper, and for your valuable comments and insightful feedback. We also appreciate your recognition of “Well-Motivated Problem Setting”, “Practical Method”, and “Abundant Evaluation”. Below, we address each question individually.

(W1) Clarifying the Advantages of Collective Search over ThoughtICR

We thank the reviewer for this valuable comment. We first clarify that ThoughtICR consists of two steps: (1) constructing instance-specific thought cards for each ICD via MCTS, and (2) adaptively retrieving relevant thought cards, generating multiple candidate answers, and selecting the most frequent answer through self-consistency strategy. Therefore, ThoughtICR's advantage over our weaker variant (i.e., ICDs-MCTS (R∗R^*), reported in Tab. 2) cannot be attributed solely to instance-specific search, as it also benefits from the additional test-time scaling in the second step.

To provide a more direct comparison, we further conduct an ablation retaining only the first step of ThoughtICR. Specifically, we follow its original MCTS procedure to construct thought cards for individual ICDs, concatenate the corresponding thought-card-guided reasoning trajectories and answers into the inference context, and adopt the same answer-format instruction as our method. As shown below, this variant underperforms our weaker variant ICDs-MCTS (R∗R^*) (60.7% vs. 62.5%), and even the standard instance-specific MCTS baseline (60.7% vs. 61.6%). Overall, while differences in the underlying frameworks prevent us from precisely isolating the gain attributable solely to collective search over ThoughtICR, our ICDs-MCTS outperforms ThoughtICR (Stage I only) (62.5% vs. 60.7%) across all three benchmarks without incorporating Chain-Local Experiences, providing direct evidence of the effectiveness of collective search.

Table R1: Comparison of instance-specific and collective reasoning-chain search using Qwen3-VL-32B.

TypeMethodsSMMILEEndoBenchFinMMEAvg.
Instance-SpecificThoughtICR (Stage I only)64.551.066.760.7
Instance-SpecificStandard MCTS (Copy from Tab. 2)65.452.067.461.6
CollectiveICDs-MCTS (R∗R^*, Copy from Tab. 2)66.353.268.062.5

We have realized that this experiment can better verify the effectiveness of our collective search reasoning chain, and added this comparison into Appendix C.3 of the revised manuscript.

(W2) More Details

We thank the reviewer for pointing out these missing details. We clarify them below and have incorporated the corresponding details into the revised manuscript:

(1) Dataset and Episode Configuration. We provide the relevant details for each benchmark, including the number of ICDs (MM), the number of queries per episode, and the number of queries sharing the same ICD set, as summarized below.

Table R2: Details for each benchmark.

BenchmarkICDs (MM)Queries per EpisodeQueries per ICD Set
SMMILE (official setting)2-191111
EndoBench2-101,0401-5
FinMME2-162,8401-6
MMAD2-89501-6
PlantVillageVQA2-211,2601-21

These details have also been added to the Appendix A (Tab. 7) of the revised manuscript.

(2) Handling of Invalid Outputs. All tasks are formulated as multiple-choice questions, and we apply a unified answer extraction rule across all methods. Specifically, we extract the predicted option letter from the designated <answer>LETTER</answer> tag and validate it against the available options. If an output is unparseable, we retry generation up to five times. Outputs that remain invalid after these retries are counted as incorrect. Notably, no outputs remained unparseable after this procedure across our evaluations. The relevant details have been added to Section 4.1 (“Evaluation Protocol”) of the revised manuscript.

(3) Hyperparameter Settings. We set both λ (Eq. (3)) and c (Eq. (5)) to 1 in our experiments. We have additionally included the hyperparameter search results in the Appendix C.1 (Fig. 6) of the revised manuscript.

AuthorsOpenReview ↗

Author Response to Reviewer fiWJ (Part 2/2)

(W3) Clarification of Distribution Calculation and Thresholds

We thank the reviewer for this valuable comment. We elaborate on two aspects:

(1) Calculation of Conclusion Distributions. Actually, the intermediate conclusion (cm,ikc_{m,i}^{k}) defined in Sec. 3.4 represents a discrete semantic category accompanied by a human-readable textual description, rather than unrestricted free text. Specifically, we use prompt-based normalization to instruct the LVLM to assign each step-wise conclusion a discrete semantic label and its corresponding textual description. Different rollouts use a unified set of semantic categories under the same normative chain node. For example, at the second reasoning step in Fig. 1, conclusions are categorized as "1: right peripheral lung markings are absent" or "2: peripheral lung markings are present." For clarity, Fig. 1 only displays their textual descriptions. Thus, for each JSD calculation in Eqs. (7) and (8), we obtain two sequences of consistently aligned discrete semantic labels from the corresponding rollouts, estimate their respective empirical probability distributions, and compute the JSD between these distributions.

We acknowledge that omitting this implementation detail caused confusion and have clarified it in the revised manuscript (Line 241), with the corresponding LVLM prompt template provided in Appendix B.2.

(2) Explanation of Two Thresholds. The two thresholds (γp\gamma_p and γf\gamma_f) specify the minimum divergence required to identify critical nodes. With JSD normalized to [0,1][0,1] (using log base 2), larger values indicate greater differences between conclusion distributions. Specifically, (γp\gamma_p=0.30) identifies pivotal nodes with sufficiently divergent conclusions across successful rollouts of ICDs with different answers, while (γf\gamma_f=0.25) identifies fragile nodes with divergent conclusions between successful and failed rollouts of the same ICD. These thresholds are empirically determined rather than theoretically prescribed, as supported by the sensitivity analysis in Appendix C.1 (Fig. 5).

(Q1) Prompt Template

Due to differences in contextual components across methods, we maintain prompt consistency as much as possible, although some variations remain. Nevertheless, all methods follow the same answer-format instructions and answer extraction rules. Additionally, we have included the prompt templates for Zero-shot, Few-shot ICL, and ThoughtICR in Appendix B of the revised manuscript.

(Q2) Figure Revision

Thanks for pointing this out. We have optimized Figures 1 and 3 in the revised manuscript.

(Q3) Typo and Model Usage

Thank you for pointing this out. We have corrected the typo. We use Gemini 3.1 Flash-Lite, and we have amended the model name in the revised manuscript to avoid any misunderstanding.

Thanks for your detailed response and additional experiments.

For W1, the ThoughtICR (Stage I) comparison in Table R1 is just what I need. I also read the experiment results in your reponse to Reviewer ZTtw.

For W2~3 and Q1, your Table R2 and the provided additional information resolve my original concerns. My attitude is now more positive.

Since the concerns I raised have been addressed, I will raise my rating to 3 (weak acceptance).

AuthorsOpenReview ↗

Author Response to Reviewer fiWJ

Thank you for your positive reassessment and for taking the time to review our additional experiments and clarifications. We are glad that the additional comparisons and information have addressed your concerns. We sincerely appreciate your constructive feedback, which has helped us further improve the quality and clarity of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.