acceptodds
Under review as a conference paper at ICLR 2027

ActICL: Post-Training for Agentic Multimodal In-Context Learning

Abstract

Multimodal in-context learning (ICL) is an important capability of multimodal large language models (MLLMs). Its performance is closely tied to the demonstrations it uses. Thus, retrieving demonstrations from a given bank is a core problem in many ICL-based systems. Common methods use fixed metrics to retrieve all the demonstrations at once, which may introduce redundancy and noise. Context-aware retrieval methods have emerged to address this problem. However, none of them lets the MLLM itself learn how to judge what new demonstrations it needs through its reasoning and then carry out progressive retrieval on its own. To this end, we introduce ActICL, a framework that trains an MLLM agent to acquire and use an ICL context via tool-augmented reasoning. ActICL builds an agent that analyzes its own information gaps during iterative reasoning and tool-use actions. Based on these gaps, it performs targeted retrieval to acquire a set of initial candidates. It can then use the existing context to further filter the candidates and refine the context, or it can iteratively perform a new retrieval. When the agent judges that the context is sufficient for the task, or when the resource limit is reached with the user request met, it sends the acquired context to an answering model, either the agent itself or another MLLM. We use a two-stage post-training procedure that jointly optimizes the agent's acquisition and answering. When the agents adaptively decide the number of demonstrations, ActICL improves average performance by 11.6 percentage points over standard similarity-based retrieval across seven benchmarks and three MLLMs, and by 4.4 points compared to the strongest prior baseline. The contexts acquired by ActICL agents can also be used effectively by frontier models. When GPT-5.6-Sol performs 4-shot ICL, ActICL's contexts improve its performance by 3.5 points over standard retrieval, averaged over three benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Models, Multimodal In-context Learning, Multimodal Agents, Multimodal Post-training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2uRJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper trains an MLLM agent to construct demonstration contexts for multimodal ICL. During iterative reasoning, the agent identifies its own information gaps, performs targeted retrieval to obtain candidate demonstrations, and then uses the existing context to filter, refine, or further expand the candidate set. Once the agent judges the context to be sufficient, it passes it to an answering model. Training uses a two-stage post-training procedure that jointly optimizes context acquisition and answering. Across 7 benchmarks and 3 MLLMs, the authors report an average improvement of 11.6 percentage points over similarity-based retrieval, and 4.4 points over the strongest baseline.

Critical strengths
  1. Demonstration selection is a key factor in multimodal ICL performance; the research direction of this paper has practical value.
  2. The paper is clearly written and well organized, and the experiments include ablations on training stages, the number of demonstrations, and retrieval modes.
Critical weaknesses
  1. In terms of problem novelty and related-work coverage, the core idea has appeared in published work, and the closest works are not discussed. MetaFlowLLM (NeurIPS 2025) likewise retrieves and reuses the experience of past tasks to assist solving new tasks, and Agent Learning via Early Experience (ICML 2026) likewise leverages the agent's own interaction experience to improve future behavior. The core mechanism of this paper overlaps with both, and the paper does not cite or discuss either of them. In addition, Spinning Straw into Gold (ICLR 2026) constructs successful demonstrations from the model's own rollouts for subsequent training, and Auto-regressive In-context Demonstration Selection (ICML 2026) studies how demonstrations are matched to and organized around a query; neither is mentioned, leaving readers without the information needed to position this paper within this line of work and identify its increment.
  2. In terms of generality of the method, the acquisition pipeline (retrieve, filter, iteratively retrieve more) and the training pipeline (SFT plus group-relative RL) are combinations of standard components, and all three trained agent backbones in the experiments come from the Qwen family. It remains unverified whether the post-training gains transfer to other model families.
Questions for authors

See Critical Weaknesses. In particular: is there evidence that the post-training gains transfer to backbones outside the Qwen family?

3 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer 2uRJ

Thank you very much for the effort you have put into reviewing our paper, and for your recognition of the significance of our research topic, the quality of our writing, and the thoroughness of our experiments. Below, we provide our responses to the valuable comments you have raised. Changes or additions made in the submission PDF have been highlighted in blue.

W1: Four additional works should be discussed.

Thank you for pointing out these works. We would like to respectfully question the relevance of some of these papers, as their research focuses are not closely aligned with the scope of our work. Specifically, the first three works you mentioned, MetaFlowLLM, Agent Learning via Early Experience, and Spinning Straw into Gold, have a fundamentally different purpose from ours. These methods treat the past execution history of an agent as experience and use it to improve future decisions. Agent Learning via Early Experience and Spinning Straw into Gold use these experiences directly as data to retrain the agent, while MetaFlowLLM trains a rewriter that turns them into code or prompts. In contrast, our work studies the general ICL setting, where demonstrations from an external bank are placed in the input context of the model to support its reasoning. These demonstrations come from pre-defined banks rather than from the agent’s own interaction history. Our pipeline also differs from these methods. They focus on how to collect and effectively use experience, whereas our work focuses on enabling the agent to retrieve suitable demonstrations by itself.

Unlike the first three papers, Auto-regressive In-context Demonstration Selection (AutoSelect) does fall within the same scope as our work. However, it does not address multimodal tasks, and it belongs to one category of our baselines, trained retrievers, as illustrated in Figure 1(b). We have added it to the Related Work section in lines 129--130. Unlike this category, ActICL does not rely on a separately trained retriever. It lets the MLLM itself decide through reasoning what to retrieve, which candidates to inspect, and which demonstrations to keep. In summary, ActICL enables the model itself to reason about and retrieve multimodal demonstrations, and thus the novelty and contributions claimed in our paper do not overlap with these works.

W2: It remains unverified whether the post-training gains transfer to other model families.

First, we respectfully argue that the three backbones in our experiments already provide good evidence for the generality of ActICL; nevertheless, based on this comment, we have further added a fourth backbone. Within Qwen3-VL, we test both a dense model (Qwen3-VL-8B) and a Mixture-of-Experts model (Qwen3-VL-30B-A3B). Moreover, although Qwen3.5 shares the family name, it differs fundamentally from Qwen3-VL in both architecture and training strategy. In architecture, Qwen3-VL builds on the Qwen3 language model with standard full attention, whereas Qwen3.5 adopts a hybrid design in which three out of every four layers use Gated DeltaNet linear attention and only the remaining layer uses gated full attention. In training, Qwen3-VL starts with a pretrained language model and a vision encoder, trains a projection module to align them, and then jointly trains all components. By contrast, Qwen3.5 trains on interleaved text, image, and video tokens from the outset of pretraining. ActICL brings consistent gains on all three backbones, which shows that its effect does not depend on a specific architecture or training recipe.

To further demonstrate the generality of ActICL, we additionally train ActICL agents on InternVL3.5-8B and report the results in Table 14 of the revised appendix, which is also shown in Table R.6 below. On InternVL3.5-8B, 4-shot ActICL outperforms all baselines on all seven benchmarks, with average gains of 9.0 percentage points over MMICES and 2.9 points over ProMSA. Adaptive acquisition further improves average accuracy by 1.8 points. These results show that the post-training gains of ActICL transfer to other model families.

Table R.6 Comparison of ICL methods on InternVL3.5-8B.

MethodsTextVQAMME-RCharXivGVLMathVisionMDK12VL-ICLAvg.
InternVL3.5-8B0.7260.1690.4240.5070.4410.4300.2520.421
+Rand. Sample0.7210.1730.4310.4950.4600.4230.4740.454
+MMICES0.7380.1850.4460.5110.4720.4670.5620.483
+ContextNav0.7420.1910.4600.5290.4810.4810.6440.504
+LSD0.7450.1960.4570.5320.4930.4900.6290.506
+ProMSA0.7830.2420.4870.5630.5350.5310.6680.544
+ActICL0.7950.2830.5140.5920.5570.5690.7030.573
+ActICL (adaptive)0.8040.2790.5260.6330.5720.5840.7390.591

Dear Reviewer 2uRJ,

There are only 3 days left in the discussion period. In our response, we discuss the papers you highlighted and provide further evidence of ActICL's generalizability across different model families. We would be grateful if you would consider revisiting your score in light of our responses. If you have other unresolved questions, please let us know. We would be happy to engage in the days that remain. Thank you again.

Dear Reviewer 2uRJ,

We wanted to follow up, as this is the last day of the discussion, and see if our additional experiments and clarifications have addressed your questions. Thank you again.

Reviewer_jd6AOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces ActICL, a framework that teaches multimodal LLM agents to build their own in-context demonstrations instead of relying on fixed similarity-based retrieval. Rather than pulling the top-k most similar examples, the agent reasons about what information it is still missing and gathers context step by step, using four actions: search, read, curate, and solve. Training happens in two stages. Supervised fine-tuning comes first, followed by reinforcement learning that optimizes both how the agent collects context and how well it answers. Across seven benchmarks, ActICL improves on standard similarity-based retrieval by 11.6 points on average, while using fewer demonstrations.

Critical strengths
  1. The paper introduces a novel agentic framework that transforms traditional one-shot demonstration retrieval into an iterative, reasoning-driven process. By enabling agents to search, read, and curate demonstrations based on evolving context needs, ActICL effectively addresses the redundancy and limited adaptability of existing retrieval methods.

  2. ActICL shows consistent improvements across seven benchmarks and three multimodal LLMs. The learned context acquisition strategy generalizes to stronger, unseen answering models, demonstrating its potential as a reusable component in broader multimodal ICL systems.

Critical weaknesses
  1. Because ActICL reasons iteratively and makes several tool calls per query, it is likely to be more expensive at inference time than one-shot similarity retrieval. The paper does not report end-to-end latency or compute cost, so it is hard to tell whether the accuracy gains are worth the extra overhead in practice.

  2. In Table 1, ActICL uses its jointly post-trained model for answering, while most baselines use the original backbone, making it difficult to isolate the gains from context acquisition. The authors should provide a more systematic attribution of the reported improvements to better context acquisition versus enhanced answering ability.

Questions for authors
  1. What is the acquisition failure rate at test time, how often does the adaptive agent commit an empty context, and how do these vary by benchmark?
5 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer jd6A (1/2)

Thank you very much for the time and effort you have dedicated to reviewing our paper, and for your recognition of the innovation and excellent performance of our proposed method. Below, we provide our responses to the valuable comments you have raised. Changes or additions made in the submission PDF have been highlighted in blue.

W1: The authors should report end-to-end latency to tell whether the accuracy gains are worth the extra overhead in practice.

Thank you for this key suggestion on the practicality of agentic frameworks. Like other agentic methods, ActICL involves a trade-off between test-time cost and performance, which we have discussed in Appendix A. Nevertheless, ActICL achieves a favorable balance between the two. Following your suggestion, we report the end-to-end latency in Table R.3 below and in Table 6 of the revised paper. ActICL is slower than one-step similarity retrieval such as MMICES, but it improves average accuracy by 10.2 points in the 4-shot setting and 11.6 points in the adaptive setting. Compared with ProMSA, another trained agentic method, ActICL (adaptive) is slightly faster (8.75 s vs. 8.87 s) and 4.4 points more accurate. Measured by the accuracy gained per additional second over MMICES, ActICL (adaptive) is the most efficient method, with 1.75 points per second, compared with 0.50 for ContextNav and about 1.0 for LSD and ProMSA. It is also worth noting that the contexts constructed by ActICL generalize well across answering models. Once constructed, a context can be reused by different answering models without repeating acquisition, which amortizes the acquisition cost and further increases the practical value of ActICL. In summary, the accuracy gains of ActICL are worth the extra overhead in practice. We have added these results and analyses to the limitation section in Appendix A.

Table R.3 End-to-end latency and accuracy averaged over the seven benchmarks and the three backbones in Table 1.

MethodLatency (s/query)Acc.
MMICES2.140.607
ContextNav7.930.636
LSD5.320.639
ProMSA8.870.679
ActICL9.580.709
ActICL (adaptive)8.750.723

W2: The authors should provide a more systematic attribution of the reported improvements to better context acquisition versus enhanced answering ability.

First, we would like to note that this comparison is largely available in three parts of our existing results. In Table 3, the ActICL agent serves only as a context constructor for frontier models, which perform the answering. Comparisons with MMICES and ContextNav demonstrate ActICL’s effectiveness in context acquisition by evaluating the contexts constructed by its learned agent against those of alternative retrievers. Conversely, in Table 2, the contexts are built by MMICES and remain fixed, so the gains of the ActICL-trained model over LoRA and DARA reflect its improved answering abilities. Table 1 then reports the full pipeline, where both abilities work together.

To make the attribution more explicit within the same backbone, we additionally report results in Table 1 and Table R.4 below where the original backbone answers with contexts constructed by the ActICL agent. In this setting, contexts constructed by ActICL outperform all prior baselines that use the same answering model, with average gains of 4.0--5.6 percentage points over MMICES and 1.3--1.8 points over the strongest prior baseline. Since these methods differ only in how the context is acquired, the gains isolate the contribution of context acquisition. We have revised the writing in Section 4.3 to make the analysis of the results clearer and more systematic.

Table R.4 Results of ActICL with the original model as the answering model. The ActICL agent constructs the context, and the original backbone answers with it. These results are added to Table 1 of the revised paper.

ModelMethodTextVQAMME-RCharXivGVLMathVisionMDK12VL-ICLAvg.
Qwen3-VL-8B+ActICL0.7950.3140.5330.6360.5640.5380.8360.602
+ActICL (adaptive)0.8040.3120.5390.6410.5600.5510.8540.609
Qwen3-VL-30B-A3B+ActICL0.8760.3620.5590.6410.6450.5430.8460.639
+ActICL (adaptive)0.8830.3650.5630.6340.6500.5460.8570.643
Qwen3.5-9B+ActICL0.8800.3920.7260.7250.8150.6870.8850.730
+ActICL (adaptive)0.8910.4010.7450.7340.8320.7040.8970.743
AuthorsOpenReview ↗

Rebuttal for Reviewer jd6A (2/2)

W3: What is the acquisition failure rate at test time, how often does the adaptive agent commit an empty context, and how do these vary by benchmark?

We have added the per-benchmark failure rate and empty-context rate of ActICL on Qwen3-VL-8B and Qwen3.5-9B, shown in Table 18 of Appendix H and in Table R.5 below. The most common failures in ActICL fall into three types: reaching the budget before acquiring the required number of demonstrations in the fixed-count setting, receiving a search observation that alone exceeds the token budget, and failing to parse the retrieval-channel arguments of a search call. Because our training emphasizes valid tool calls and context management, failures are relatively rare, with average rates of 5.8% for Qwen3-VL-8B and 3.0% for Qwen3.5-9B. They are most frequent on VL-ICL, which relies heavily on demonstrations, and on MathVision, whose hard reasoning problems likely require longer trajectories. The adaptive agent commits an empty context for 13.8% and 15.5% of the queries, respectively. This rate is lower on VL-ICL, MathVision, and MME-R, which benefit most from demonstrations, and higher on TextVQA, GVL, and MDK12, where many queries can be answered without them. We also find that on the queries for which the adaptive agent commits an empty context, ActICL in the fixed 4-shot setting performs slightly worse than answering with the empty context. This shows that these empty contexts are justified decisions and demonstrates the advantage of supporting the adaptive setting in ActICL. Across backbones, Qwen3.5-9B has a lower failure rate on every benchmark and commits empty contexts more often where its zero-shot accuracy is much higher, which is consistent with its stronger agentic ability.

Table R.5 Acquisition failure rate and empty-context rate (%) at test time. The empty-context rate (%) is the proportion of queries for which the adaptive agent commits an empty context.

BackboneMetricTextVQAMME-RCharXivGVLMathVisionMDK12VL-ICLAvg.
Qwen3-VL-8BFailure4.35.25.73.57.26.18.45.8
Empty16.510.713.820.29.818.96.913.8
Qwen3.5-9BFailure2.53.02.72.53.92.63.53.0
Empty23.79.811.624.39.520.49.215.5

Dear Reviewer jd6A,

There are only 3 days left in the discussion period. In our response, we provide further efficiency analyses and a more detailed attribution of the performance gains, along with additional results on the failure rate and empty-context rate for each benchmark. We would be grateful if you would consider revisiting your score in light of our responses. If you have other unresolved questions, please let us know. We would be happy to engage in the days that remain. Thank you again.

Reviewer_i6XCOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ActICL, a framework for post-training multimodal agents for in-context learning. Instead of retrieving demonstrations in one step, the method trains the MLLM agent to iteratively acquire context through reasoning and tool use, including search, read, curate, and solve actions. The agent is trained with a two-stage pipeline combining rejection-sampling SFT and joint RL. Experiments on multiple multimodal benchmarks and several backbones show improvements over standard retrieval baselines and prior approaches. The paper also evaluates using ActICL as a context constructor for stronger external models.

Critical strengths
  1. The paper is well written and logically organized. The motivation, method, and experiments are presented clearly, and the overall framework is easy to follow.
  2. The experimental evaluation is fairly comprehensive. The paper includes multiple backbones, several benchmarks, comparisons with strong baselines, and a range of ablations and transfer settings.
Critical weaknesses
  1. ActICL introduces a more complex multi-step agent pipeline and additional training. While the gains are significant, it is still unclear how much comes from the agent architecture itself or simply from more training, and it is unclear how this compares to directly training a model to select suitable examples in one step;
  2. there is no sufficient ablation on the multi-step agent pipeline, especially the individual contributions of search/read/curate/solve;
  3. it is unclear how the quality of training ICL demonstrations is ensured, since some demonstrations may have correct answers but still not positively help the reasoning process.
Questions for authors

See Weaknesses above. If the authors can address the concerns above, I would increase my score.

6 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer i6XC (1/2)

Thank you very much for the effort you have put into reviewing our paper, and for your recognition of our research topic, the clarity of our presentation, and the comprehensiveness of our experiments. Below, we provide our responses to the valuable comments you have raised. Changes or additions made in the submission PDF have been highlighted in blue.

W1: It is unclear how much comes from the agent architecture itself or simply from more training, and it is unclear how this compares to directly training a model to select suitable examples in one step.

Thank you for your positive assessment of ActICL's performance. To examine the contribution of the agent architecture, we add a Tool-use baseline in Table 1 and Table R.1 below, which gives the original model the same tools and environment as ActICL without any training. Without training, the gains from the agentic architecture are not always stable, especially for backbones with weaker agentic ability and when the model needs to perform adaptive ICL. In Table R.1, switching Tool-use to the adaptive setting lowers its average accuracy from 0.589 to 0.581 on Qwen3-VL-8B and from 0.626 to 0.619 on Qwen3-VL-30B-A3B, which falls below LSD on both backbones, while it improves from 0.721 to 0.729 on Qwen3.5-9B. By contrast, the trained ActICL agent benefits from the adaptive setting on all three backbones. This is why ActICL couples the architecture with targeted post-training. The goal of ActICL is to enable the model to interact agentically with an external demonstration bank through its own reasoning. Thus, the targeted post-training and our agentic architecture are organically combined, and their effect cannot be characterized simply as "more training". The results where ActICL uses the original model for answering, which we also newly add in Table 1, support this point. With the same architecture and the same answering model, ActICL outperforms Tool-use by 0.9-1.3 percentage points with four shots and by 1.4-2.8 points in the adaptive setting. We also have further evidence that the gains cannot be attributed simply to a larger amount of training. ProMSA is also trained with SFT and RL on the same training data and with a comparable training cost, but ActICL outperforms it by 2.8-3.2 points in Table 1.

Table R.1 Results of the Tool-use baseline, which provides the original model with ActICL’s tools and environment without any training. These results are added to Table 1 of the revised paper.

ModelMethodTextVQAMME-RCharXivGVLMathVisionMDK12VL-ICLAvg.
Qwen3-VL-8B+Tool-use0.7830.3000.5230.6170.5630.5320.8030.589
+Tool-use (adaptive)0.7670.2840.5150.6200.5680.5250.7870.581
Qwen3-VL-30B-A3B+Tool-use0.8590.3500.5410.6200.6470.5340.8320.626
+Tool-use (adaptive)0.8610.3470.5270.6160.6360.5270.8180.619
Qwen3.5-9B+Tool-use0.8870.3910.7090.7160.7910.6840.8710.721
+Tool-use (adaptive)0.8940.3980.7170.7210.8090.6920.8730.729

Regarding the second subquestion, if you are referring to the agent selecting suitable demonstrations in one step, we add such a one-step variant of ActICL to Table 5 of the revised submission and Table R.2 below. It is trained to retrieve the required number of suitable demonstrations through a single search and then directly commits them through solve, without read or curate. This variant reaches only 0.668 on average, which is 4.1 percentage points below the full ActICL. Selecting all demonstrations in one step does not allow the agent to inspect candidates, compare alternatives, or revise its context, and training alone cannot make up for this. If you are referring to training a separate model dedicated to selecting demonstrations for an answering model, our original submission already includes LSD [1] as a baseline. LSD trains a retriever with answering-model feedback, and we train it on the same data sources as ActICL. With the newly added results in Table 1, where ActICL also uses the original model for answering, the two methods share the same answering model and differ only in how demonstrations are selected. In this setting, ActICL outperforms LSD by 1.8 percentage points on average on the three backbones.

[1] Lee, Eugene, Yu-Chi Lin, and Jiajie Diao. "Learning to select visual in-context demonstrations." arXiv preprint arXiv:2603.26775 (2026).

AuthorsOpenReview ↗

Rebuttal for Reviewer i6XC (2/2)

W2: There is no sufficient ablation on the multi-step agent pipeline.

First, we would like to clarify that the original submission already includes an ablation on read and curate in Appendix H, the two key actions that separate what the agent sees from what it keeps. We have added it to Section 4.3 of the main text. To summarize the existing ablation: for each variant, we train the agent with the corresponding action removed, and every removal lowers the score on all three benchmarks. Removing read hurts most. Without previews, every retrieved candidate enters later observations in full, so the acquisition budget runs out after fewer searches and the agent compares fewer alternatives before committing. Removing curate also lowers the score, since the agent must keep every demonstration it opens, including those that prove unhelpful. We also tested an add-only variant of curate. It recovers only part of this gap, which shows that both adding and removing demonstrations contribute. The “Context-aware multimodal retrieval” paragraph of Section 4.3 also ablates the retrieval strategy of search. The solve action ends the agentic workflow and commits the constructed context, so it is necessary and cannot be removed.

Moreover, based on your question, we have added a variant in which search is replaced by uniform random sampling from the demonstration bank. This variant causes the largest drop, from 0.709 to 0.659 on average, which shows the importance of goal-conditioned and modality-specific retrieval. The one-step variant described above can also be seen as removing read and curate together. It performs worse than removing either action alone, which further shows that the two actions are complementary. Together, these results show that each action contributes to the final performance. These results are shown in Table 5 of the revised submission and in Table R.2 below.

Table R.2 Ablation on the action space with Qwen3.5-9B under the adaptive setting. Each variant is trained with the corresponding actions removed.

VariantMME-RCharXivMathVisionAvg.
w/o search0.4070.7490.8200.659
w/o read0.4450.7580.8520.685
w/o curate0.4480.7710.8590.693
w/o read & curate0.4130.7460.8440.668
Add-only curate0.4570.7820.8620.700
ActICL0.4680.7880.8700.709

W3: It is unclear how the quality of training ICL demonstrations is ensured.

We address this question from three aspects: why ActICL does not require every demonstration to be helpful, why we include potentially misleading demonstrations in the bank, and how they benefit training. The ActICL pipeline does not require every demonstration in the bank to help the reasoning process. Our training is on-policy. In RL, the agent explores the bank and constructs its own context, and the SFT trajectories are also generated by the same backbone rather than constructed by humans. The answer-quality reward is computed on the answer produced with the committed context. Thus, the training signal reflects whether the context as a whole helps the model reason. For this reason, our training data does not need to exclude demonstrations that have correct answers but may still mislead the reasoning process. We include them in the bank so that the agent learns to judge demonstrations robustly by whether they help its reasoning, and this training data design is one of the contributions of our work. The similarity-based sampling route adds demonstrations that are close to the query and have correct answers but may still mislead the model, a kind of noise that is likely in real-world scenarios and that requires ICL to go beyond textual or visual similarity. During training, rollouts misled by these demonstrations receive lower rewards than those that avoid them, which gives a clear positive advantage to leaving them out of the context. Moreover, in our preliminary exploration of the data recipe, we found that including such demonstrations helps the model better learn to revise its existing context via the action we defined, especially curate.

Dear Reviewer i6XC,

There are only 3 days left in the discussion period. In our response, we provide further attribution of the performance gains, more detailed action-level ablation studies, and clarifications regarding the quality of the training data to address your concerns. We would be grateful if you would consider revisiting your score in light of our responses. If you have other unresolved questions, please let us know. We would be happy to engage in the days that remain. Thank you again.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the rebuttal

Dear Area Chair,

We sincerely thank you and the reviewers for the constructive discussion. We summarize it below to assist your decision.

Overall status. Reviewers i6XC and jd6A both rate the paper as Weak Accept. Reviewer i6XC raised the rating after the rebuttal, and jd6A maintained the positive rating after stating that most concerns were resolved. Reviewer 2uRJ maintains Weak Reject but explicitly acknowledges that "the added comparisons and ablations support the effectiveness of the individual components and the overall framework."

Strengths recognized by reviewers. All three reviewers find the paper clear and well organized, and they describe the evaluation as comprehensive (i6XC, 2uRJ). Reviewer jd6A states that the paper "introduces a novel agentic framework that transforms traditional one-shot demonstration retrieval into an iterative, reasoning-driven process," and highlights that the learned acquisition generalizes to stronger, unseen answering models.

Concerns resolved during the rebuttal.

  • Attribution of the gains (i6XC, jd6A). We added two controls. The Tool-use baseline gives the untrained model the same tools and environment. We also evaluated ActICL with the original, untrained model as the answerer. With the answering model fixed, ActICL outperforms Tool-use by 0.9–1.3 points (4-shot) and 1.4–2.8 points (adaptive). It also outperforms LSD, a trained selector, by 1.8 points on average, and the strongest prior baseline by 1.3–1.8 points. ProMSA, trained on the same data at a comparable cost, trails ActICL by 2.8–3.2 points.
  • Action-level ablations and one-step selection (i6XC). Every action matters. Replacing goal-conditioned search with random sampling costs 5.0 points, and one-step selection costs 4.1 points (Table 5).
  • Inference cost (jd6A). We report a detailed analysis of end-to-end latency and the cost–accuracy trade-off (Table 6). Adaptive ActICL is slightly faster than ProMSA (8.75 s vs. 8.87 s per query) while being 4.4 points more accurate.
  • Failure and empty-context rates (jd6A). We report per-benchmark failure and empty-context rates with a detailed analysis (Table 18). Acquisition failures are rare, at 3.0–5.8% on average, and empty contexts are justified decisions.
  • Generality (2uRJ). Our original three backbones already establish generality. They span a dense model (Qwen3-VL-8B), a Mixture-of-Experts model (Qwen3-VL-30B-A3B) and a hybrid linear-attention model trained natively on interleaved multimodal data (Qwen3.5-9B), so they differ in both architecture and training recipe. We further added InternVL3.5-8B from another model family, on which ActICL outperforms all baselines on all seven benchmarks (Table 14).
  • Related work (2uRJ). Three of the four works the reviewer suggested study learning from an agent's own interaction experience, which is a different problem from ICL with an external demonstration bank. The fourth, AutoSelect, is a text-only trained selector, and we have added it to the related work.

On the remaining novelty concern. Reviewer 2uRJ views ActICL as integrating existing ideas, while acknowledging that prior works "do not necessarily cover ActICL's complete framework." To the best of our knowledge, ActICL is the first post-training method for agentic multimodal in-context learning. Beyond providing an effective framework at the engineering level, it contains several algorithmic innovations that have not been explored by prior work. The two most representative are:

  1. Context update. In multimodal ICL, the value of a demonstration depends on the other demonstrations in the context. ActICL is the first to let the policy revise an already-built ICL context according to these interactions, a capability that nearly all existing demonstration selection methods lack. It decouples inspection from retention, so a demonstration that is later subsumed or revealed as misleading by the others can be replaced. Ablations confirm that removal and replacement contribute beyond addition alone.
  2. Credit assignment specific to agentic ICL. The committed context becomes the input of the agent's own subsequent answering call, and the two share parameters. We assign credit to acquisition and answering separately. A single policy trained this way supports both fixed-shot and adaptive multimodal ICL.

We believe these designs, together with the consistent gains across four backbones and the transfer of ActICL's contexts to frontier answering models, constitute a meaningful methodological contribution to multimodal ICL and broader related fields. Thank you for your time and consideration.

Best regards,

The Authors of Submission 36310

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.