acceptodds
Under review as a conference paper at ICLR 2027

Towards Multimodal Generative Active Learning: Efficient Learning via Uncertainty Dynamics

Abstract

Active learning (AL) aims to improve data efficiency by selectively querying the most informative samples for annotation in deep learning. However, existing AL research focuses predominantly on unimodal and multimodal discriminative learning, leaving multimodal generative learning largely unexplored. We introduce the first framework for , where the learner must actively acquire textual ground-truth for multimodal inputs rather than conventional labels. This setting differs fundamentally from existing multimodal AL paradigms, which typically rely on predefined labels obtained through annotation, an assumption that does not hold in multimodal generation. To this end, we propose an innovative AL algorithm that jointly exploits uncertainty and diversity through uncertainty dynamics, facilitating effective and representative sample selection with improved acquisition efficiency. Empirical results on benchmark datasets demonstrate that our approach substantially improves annotation efficiency, achieving comparable performance with up to 33% fewer annotated samples.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Active Learning, Large Language Models, Generative AI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_R6RYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a multimodal generative active learning setting in which the learner actively selects multimodal inputs whose ground-truth textual responses should be acquired under a limited annotation budget. To address the high computational cost of conventional generation-based uncertainty estimation, the authors propose an uncertainty-dynamics score computed from token-level entropy at two language-model layers during the prefill stage. The resulting score is used to construct a candidate set, followed by a diversity-aware coreset selection step based on multimodal hidden representations. The method is evaluated on VQAv2, ChartQA, and MMBench using SmolVLM-256M and, in an additional experiment, SmolVLM-2.2B. The reported results suggest improved annotation efficiency and reduced selection time compared with random, diversity-only, and generation-entropy-based uncertainty baselines.

Critical strengths
  1. Extending active learning from classification-oriented multimodal tasks to multimodal response generation is practically relevant, since acquiring high-quality free-form multimodal annotations can be substantially more expensive than collecting raw image--text inputs. The distinction between selecting conventional class labels and acquiring generative textual supervision is meaningful and could motivate further work in data-efficient multimodal learning.
  2. Estimating uncertainty during the prefill stage avoids autoregressive decoding for every unlabeled example, and the subsequent diversity selection is restricted to a reduced candidate pool. The runtime experiments indicate that this design can substantially reduce acquisition cost relative to a generation-based uncertainty baseline, which is important for active learning with multimodal language models.
Critical weaknesses
  1. The proposed ``uncertainty dynamics'' is not yet sufficiently justified as a measure of uncertainty for multimodal generation. The acquisition signal is constructed from the entropy of ϕ(Hi,t(q))\phi(H_{i,t}^{(q)}) at two intermediate/deep layers during prefill, followed by a finite-difference approximation, $ u_i' = -\frac{U_i^{(l)} - U_i^{(l-h)}}{h}. $ However, these entropies are computed on representations of the observed input/prompt tokens rather than on the future autoregressively generated response. It is therefore unclear why this quantity should reliably capture uncertainty about the unknown ground-truth answer rather than difficulty in representing the input sequence itself. The reported correlation with average layer-wise entropy mainly establishes agreement with another internal entropy statistic; it does not establish that the score is better aligned with generation error, expected model improvement, or annotation utility. This is consequential because uncertainty dynamics is the principal methodological contribution of the paper.
  2. The main experiments compare against only three relatively simple baselines: random selection, k-center diversity selection, and a mean-token-entropy uncertainty baseline. The proposed method itself is essentially a combination of uncertainty-based screening and standard coreset selection, making comparison with stronger hybrid or batch active learning approaches particularly important. Moreover, the main results are averaged over only four runs, several improvements appear relatively small compared with the reported variation, and no statistical significance analysis is provided. The claim of up to 33.33%33.33\% annotation savings is based on matching performance across different acquisition iterations rather than on a statistically supported sample-efficiency criterion. Consequently, it is difficult to determine whether the observed gains reflect a robust methodological improvement or a favorable combination of the selected baselines and hyperparameters.
Questions for authors
  1. Can the authors provide evidence that the proposed prefill-based uncertainty-dynamics score is directly related to uncertainty of the generated answer? For example, how strongly does sis_i correlate with per-example generation loss, answer correctness, sequence-level entropy, or expected performance improvement after the example is annotated? Such an analysis would materially strengthen the interpretation of the proposed acquisition signal.
  2. Why is the average entropy across all layers treated as the reference ``full uncertainty dynamics''? Averaging layer-wise uncertainty removes the actual trajectory of uncertainty across depth and does not necessarily represent dynamics. Would ranking samples using the full vector (Ui(1),Ui(2),…,Ui(L))\left(U_i^{(1)}, U_i^{(2)}, \ldots, U_i^{(L)}\right) or its slope/variation statistics lead to the same conclusions?
  3. Could the authors compare against stronger hybrid batch active-learning methods adapted to this setting, rather than only uncertainty-only and diversity-only baselines? In particular, it would be important to determine whether the improvement comes from the proposed layer-wise uncertainty signal or simply from combining uncertainty filtering with a standard k-center selection stage.
  4. Are the reported 33.33%33.33\% annotation savings statistically significant across random runs? Please report confidence intervals or appropriate paired statistical tests at matched annotation budgets. This is important because several differences between methods appear modest relative to the run-to-run variation.
4 comments

Dear reviewer,

We greatly appreciate your thoughtful feedback and constructive suggestions. Below, we provide detailed responses and clarifications. If our responses address your concerns, we would be grateful if you would consider updating your scores. Please let us know if further clarification is needed.

Response to Weakness 1: Thank you for this insightful comment. From an algorithmic perspective, our method is not designed to establish a direct connection between the uncertainty-dynamics score and the quality of future autoregressively generated responses. Instead, it is motivated by the trade-off between computational efficiency and final performance. By focusing on signals available during the prefill phase, we may sacrifice some information available during generation in exchange for substantially improved efficiency while maintaining strong performance.

From an empirical perspective, we further investigate the relationship between our uncertainty-dynamics score and the expected performance improvement from annotating each example, despite relying solely on prefill-phase information during data selection. Specifically, we conduct experiments on ChartQA-8K using SmolVLM-256M. Starting from the same model checkpoint, we use our method and Uncertainty_gmtes to select samples independently, fine-tune the model on the selected samples, and compare the resulting models' performance on the same validation set. Although the ideal setup would involve selecting and training on a single sample at a time, the performance changes induced by individual samples can exhibit substantial variance. Therefore, to obtain more stable and reliable results, we select 16 samples per method in each experiment. (At Step 1, we select 16 samples to fine-tune the model, while at Step 2, we select an additional 16 samples and continue fine-tuning from the model obtained at Step 1.) The results are presented below:

StepOurs Accuracy (%)Uncertainty_gmtes Accuracy (%)Ours Gain (pp)Uncertainty_gmtes Gain (pp)
151.367250.3906+2.9297+2.5391
252.539150.7813+1.1719+0.3906

Conceptually, our method selects the sample with the highest uncertainty-dynamics score (sis_i) for model fine-tuning, which we extend to the top 16 samples in our experiments. If sis_i is positively correlated with the expected performance improvement after annotation, the selected samples should, on average, yield greater improvements in validation performance. The results show that our method yields greater performance gains than the uncertainty-based baseline, suggesting that our score better captures potential training utility.

Response to Weakness 2: Thank you for raising this important point. As discussed in Section 2 of our manuscript, existing hybrid active learning methods cannot be directly applied to our multimodal generative setting; to the best of our knowledge, our method is among the first hybrid approaches developed for this setting. Furthermore, in Section 4.3.5, we introduce an additional hybrid baseline that combines uncertainty-based screening using only final-layer uncertainty with diversity-based selection, and our method consistently outperforms this baseline, demonstrating the benefit of incorporating layer-wise uncertainty dynamics.

To quantify the variability across random runs, we additionally report 95% confidence intervals (CIs) for all methods on MMBench at matched annotation budgets over four runs, as shown below. Results are presented as a mean ± 95% CI margin of error.

MethodsRound 0Round 1Round 2Round 3Round 4Round 5Round 6
Random0.677 ± 0.0150.698 ± 0.0190.701 ± 0.0240.723 ± 0.0190.732 ± 0.0170.746 ± 0.0130.750 ± 0.005
Diversity0.681 ± 0.0380.696 ± 0.0270.706 ± 0.0280.720 ± 0.0210.731 ± 0.0440.740 ± 0.0280.746 ± 0.043
Uncertainty_gmtes0.682 ± 0.0350.686 ± 0.0360.690 ± 0.0370.702 ± 0.0260.717 ± 0.0330.730 ± 0.0400.745 ± 0.024
Ours0.686 ± 0.0310.703 ± 0.0310.714 ± 0.0170.727 ± 0.0470.740 ± 0.0390.748 ± 0.0390.756 ± 0.031

Furthermore, to examine the reported 33.33% annotation savings, we conduct a paired statistical comparison across four matched random runs. Our method achieves a mean score of 0.703 using only 5% of annotated data, outperforming Uncertainty_gmtes (0.690) using 7.5%. The mean paired difference is 0.0133, with a 95% CI of [0.00003, 0.02661], excluding zero. This indicates a statistically significant performance improvement at the 5% level for this comparison, providing further evidence of the annotation efficiency of our method.

Response to Question 1: Thank you for raising this important question. Please refer to our response to Weakness 1, where we provide a detailed discussion from both motivational and empirical perspectives.

Response to Question 2: Thank you for raising this important question. As discussed in the main text, our algorithm aims to select samples with high uncertainty across layers. Motivated by this, we use the average entropy across all layers as a proxy for overall uncertainty dynamics and design our uncertainty-dynamics score to efficiently approximate it. We acknowledge that average entropy does not fully capture the layer-wise uncertainty trajectory. Investigating finer-grained dynamics, such as the rate of uncertainty changes across layers, is an important direction for our future work.

To further investigate whether alternative characterizations of layer-wise uncertainty dynamics lead to the same conclusions, we conducted additional experiments using slope and variation statistics for sample ranking on the VQAv2-6K dataset using the SmolVLM-256M model. The experimental results are reported below.

MethodsRound 0Round 1Round 2Round 3Round 4Round 5Round 6
Ours with slope0.560.560.560.570.600.630.71
Ours with variation0.560.560.560.570.590.620.68
Original Ours0.570.570.560.570.590.640.73

The results show that our original method achieves the better final performance, outperforming both slope-based and variation-based alternatives. Although these alternative statistics capture finer-grained uncertainty dynamics, they do not lead to better performance in our experiments. These findings support our design choice of using overall layer-wise uncertainty as an effective and computationally efficient proxy for sample selection. Nevertheless, more sophisticated characterizations of layer-wise uncertainty dynamics may further improve performance, which we leave for future work.

Response to Question 3: Thank you for raising this important concern. As discussed in Section 2.2, existing hybrid AL methods, such as BADGE, rely on classification-oriented gradient representations, making their direct adaptation to multimodal generation non-trivial. Our work aims to establish an initial hybrid acquisition baseline for this setting.

For the empirical perspective, we have already investigated whether the improvements arise merely from combining uncertainty filtering with k-center selection. Specifically, setting (λ\lambda=0) in Section 4.3.5 provides a standard hybrid baseline that retains uncertainty filtering and k-center selection while excluding the proposed layer-wise uncertainty dynamics signal. As shown in Figure 6 (in our manuscript), incorporating this signal improves performance from 0.70 to 0.73, providing empirical evidence of its additional contribution beyond the hybrid combination itself.

Response to Question 4: Thank you for raising this important question. Please refer to our response to Weakness 2, where we provide a detailed discussion supported by additional statistical significance analyses.

Dear Reviewer R6RY,

Thank you for your valuable feedback and thoughtful suggestions. We have submitted our rebuttal and provided further clarifications in response to your concerns.

As the discussion period is coming to a close, we would be grateful if you could revisit our responses and share any remaining concerns. We hope that our clarifications and additional evidence have adequately addressed your questions, and we would sincerely appreciate your reconsideration of the evaluation and scores if you find our responses satisfactory.

Thank you again for your time and consideration. We truly appreciate your efforts in reviewing our work!

I thank the authors for their constructive response and additional experiments, which have improved the empirical presentation, especially the confidence-interval and paired-comparison analyses. However, the main concerns remain insufficiently addressed: the prefill-based uncertainty-dynamics score is still not directly validated against generation loss, answer correctness, sequence-level entropy, or per-example expected improvement; the justification for average layer-wise entropy as a proxy for “full uncertainty dynamics” remains limited; stronger adapted hybrid/batch active-learning baselines are missing; and the statistical analysis covers only a narrow set of comparisons and runs. I therefore recommend keep the original score.

Reviewer_4D6DOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This article introduces a new method for Active Learning specifically targeting generative models with multimodal inputs. It relies on an hybrid approach, using both intra-model entropy dynamics and sample diversity to pick the samples for ground-truth labelling. Findings are supported by experiments on two models and three datasets.

Critical strengths
  1. The authors clearly state the problem targeted and motivate the pitfalls of existing approaches.

  2. The method covers both closed and open-ended settings.

Critical weaknesses
  1. The scope of the experiments is too narrow. The main results rely exclusively on SmolVLM-256M and do not show significant improvement over the baselines.

  2. The values for BB and λ\lambda seem to be arbitrarly set. The hyperparameters reported don't show a clear pattern on how they were selected, leaving uncertainty as how would a user pick them in practice. The authors conduct ablation studies for these parameters but results are not readable. The value of hh also appears to be different between ChartQA-8K and full ChartQA while they use the same model.

  3. The computational cost of this method is under-considered. Projecting layer activations to the vocabulary space sample can be expensive, the authors should report more precisely the computation time needed, with regard to the overall training time.

Questions for authors
  1. Could the authors provide experiments using another model?

  2. Could the authors add results using a baseline whose data selection process is quicker than the ones considered (e.g. random or last-token entropy)?

  3. The relation between BB and KK is unclear. KK should be smaller than BB as $\mathcal{C} \subseteq \mathcal{P}_r \implies K = |\mathcal{C}| \leq B = |\mathcal{P}_r| $, yet Table 5 indicates the opposite. This needs clarification.

  4. Why did the authors disregard the method proposed by Zhang et al. (2024) as a classification one, whereas it seems applicable in this case and shares a similar construction?

4 comments

Dear reviewer,

Thank you for taking the time to review our paper and provide thoughtful comments. We have outlined our responses and clarifications below. We hope that our efforts have addressed your concerns, and we kindly request that you consider increasing your scores.

Response to Weakness 1: Thank you for this valuable comment. To address the concern regarding the limited experimental scope, we conduct additional experiments on the ChartQA-8K dataset using the LLaVA-OneVision model to evaluate the generalizability of our method across different model architectures. For a fair comparison, we maintain the same per-round annotation budget as in the main experiments. We compare our method against the baselines and report the mean accuracy and CIDEr scores over four random runs. The results are presented below:

MethodsRound 0Round 1Round 2Round 3Round 4Round 5Round 6
Random0.680.670.690.690.690.700.70
Diversity0.680.680.680.680.690.690.70
Uncertainty_gmtes0.680.680.680.680.680.700.70
Ours0.680.680.700.700.700.700.70

The results show that our method can generally outperform the baselines on LLaVA-OneVision, achieving better performance. These results suggest that our method can generalize across different model architectures, providing additional empirical evidence of its robustness. We acknowledge that the performance improvements over the baselines are relatively modest; however, such differences are not uncommon in active learning, as illustrated by some of the empirical comparisons reported in the well-established paper BADGE [1]. We agree that achieving larger and more consistent performance gains is an important direction for future work.

Response to Weakness 2: Thank you for raising this concern. We agree that the hyperparameter selection guidelines and ablation results could be presented more clearly, and we provide a more detailed explanation here to clarify these points. As discussed in Sections 3.1–3.2, B controls the trade-off between uncertainty and diversity, while λ determines the contribution of layer-wise uncertainty evolution.

Our ablation studies in Sections 4.3.4–4.3.5 provide empirical guidance for their selection. In particular, we observed that excessively large values of B are ineffective for this dataset. Therefore, we focused our hyperparameter search on B ∈ {3, 4, 5} and selected B = 4 based on the empirical results. Moreover, incorporating uncertainty dynamics (λ > 0) improves performance over λ = 0. In practice, a coarse grid search can first be used to identify a promising range for B, followed by a joint search over B and λ within this range to determine the best hyperparameter combination.

Regarding h, although ChartQA-8K and full ChartQA use the same model architecture, they differ in dataset size and per-round annotation budget, which may affect the model's layer-wise uncertainty patterns. In practice, we consistently select one relatively shallow layer and one deeper layer, making extensive tuning of h generally unnecessary. Our experiments show that the performance remains comparable when varying the specific layer choices within these shallow and deep regions (e.g., selecting layer 2 or 3 as the shallow layer). We report the best-performing configurations for each dataset.

Response to Weakness 3: Thank you for this thoughtful comment. We clarify that the data selection time reported in Figure 6 (right) covers the entire selection pipeline, including model forward passes, hidden representation extraction, vocabulary-space projections, uncertainty score computation, and sample selection. Thus, the computational cost of vocabulary-space projections is already included in our measurements. Furthermore, the remaining costs, mainly model fine-tuning and evaluation, are approximately the same across all four methods.

To provide a more precise breakdown of the computational cost of our method, we ran the experiments on VQAv2-6K using the SmolVLM-256M and measured the time required for (1) intermediate-layer vocabulary projection: 2.58s and (2) the complete data selection process in each acquisition round: 320.09s. The vocabulary projection accounts for only 0.81% of the total selection time, indicating that it does not introduce a substantial computational bottleneck. We will clarify these details in the revised manuscript.

Response to Question 1: Thank you for your question. We conducted additional experiments using a model from a different VLM family. Please refer to our response to Weakness 1 for a detailed discussion of the experimental results.

Response to Question 2: Thank you for raising this question. We would like to clarify that our main paper already includes Random, a baseline with minimal data selection overhead. To further address this concern, we additionally evaluate Last-token entropy, a computationally lighter uncertainty-based baseline that computes entropy only at the final input token, without modeling layer-wise uncertainty dynamics. We perform the experiments on VQAv2-6K using the SmolVLM-256M model over four runs and record the mean of accuracy and CIDEr score here. The results are reported below.

MethodsRound 0Round 1Round 2Round 3Round 4Round 5Round 6
Last-token entropy0.5650.5560.5640.5810.5890.6340.684
Ours0.5660.5650.5570.5710.5890.6410.726

Although Last-token entropy achieves competitive performance in some early rounds, our method demonstrates stronger performance in later rounds, achieving 0.726 versus 0.684 in Round 6. These results suggest that incorporating layer-wise uncertainty dynamics provides additional benefits over relying solely on last-token entropy.

Response to Question 3: Thanks for your question. We acknowledge that there is a typo in the manuscript. Specifically, we select the top-(B ×\times K) data points based on their uncertainty-dynamics scores. We will correct this typo and clarify the selection procedure in the revised manuscript.

Response to Question 4: Thanks for your question. Unfortunately, possibly due to a technical issue with the review system, we are currently unable to access the full reference to Zhang et al. (2024). As discussed in Section 2 of our manuscript, we explain why conventional active learning methods designed for classification tasks cannot be directly applied to our multimodal generative setting. We would greatly appreciate it if you could provide the specific reference to Zhang et al. (2024). We would carefully examine the proposed method and further investigate its applicability to our setting.

REFERENCES

[1] Jordan T. Ash, et al. "Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds" International Conference on Learning Representations. 2020.

Dear Reviewer 4D6D,

Thank you for your valuable feedback and thoughtful suggestions. We have submitted our rebuttal and provided further clarifications in response to your concerns.

As the discussion period is coming to a close, we would be grateful if you could revisit our responses and share any remaining concerns. We hope that our clarifications and additional evidence have adequately addressed your questions, and we would sincerely appreciate your reconsideration of the evaluation and scores if you find our responses satisfactory.

Thank you again for your time and consideration. We truly appreciate your efforts in reviewing our work!

Dear authors,

Thank you for the response and the additional experiments. My main concerns remain, however, and I will keep my rating.

The new results confirm that the gains are small: on LLaVA-OneVision all methods end at 0.70 and never differ by more than 0.02, and against last-token entropy the advantage comes almost entirely from the final round. Without standard deviations over the four runs, I cannot tell whether these differences are significant. The hyperparameters still appear to be chosen by a per-dataset grid search with the best configuration reported, which gives a user little guidance, and the selection cost relative to the overall training time is still not given.

Regarding Zhang et al. (2024), I was refering to the entry which is in your bibliography: Yiqian Zhang, Buyu Liu, Jun Bao, Qiang Huang, Min Zhang, and Jun Yu. Learnability matters: active learning for video captioning. Advances in Neural Information Processing Systems, 37: 37928–37954, 2024

I appreciate the effort put into the rebuttal and hope these comments help with a future version of the paper.

Reviewer_FVqBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an active learning method for multimodal question answering that selects annotation candidates using estimated changes in token entropy across Transformer layers, followed by diversity-based coreset selection.

Critical strengths
  1. The paper addresses a practical problem. Reducing annotation cost for MLLM training is important, particularly when collecting high-quality responses requires substantial human effort.
  2. The method combines uncertainty and diversity in a simple framework. The two-stage design is easy to implement and can be applied to different MLLMs.
  3. The experiments show improvements across several settings.
Critical weaknesses
  1. If I understand correctly, the method computes entropy over prefilled input tokens instead of the unknown output response, so the resulting quantity may reflect the predictability of the question or prompt instead of uncertainty about the correct answer. It would be better to establish why this surrogate should measure the expected informativeness of acquiring the corresponding response.
  2. The authors apply the language modeling head to hidden representations from intermediate Transformer layers, although these representations are not necessarily trained to produce calibrated token probabilities. It would be better to elaborate more on the insight of using intermediate layers.
  3. Fundamental question, the proposed score rewards high final layer entropy and large entropy reduction across layers, but neither property guarantees that labeling the example will improve the model. In particular, high uncertainty may result from ambiguity or irreducible uncertainty rather than insufficient model knowledge.
Questions for authors
  1. What theoretical or probabilistic relationship connects input token entropy and its evolution across Transformer layers to uncertainty about the unknown target response?
  2. Why should entropy changes across Transformer layers indicate learning informativeness?
3 comments

Dear reviewer,

We sincerely thank the reviewer for the valuable feedback and constructive suggestions. Below, we provide our responses and clarifications. We hope these help address the reviewer’s concerns.

Response to Weakness 1: Thank you for raising this point. From an algorithmic perspective, our method is not designed to establish a direct connection between the uncertainty-dynamics score and the quality of future autoregressively generated responses. Instead, it is motivated by the trade-off between computational efficiency and final performance. By focusing on signals available during the prefill phase, we may sacrifice some information available during generation in exchange for substantially improved efficiency while maintaining strong performance.

From an empirical perspective, we further investigate the relationship between our uncertainty-dynamics score and the expected performance improvement from annotating each example, despite relying solely on prefill-phase information during data selection. Specifically, we conduct experiments on ChartQA-8K using SmolVLM-256M. Starting from the same model checkpoint, we use our method and Uncertainty_gmtes to select samples independently, fine-tune the model on the selected samples, and compare the resulting models' performance gains on the same validation set. Although the ideal setup would involve selecting and training on a single sample at a time, the performance changes induced by individual samples can exhibit substantial variance. Therefore, to obtain more stable and reliable results, we select 16 samples per method in each experiment. (At Step 1, we select 16 samples to fine-tune the model, while at Step 2, we select an additional 16 samples and continue fine-tuning from the model obtained at Step 1.) The results are presented below:

StepOurs Accuracy (%)Uncertainty_gmtes Accuracy (%)Ours Gain (pp)Uncertainty_gmtes Gain (pp)
151.367250.3906+2.9297+2.5391
252.539150.7813+1.1719+0.3906

Conceptually, our method selects the sample with the highest uncertainty-dynamics score for model fine-tuning, which we extend to the top 16 samples in our experiments. If score is positively correlated with the expected performance improvement after annotation, the selected samples should, on average, yield greater improvements in performance. The results show that our method yields greater performance gains than the uncertainty-based baseline, suggesting that our score better captures potential training utility.

Response to Weakness 2: Thank you for this insightful comment. We acknowledge that intermediate hidden representations are not explicitly trained to produce calibrated token probabilities. However, prior work, Entropy-lens [1], applies the language modeling head to intermediate representations and demonstrates that layer-wise entropy changes reflect the expansion.

Inspired by this insight, we adopt a similar approach to characterize uncertainty evolution across Transformer layers. Rather than treating intermediate-layer distributions as calibrated probabilities, we use their entropy as a proxy for layer-wise uncertainty dynamics. By combining this signal with final-layer uncertainty, our method captures complementary information about the model's internal processing. Our ablation results (Please refer to section 4.3.5 for details) further support its effectiveness, showing that incorporating layer-wise dynamics (λ\lambda>0) improves performance over using final-layer uncertainty alone (λ\lambda=0).

Response to Weakness 3: Thank you for raising this important concern. We agree that neither high final-layer entropy nor large entropy reduction guarantees that labeling an individual sample will improve model performance from the theoretical perspective. Our method does not assume such a theoretical guarantee. Instead, we use these signals as complementary heuristics for identifying potentially informative samples. Specifically, final-layer entropy reflects the model's remaining predictive uncertainty, while layer-wise entropy reduction characterizes how the uncertainty evolves during processing. Their combination provides a richer selection signal than final-layer uncertainty alone.

To investigate whether our selection favors ambiguous samples, we conducted an exploratory automated agent audit of the top-10 ChartQA samples selected by each method across four random runs. Within each run, all methods used the same model checkpoint and candidate pool, while method identities and reference answers were hidden from the auditor. Using visual transcriptions, a single language model conducted a three-stage assessment consisting of interpretation review, evidence review, and final adjudication. Among successfully audited samples, 59.0% of those selected by our method were flagged as ambiguous, compared with 76.3% for Uncertainty_gmtes, a difference of 17.3 percentage points. These preliminary findings suggest that our selection strategy may be less prone to selecting ambiguous samples than conventional uncertainty-based sampling. (The criteria for this exploratory automated audit were informed by [2] and [3], adapting their treatment of ambiguity in visual questions and context underspecification to assess the selected samples.)

Response to Question 1: Thank you for your questions. Please refer to our response to Weakness 1 for a detailed discussion from both algorithmic and empirical perspectives.

Response to Question 2: Thank you for this important question. Prior work Entropy-Lens[1] demonstrates that entropy computed from intermediate-layer representations via the LM head captures meaningful token prediction dynamics, including candidate expansion and pruning. Motivated by this observation, we use layer-wise entropy reduction to characterize how the model refines its predictions across depth, complementing the residual uncertainty captured by final-layer entropy. We hypothesize that combining these signals provides a useful heuristic for identifying informative samples, rather than a theoretical guarantee of learning informativeness. Our empirical results further support its effectiveness for active learning.

REFERENCES

[1] Ali, Riccardo, et al. "Entropy-Lens: Uncovering Decision Strategies in LLMs." arXiv preprint arXiv:2502.16570 (2025).

[2] Pu Jian, et al.“Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions”Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025

[3] Zongwan Cao, et al.”Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specification”arXiv preprint arXiv:2601.16400 (2026).

Dear Reviewer FVqB,

Thank you for your valuable feedback and thoughtful suggestions. We have submitted our rebuttal and provided further clarifications in response to your concerns.

As the discussion period is coming to a close, we would be grateful if you could revisit our responses and share any remaining concerns. We hope that our clarifications and additional evidence have adequately addressed your questions, and we would sincerely appreciate your reconsideration of the evaluation and scores if you find our responses satisfactory.

Thank you again for your time and consideration. We truly appreciate your efforts in reviewing our work!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.