acceptodds
Withdrawn from ICLR 2027

TIMBRE: Reliability-Aware Harmonic Consensus for Label-Free Multimodal Test-Time Reinforcement Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Label-free test-time reinforcement learning adapts vision–language models using rewards derived from their own predictions. However, misinterpreted visual evidence and sensitivity to textual formulation can produce a consistent but incorrect consensus that majority voting reinforces. Cross-view aggregation can expose such errors, but image and text transformations may themselves degrade task-relevant evidence to different extents. We introduce Timbre, a reliability-aware harmonic consensus framework for multimodal test-time reinforcement learning. Timbre constructs a grid combining original and perturbed images with direct-solving and reframing modes, selects pseudo-labels through weighted harmonic aggregation, and adapts modality-level weights using batch-averaged, temporally smoothed shifts in pseudo-label confidence as a heuristic reliability signal. We formulate consensus as a weighted -barycenter problem whose unique minimizer is the normalized weighted harmonic aggregate, without requiring correct-answer invariance. An exact support–dispersion decomposition characterizes when harmonic aggregation preserves or corrects arithmetic rankings, while a sensitivity analysis establishes how modality reweighting changes pairwise ranking margins, informing interpretation of the heuristic updates. Across six benchmarks and three Qwen3-VL model scales, Timbre outperforms TTRV in 16 of 18 benchmark–model settings on adaptation instances and 17 of 18 settings on held-out instances under matched estimation and optimization rollout counts. These results demonstrate improved target-task adaptation and transfer to unseen instances without further updates.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
test-time reinforcement learning, test-time adaptation, vision-language models, pseudo-labeling, multimodal reasoning, label-free learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bjbROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TIMBRE, which builds four views by combining original/perturbed images with direct-solving/reframing prompts. It selects pseudo-labels using weighted harmonic aggregation, while the view weights are updated based on how much the confidence of the current pseudo-label changes after image or text transformations. The paper also gives a theoretical interpretation of this aggregation through a weighted Pearson-X^2 barycenter and analyzes its ranking behavior. Experiments on six multimodal reasoning benchmarks with Qwen3-VL 2B/4B/8B show that TIMBRE generally outperforms TTRV under matched rollout counts, on both adaptation and held-out examples.

Critical strengths
  • The problem is important and well motivated. Wrong consensus is a real issue for label-free test-time RL, and this problem becomes even more serious for VLMs because visual errors can be very consistent across rollouts.

  • The theory is cleaner in this kind of work. The paper gives a clear optimization interpretation and some useful ranking analysis.

  • The main evaluation is reasonably broad. The paper covers six benchmarks and three model sizes, and it also reports both adaptation-set and held-out performance rather than only showing test-time fitting.

Critical weaknesses
  • The “reliability” signal is not fully convincing to me. The method basically treats confidence stability as reliability. But a transformed view may change the confidence exactly because it corrects a mistake in the original view. In that case, TIMBRE may actually downweight the useful view.
  • The Adaptive Weighting does not seem very strongly supported by the ablation. In Table 2, weighted harmonic aggregation is only slightly better than uniform harmonic aggregation, and on DynaMath, the Val-TTT score is even identical. This makes me wonder whether most of the gain is simply from using multiple views + harmonic aggregation.
  • It would be useful to separate the effects of image perturbation, text reframing, four-view construction, harmonic aggregation, adaptive weighting, and the diversity bonus.
Questions for authors
  • Have you analyzed cases where the original view is wrong but the transformed/reframed view becomes correct? In those cases, does the confidence-shift heuristic tend to downweight the correcting view?
  • Figure 3 seems to show that the learned weights stay fairly close to uniform. How much performance is actually lost if the adaptive weighting is completely removed on all six benchmarks?
  • How much extra compute does TIMBRE use compared with TTRV in terms of generated tokens, GPU time, or wall-clock time? A token-budget-matched comparison would make the efficiency story much clearer.
Reviewer_SLAXOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies label-free multimodal test-time reinforcement learning. It combines original images, perturbed images, direct solving, and reframed solving into four views. It uses harmonic aggregation to produce pseudo-labels, and adjusts the view weights according to changes in confidence across views.The paper evaluates Qwen3-VL models at 2B, 4B, and 8B scales on several visual-mathematical and multidisciplinary tasks. It also includes ablations with weighted arithmetic aggregation and uniform harmonic aggregation.

Critical strengths
  • The use of four views has a clear motivation. Image perturbations and textual reframing can indeed lose different amounts of task-relevant information.
  • TIMBRE outperforms TTRV in most reported settings. The paper also includes weighted-arithmetic and uniform-harmonic ablations rather than comparing only against a single weak baseline.
Critical weaknesses
  • The independent contribution is limited. Self-Harmony already uses harmonic aggregation over original and reframed views to construct pseudo-labels. TIMBRE mainly adds image views and reliability weights. Since the paper does not directly compare with Self-Harmony under the same budget, it is unclear whether these additions provide a new insight.
  • The incremental gains are small, and no clear scaling effect is shown. In Table 2, TIMBRE improves over weighted arithmetic aggregation by only 0.1 percentage points on the held-out results. Its gains over uniform harmonic aggregation are 0.5 and 0.2 points. Based on Table 1, the average six-task held-out gains over TTRV are approximately 2.08, 0.55, and 0.73 points for the 2B, 4B, and 8B models, respectively. The advantage does not grow consistently with model size. On MMMU-Pro (Science), the 8B result is also below both TTRV and the pre-adaptation model.
Questions for authors
  1. Under the same rollout and optimization budgets, could you directly compare the two-view Self-Harmony procedure with the four-view TIMBRE procedure? All non-core settings should be kept fixed.
  2. Beyond proving mathematical properties of weighted harmonic aggregation, what new, testable prediction does the paper make about when a view should receive less weight? Please separately analyze beneficial corrections and harmful perturbations.
  3. Could you independently repeat the test-time adaptation process for the 2B, 4B, and 8B models, report confidence intervals for the incremental gains, and explain why the gains for 4B and 8B are not larger than those for 2B?
Reviewer_ejGvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TIMBRE addresses label-free test-time reinforcement learning for vision-language models by combining four views: original or perturbed images with direct-solving or reframe-and-solve prompts. It selects pseudo-labels using smoothed weighted harmonic consensus, updates modality-level weights from temporally smoothed shifts in pseudo-label support, and applies pooled GRPO advantages with a confidence-gated reframing bonus. The theory gives an exact Pearson chi-squared barycenter characterization, a support-dispersion decomposition, and conditional ranking and weight-sensitivity results. Across six benchmarks and Qwen3-VL models at 2B, 4B, and 8B, the method exceeds TTRV in 16 of 18 adaptation settings and 17 of 18 held-out settings, with matched totals of 64 estimation and 32 optimization rollouts per instance. The main unresolved issue is whether the gains specifically validate the proposed adaptive reliability mechanism.

Critical strengths
  1. Lemma 4.1 and Proposition 4.2 explain exactly what harmonic aggregation optimizes and how it discounts cross-view dispersion. Theorem 4.4 identifies when changing a modality weight improves a fixed pairwise margin (§4, pp. 6-7; Appendix A, pp. 13-16). The proofs appear correct under their stated assumptions. The paper also explicitly acknowledges that stability is not correctness, making the method interpretable without overstating the theory as a general accuracy guarantee.
  2. Table 1 covers six benchmarks and three model scales, separately evaluating adaptation instances and held-out instances without further updates. It supports the 16/18 and 17/18 win counts over TTRV. Table 2 compares arithmetic versus harmonic aggregation and uniform versus adaptive weighting with the same four views and optimization settings. These controls, alongside disclosed prompts and most training settings, provide a credible starting point for assessing the method.
Critical weaknesses
  1. Equations (4)–(6) use absolute confidence shifts, so beneficial corrections and harmful perturbations can both reduce the corresponding reliability factor. Moreover, initialization at 1 and exponential targets no larger than 1 imply that both factors remain at most 1: the original-image/direct-solving view always has the largest weight, and the doubly transformed view the smallest. Theorem 4.4 gives a correct-answer-dependent condition under fixed distributions; it does not show that the actual heuristic meets that condition. The authors acknowledge the ambiguity, but Appendix B.4 provides only one qualitative DynaMath/8B trajectory. Without a fixed original-favoring weight control and quantitative evidence connecting updates to improved pseudo-label correctness, the central reliability claim remains only partially validated. 2.Section 3.1 identifies the original-image/direct-versus-reframed pair as a multimodal Self-Harmony variant, yet no matched-budget two-view result is reported. Table 2 retains all four views, so it cannot establish the incremental benefit of adding image perturbations. Its held-out gains are also small: weighted harmonic exceeds weighted arithmetic by 0.1 percentage point on both MathVista and DynaMath, and exceeds uniform harmonic by 0.5 and 0.2 points, respectively; DynaMath adaptation ties uniform harmonic at 69.2. No repeated-training results or uncertainty intervals are reported. Eight evaluation samples per instance do not measure training variability. These omissions limit how confidently the observed gains can be attributed to the proposed additions.
Questions for authors
  1. Does adaptive weighting improve pseudo-label accuracy and final performance over fixed original-favoring weights, with the four views and rewards unchanged? Using labels only for post-hoc analysis, how often do actual updates help versus harm the consensus decision, especially when a transformed view is more accurate?
  2. Can the authors provide a faithful two-view multimodal Self-Harmony comparison with the same total 64 estimation and 32 optimization rollouts, and repeated-training results for the key Table 2 comparisons? Means, run-to-run variation, and paired uncertainty estimates at the problem level would help determine whether the small held-out differences are robust.
  3. What numerical smoothing value alpha is used in Eq. (2), and what are the image-perturbation ranges and Gaussian noise scale in §3.1? These are absent from Table 4 and directly affect reciprocal scores estimated from only 16 rollouts per view. Please also specify the split identifiers or seeds and whether Table 1 uses the final checkpoint or a selected checkpoint; if selected, what rule preserves the label-free protocol?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.