acceptodds
Withdrawn from ICLR 2027

TTA-Vid: Generalized Test-Time Adaptation for Video Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent video reasoning models have shown strong results on temporal and multimodal understanding, yet they depend on large-scale supervised data and multi-stage training pipelines, making them costly to train and difficult to adapt to new domains. In this work, we leverage the paradigm of Test-Time Reinforcement Learning on video-language data to allow for adapting a pretrained model to incoming video samples at test-time without explicit labels. The proposed test-time adaptation for video approach (TTA-Vid) performs step-by-step reasoning at inference time on multiple frame subsets. We then use a batch-aware frequency-based reward computed across different frame subsets as pseudo ground truth to update the model. It shows that the resulting model trained on a single batch or even a single sample from a dataset, is able to generalize at test-time to the whole dataset and even across datasets. Because the adaptation occurs entirely at test time, our method requires no ground-truth annotations or dedicated training splits. Additionally, we propose a multi-armed bandit strategy for adaptive frame selection that learns to prioritize informative frames, guided by the same reward formulation. Our evaluation shows that TTA-Vid yields consistent improvements across various video reasoning tasks and is able to outperform current state-of-the- art methods trained on large-scale data. This highlights the potential of test-time reinforcement learning for temporal multimodal understanding.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Learning, Reinforcement Learning, Test-time Adaptation, Video Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7VD3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adapts a pretrained VLM to a video QA benchmark at test time without labels. For each of 32 unlabeled test questions, it samples K=4 subsets of F=4 frames, generates N=8 reasoning rollouts per subset, and rewards each rollout by the frequency of its extracted answer among all 32 rollouts minus an entropy penalty. The model is updated with GRPO for 5 epochs. The same reward updates per-frame sampling weights through a multiplicative-weights bandit initialized from CLIP scores, and the averaged distribution is used to select 32 frames at inference. The adapted InternVL3-8B reaches a 55.13 average over VideoMMMU, MMVU, SciVideoBench, Video-MME and LongVideoBench (base 51.37), above SFT+RL video reasoning models such as Video-R1 (51.94) and Video-R2 (53.92); the adapted Qwen2.5-VL-7B reaches 51.84 (base 48.82). Ablations report transfer across datasets, gains from as few as one adaptation sample, and higher accuracy with majority-vote rewards than with ground-truth rewards.

Critical strengths

A label-free adaptation signal that uses redundancy specific to video. Pooling answers across different frame subsets of the same video gives a consensus estimate that a single-image method such as TTRV cannot form. With InternVL3-8B, where the base and the adapted model both use 32 frames, the adapted model improves on all five benchmarks (average 51.37 → 55.13). Table 4 shows that adapting on one benchmark improves others: adapting on MMVU raises VideoMMMU from 49.33 to 53.66, and adapting on VideoMMMU raises SciVideoBench from 26.50 to 30.20. If these gains survive the controls requested below, a cheap label-free procedure that transfers across benchmarks would be useful in practice.

Critical weaknesses
  1. The experiments do not show that the gain comes from better video reasoning, and two results point to a change in output behavior. (a) Adapting Qwen2.5-VL-7B on a single unlabeled sample (V=1, Table 5) raises VideoMMMU-Perception from 58.33 to 67.00, more than three quarters of the gain obtained with 32 samples. A single question is unlikely to teach transferable video reasoning; it can, however, teach the model to finish within the length limit and to emit an answer that the extractor h parses. (b) Training with ground-truth answers gives 60.96 on MMVU, below the 65.11 obtained with majority-vote pseudo-labels. If the reward improved the model by teaching correct answers, true labels should not do worse. The paper reads both results as evidence of generalization but does not test the simpler explanation. The following are needed: the rate of unparseable or length-truncated outputs and the distribution over answer options, before and after adaptation; and a control that adapts on mismatched video–question pairs or on blank frames. If the gains persist under that control, they do not come from video understanding, and the title and main claim would need to change.
  2. The headline comparison is not like-for-like, and the protocol behind Table 1 is unclear. (a) The claim that TTA-Vid outperforms models trained on large-scale data holds only for InternVL3-8B. Video-R1, VideoChat-R1 and Video-RFT build on Qwen2.5-VL-7B; with that backbone TTA-Vid averages 51.84, below all five reasoning models that report an average (51.94–53.92), although it uses 32 frames against their 128. (b) Section 4.3 states that TTA-Vid "consistently improves the base backbones", but in the Qwen2.5-VL-7B rows LongVideoBench falls from 65.10 to 57.81. The base row's Video-MME (56.00) and LongVideoBench (65.10) values appear swapped: the Qwen2.5-VL technical report (Table 8) gives 65.1 on Video-MME without subtitles and 56.0 on LongVideoBench val. With the two values swapped back, Video-MME falls from 65.10 to 61.14. That base also uses up to 768 frames against TTA-Vid's 32. (c) It is unclear which adaptation set produced each number in Table 1. The InternVL3-8B VideoMMMU entry (53.66) equals the MMVU-adapted entry of Table 4, while the VideoMMMU-adapted (in-domain) entry is 51.48; Table 2 reports the same 53.66 as an in-domain result averaged across runs; the SciVideoBench entry (29.80) matches no entry of Table 4. If each column reports the best source dataset, the selection is made on the test set. (d) Sections 3.3 and 4.2 state that the adapted model is evaluated on the full test set, which includes the 32 adaptation questions, while the Table 3 caption says evaluation is on unseen batches. On MMVU these questions are 5% of the 625-question set, against a reported gain of 4.8 points.
Questions for authors

Clarifications that may change my rating:

  1. Which adaptation set, and how many runs, produced each number in Table 1?
  2. What are the accuracies with the 32 adaptation questions excluded from the test sets?
  3. Are the Video-MME and LongVideoBench values of the Qwen2.5-VL-7B base row swapped? What does that base score with 32 frames?
  4. For each benchmark, what fraction of outputs fails answer extraction or hits the length limit, before and after adaptation? How does the distribution of predicted options change?
  5. What accuracy do you obtain when adaptation uses mismatched video–question pairs, or the same questions with blank frames?
  6. Why does the ground-truth reward underperform the majority-vote reward? Were the hyperparameters and number of steps identical?
  7. In Section 4.4, self-consistency (62.72 on MMVU) is compared with 65.60, which is the InternVL3-8B result of Table 1, while the preceding experiment in the same paragraph uses Qwen2.5-VL-7B. Which backbone and how many samples does the self-consistency baseline use? The same section argues that adaptation is cheaper because it runs once per batch; how many GPU-hours does one adaptation run (5 epochs of 1,024 rollouts of up to 1,024 tokens) take?

Presentation fixes: Section 4.4 attributes +4.33 (VideoMMMU) and +4.8 (MMVU) to frame sampling, but in Table 2 these are the total gains over the base model; the batched reward gives +3.0 and +4.0, and generalized frame sampling adds +1.33 and +0.8. Equation references are unresolved ("Equation ??"); the Table 1 caption mentions "two instructional video question-answering tasks" for five benchmarks; the example "Perception (72.67 → 74.67)" in Section 4.4 does not appear in any table.

Suggestion that does not affect my rating: the frequency reward requires a discrete answer extractor, and all five benchmarks are evaluated with closed-form answers. Given the title "video reasoning", either evaluate one open-ended task or state the closed-form scope in the title and abstract.

Reviewer_ikRKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TTA-Vid adapts a pretrained vision-language model using a small batch of unlabeled video-question pairs. It samples several frame subsets per video, pools answers from multiple reasoning rollouts, and constructs a reward from answer frequency and a proposed entropy penalty. GRPO updates the model, while a multiplicative-weights sampler uses the same rewards to update frame selection. The per-video sampling distributions are averaged into a temporal prior for inference on other videos. Experiments cover five benchmarks and two backbones, with studies of adaptation-set size, frame sampling, and cross-dataset transfer.

Critical strengths
  1. Using different temporal views is a sensible way to obtain diverse evidence for label-free video adaptation. The method connects model adaptation and frame selection through the same rollout pool, and the reported gains from only 32 unlabeled examples make this setting worth studying.

  2. The InternVL3-8B comparison is encouraging: the frozen and adapted models both use 32 frames at inference, and average accuracy increases from 51.37 to 55.13. The paper also includes an unseen-batch comparison in Table 3 and a frame-sampling comparison on the same adapted model in Table 6. These controls provide useful evidence beyond the main leaderboard results.

Critical weaknesses
  1. The claimed role of the added answer-entropy penalty is not established by the update equations. For each video-question pair, alpha H_norm(p) is shared by all K x N rollout rewards. It therefore cancels exactly when the subset rewards are centered in Eq. (5), leaving the stated MAB update independent of alpha. It would also cancel from standard group-normalized GRPO advantages if each group uses the same video-question reward pool and sampled rewards are treated as fixed. The grouping, normalization, and reward preprocessing are not specified sufficiently to check this condition. Answer-frequency rewards can still drive learning; the unresolved point is whether the added entropy term contributes to either update, as claimed. An explicit implemented objective and an alpha = 0 control are needed to support that mechanism.

  2. The evaluation does not yet make the strength and reliability of generalized adaptation clear enough. Tables 2 and 3 already report repeated-run and unseen-batch experiments, but the number of runs and variability across independently sampled adaptation batches are missing. The main protocol evaluates the full test set without clearly stating whether adaptation questions and their source videos are excluded from the unseen-data results. The adaptation settings behind Tables 1 and 4 also need clarification: the VideoMMMU/SciVideoBench values are 53.66/29.80 in Table 1 and 51.48/28.70 on Table 4's in-domain diagonal. These may reflect different batches or adaptation sources rather than incorrect results. Finally, the frozen Qwen baseline uses up to 768 frames, versus 32 for TTA-Vid. Its LongVideoBench change from 65.10 to 57.81 is therefore inconclusive about adaptation itself. TTRV already matches the four-frame training input and 32-frame inference setting, but its adaptation batch, rollout count, and update budget need to be specified before attributing the remaining gap to the proposed design.

Questions for authors
  1. What are the exact GRPO groups, advantage normalization, and reward preprocessing? Where does alpha affect the implemented updates after centering? Please compare alpha = 0 using matched rollouts and explain how normalized answer entropy is defined when only one distinct answer is sampled.

  2. Which adaptation dataset, sampled batch, and checkpoint-selection rule produce each Table 1 entry? Please explain how those settings relate to Table 4, particularly the VideoMMMU/SciVideoBench values of 53.66/29.80 versus the in-domain values of 51.48/28.70.

  3. How do the frozen Qwen backbone, repurposed TTRV, and TTA-Vid compare at the same 32-frame inference budget? For TTRV and TTA-Vid, please match the adaptation examples, rollout count, and optimization budget, and report adaptation runtime or compute.

  4. Across independently sampled batches of 32 adaptation examples and random seeds, what are the mean gains and confidence intervals? Please state the run count and distinguish evaluation on held-out question pairs from evaluation on videos that were entirely absent during adaptation.

Reviewer_6RqSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

TTA-Vid generates candidate answers from multiple frame subsets for the same video question and constructs rewards using answer frequencies across subsets and an entropy term. It combines GRPO with a multi-armed bandit to update the model and the frame distribution, then averages the distributions of the adaptation videos to form a sampling prior for testing. The goal is to achieve adaptation that transfers across samples and datasets using only 32 unlabeled samples.

Critical strengths
  1. Adaptation using a small number of unlabeled samples has practical value.
  2. The experiments cover two backbone families and already include comparisons ranging from a single sample to 32 samples, full versus generalized TTA, different sampling strategies, and self-consistency; the evidence is not limited to the main results table.
Critical weaknesses
  1. There is an issue with the effectiveness of the entropy term at the level of the equations. All candidates for the same question share the same entropy penalty; under Eqs. (4)–(5), this term cancels exactly when the baseline is subtracted from the subset reward. If GRPO uses within-group centering, it also cancels. Thus, the mechanistic explanation that the entropy term suppresses uncertainty is not yet established, unless the implementation handles it differently
  2. The contributions of the training components have not been isolated. Table 2 enables TTRL and MAB Rewards simultaneously; Table 6 keeps the model trained with adaptive frame selection fixed and changes only the test-time sampling. The existing ablations support the gains from global sampling, but do not separately establish the contributions of the frequency reward and MAB frame selection during training.
Questions for authors
  1. Please provide the GRPO advantage calculation and the implementation of the entropy term, together with an α=0 control and a separate MAB ablation; if the entropy term does indeed cancel, the mechanistic explanation should be revised.
  2. Does the generalization evaluation exclude the 32 adaptation samples and videos from the same sources? How are the hyperparameters and adaptation sources selected, and were evaluation labels accessed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.