acceptodds
Under review as a conference paper at ICLR 2027

Where Should Multimodal Test-Time Adaptation Capacity Live? A Controlled Study of Prompts and Residual Adapters

Abstract

We study where to place a fixed online adaptation budget in a frozen audio-visual transformer. Multimodal test-time adaptation must respond to audio-only, video-only, and paired shifts, yet prompt capacity and residual adapter capacity impose different update structures. We isolate this choice under a fixed BriMPR objective and a near-matched trainable-parameter budget, comparing prompt capacity with identity-initialized Prompt+Adapter capacity. The adapter is exactly identity at initialization, while a two-step integrity test verifies the intended gradient flow. Under a validated batch-32 severity-5 common-batch protocol (48 cases: 2 datasets 3 corrupted modalities 8 methods, three target-stream seeds), Prompt+Adapter attains the highest mean accuracy in all six dataset-modality settings, improving over prompt-only capacity by 0.04–2.41 percentage points and over the unadapted source model by 8.28 points on average. The ranking is stable across severities 1, 3, and 5, and placement and width pilots show that visual-side adapters match dual-side adapters while wider capacity helps both families. We deliberately scope our claims: we make no claims of universal superiority, causal mechanisms, or deployment efficiency.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
test-time adaptation, multimodal learning, audio-visual learning, prompt tuning, adapters, parameter-efficient fine-tuning, domain shift

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9mKdOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the structural choices of multimodal test-time adaptation methods, including prompt tuning and residual adapters. It mainly focuses on comparing the combination of adapters and prompt tuning, termed Prompt+Adapter, with the prompt-tuning-only method. It also introduces identical initialization for adapters and gradient analysis. The results claim that Prompt+Adapter exceeds the performance of prompt78 while introducing negligible parameter capacity across several multimodal settings.

Critical strengths
  • It provides empirical evidence on comparing the performance of the combination of prompt and adapter with the prompt-only method.
  • This paper includes comprehensive and solid experimental designs supporting the effect of Prompt+Adapter.
  • It seeks a balance between the parameters of prompts and adaptors that could achieves superior performance of Prompt+Adapter over Prompt78 with comparable trainable parameters.
Critical weaknesses
  • The contribution is incremental, which simply combines two existing methods under controlled capacity.
  • The improvements of Prompt+Adapter over Prompt78 and BriMPR is limited in Mean-test-accuracy.
  • Its paper writing is poor and hard to follow.
  • The experiments are only conducted on one backbone CAV-MAE, lacking backbone ablations.
Questions for authors
  • What’s the crucial motivation for combining adapters and prompts?
  • What might be the differences or advantages of adapters and prompts for tailoring the multimodal test-time adaptation task?
  • Why is the adapter-only configuration absent in the comparison?
  • This paper only reported one prompt-adapter parameter allocation in its main experiments. Could different ratios yield better performance, and is the reported one the best?
  • Will the adaptation capacity for multimodal TTA tasks be different compared to other tasks? As maybe some tasks will be more influenced by the context provided by prompt tuning, while others might exhibit stronger performance when adapters are applied to adapt representations. What makes the reported prompt-adapter parameter ratio effective for multimodal TTA tasks?
Reviewer_78n4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the allocation of trainable capacity in multimodal test-time adaptation. Using a frozen CAV-MAE backbone, the authors compare Deep Prompt Tuning with Prompt+Residual Adapter under a fixed BriMPR objective and nearly identical trainable parameter budgets. Experiments cover audio-only, video-only, and paired corruptions on Kinetics50-C and VGGSound-C. Prompt+Adapter achieves the highest mean accuracy across all six main settings, with larger gains under paired corruptions. The authors further investigate the effects of corruption severity, adapter placement, capacity width, and batch size.

Critical strengths

Capacity allocation in multimodal TTA is a meaningful research question that may inform parameter-efficient online adaptation.

Critical weaknesses
  1. The research motivation and the rationale for the comparison framework are insufficiently established. The paper does not adequately explain what makes capacity allocation distinctive in multimodal TTA. Although parameter matching isolates one aspect of capacity allocation, the paper does not sufficiently explain why this constraint yields practically useful design guidance, given the different computational and memory costs of prompts and adapters. Furthermore, the findings are established only under the BriMPR objective. Comparisons with Tent, EATA, SAR, and READ provide performance context but cannot establish whether the allocation findings extend to other objectives, since these methods change both the objective and the trainable parameter configuration.

  2. The experimental analysis provides insufficient new understanding or actionable design insights. Although the paper includes severity, placement, and width analyses, these experiments mainly characterize performance trends rather than explain the observed differences. Why are adapter gains larger under paired corruptions but limited under single-modality corruptions on KS50-C? Why do visual-only adapters perform similarly to dual-branch adapters in the placement pilot? Considerable space is devoted to the existing BriMPR objective, identity initialization, and gradient derivations, but these properties do not explain the accuracy differences. Even within its stated empirical scope, the study provides limited insight into when and why a particular capacity allocation should be preferred.

Questions for authors
  1. Figure 2 contains distorted text, unclear labels, and irregular connections, suggesting that generative AI may have been used without sufficient subsequent verification. In particular, subfigure (c) labels the input and output dimensions as 768 and 788, contradicting both the residual structure and the definitions in the text. Please explain how the figure was produced and verified, confirm the actual input and output dimensions used in the implementation, and provide a corrected figure consistent with the model description.

  2. What motivates trainable parameter count as the primary budget constraint? Do you have existing measurements or results indicating whether the ranking persists under comparable memory usage or adaptation time?

  3. Do you have controlled comparisons between parameter-matched prompt and adapter configurations under the same entropy-minimization objective? How do the individual BriMPR loss components affect their relative performance?

  4. What evidence explains the larger gains under paired corruptions and the similar performance of visual-only and dual-branch adapters in the six-domain placement pilot?

Reviewer_LLAWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper examines the choice of trainable components for multimodal test-time adaptation. Based on the BriMPR framework, the authors compare a prompt-only approach with a combination of prompts and adapters, using nearly equal numbers of trainable parameters. Experiments on Kinetics50-C and VGGSound-C show that the combined approach performs better in several settings, particularly when both audio and visual inputs are corrupted. Additional experiments examine the effects of corruption severity and different parameter configurations.

Critical strengths
  1. The paper explores a relevant design choice in multimodal test-time adaptation. Understanding how different trainable components affect adaptation performance could be useful for developing more effective methods.

  2. The authors make an effort to compare different configurations under consistent experimental conditions. The evaluation covers several corruption settings and provides some useful observations about the relative performance of prompts and adapters.

Critical weaknesses
  1. The paper does not provide a sufficiently clear motivation for the research problem. While the authors investigate how to distribute trainable parameters between prompts and adapters, it is not clear why this particular choice represents an important research challenge. The introduction does not adequately identify the limitations of existing approaches or explain what new understanding this investigation is expected to provide. Consequently, the significance of the research question remains unclear.

  2. The contribution appears rather limited, and the work is closer to an empirical comparison than a substantial methodological study. The authors primarily reuse existing prompt and adapter techniques within an established adaptation framework. The main results show that one configuration performs better than another under certain conditions, but the paper does not sufficiently explain the reasons behind these differences. More importantly, the findings do not lead to clear or generalizable insights that could guide the design of future multimodal adaptation methods. Without deeper analysis, the work offers limited contributions beyond the reported experimental observations.

  3. The manuscript needs considerable improvement in its presentation and organization. The formatting is inconsistent in places, with excessive empty space, poorly arranged figures, and some diagrams that are difficult to read. Beyond these visual issues, the paper gives too much attention to experimental procedures and implementation details without developing a clear and coherent research narrative. The motivation, key contributions, and implications of the findings are not presented with sufficient clarity. Overall, the manuscript feels more like an experimental report than a mature research paper.

Questions for authors

None. The main concerns are described in the weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.