acceptodds
Under review as a conference paper at ICLR 2027

Select, Share, Protect: MANOVA-Guided Adaptation for Continual Multimodal Learning

Abstract

Continual learning enables multimodal large language models (MLLMs) to adapt to evolving audio, visual, and joint audio–visual understanding tasks as new data arrive. Effective adaptation requires learning plasticity and memory stability, yet balancing them remains challenging. Updating shared parameters supports new learning but risks forgetting, while freezing them helps preserve earlier knowledge but restricts adaptation. These trade-offs motivate M**A**NOVA-guided **s**election and shared **a**daptation with selective **p**rotection (**ASAP**), a sparse fine-tuning method for continual multimodal learning. We *select* shared, audio, video, and interaction blocks of pretrained matrices using a multivariate analysis of variance (MANOVA) factorial decomposition of loss gradients. Tasks *share* updates at modality-relevant blocks to reuse learned knowledge. To balance memory stability and learning plasticity, we *protect* updates important to earlier tasks using loss-gradient importance scores while leaving other active updates trainable for new learning. Unselected weights remain frozen, and active updates merge into pretrained weights to avoid additional forward-pass latency. Under mild assumptions, our analysis bounds increases in earlier-task loss in terms of the importance left unprotected. Experiments on MUSIC-AVQA and CMU-MOSI across multiple task orders demonstrate superior new-task learning and final accuracy compared with the state-of-the-art baselines, alongside competitive retention. Efficiency comparisons show comparable memory and latency among selected methods. The implementation of ASAP is available at [https://anonymous.4open.science/r/ASAP-ICLR/](https://anonymous.4open.science/r/ASAP-ICLR/).

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual learning, multimodal large language models, sparse fine-tuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WS9AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the stability–plasticity trade-off in continual learning for multimodal large language models. The authors propose ASAP, which employs MANOVA-based gradient effect decomposition to select shared, audio-specific, video-specific, and interaction parameter blocks. These blocks are selectively activated according to the available modalities, while gradient-based importance estimation is used to protect parameters critical to previously learned tasks. Experiments on MUSIC-AVQA and CMU-MOSI, covering six task orders and different backbone architectures, demonstrate improvements in new-task learning, final accuracy, and knowledge retention.

Critical strengths
  1. Well-motivated and coherent framework. The paper integrates modality-aware parameter selection, shared adaptation, and importance-based protection to balance stability and plasticity under a fixed adaptation budget. The three components are logically connected and form a coherent continual learning framework.
Critical weaknesses
  1. The necessity and distinctive contribution of MANOVA-based parameter selection are not sufficiently established. MANOVA-guided block selection is a central technical contribution of this paper. Although the random-selection ablation demonstrates its effectiveness, this comparison alone does not establish the necessity of MANOVA-based modality decomposition over conventional gradient-driven parameter selection. Existing methods such as SMT and SPU also employ gradient information for selective parameter updates. However, the paper lacks a controlled comparison with simpler gradient-importance-based or similar sparse block-selection strategies under the same parameter budget and sharing/protection mechanisms. Furthermore, the proposed modality-specific block assignments are derived from gradient differences across input conditions, but more direct evidence is needed to demonstrate their advantages for the corresponding modality-specific tasks. Consequently, the unique contribution of MANOVA decomposition remains insufficiently isolated. 
  2. The task setting and applicability of the protection mechanism are relatively limited. The framework associates task identities with three predefined modality configurations (A, V, and AV) and uses these configurations to determine parameter activation and protection. Although six training orders are evaluated, the experiments remain restricted to these three task types. The paper does not investigate scenarios where different learning objectives share the same modality configuration. In such cases, distinct tasks may share the same activated parameter blocks and protection records, raising questions about whether their previously acquired knowledge can be adequately preserved. Additionally, the experiments involve only five training stages, leaving the long-term behavior of selective protection insufficiently explored. As more tasks are learned, accumulated protection constraints may reduce the trainable capacity available for subsequent adaptation. The current evaluation therefore provides limited evidence of the framework's effectiveness in more complex and extended continual learning scenarios.
Questions for authors
  1. Could the authors provide a controlled comparison between MANOVA-based selection and conventional gradient-importance-based or SMT-style block selection, while maintaining the same parameter budget, sharing mechanism, and protection strategy? Such an experiment would help clarify the necessity and distinctive advantages of MANOVA decomposition. 
  2. How would ASAP distinguish and protect knowledge from different tasks that share the same modality configuration but have different learning objectives? Additionally, in longer continual learning sequences, could the accumulated protection constraints progressively reduce the number of trainable blocks available for subsequent tasks?
6 comments
AuthorsOpenReview ↗

Response to Reviewer WS9A (1/3)

We sincerely thank Reviewer WS9A for the careful review and for the constructive questions. Table R1 below reports the controlled comparison you ask for, Table R8 below adds the direct per-task evidence you request, and Tables R9 and R10 in the second and third parts of this reply answer your two questions on the task setting.

[W1a / Q1] Controlled comparison with gradient-importance or SMT-style selection

Thank you for this question, which helped us isolate what the MANOVA-guided selection contributes. To show its necessity and contribution, we ran the controlled comparison you describe as a new experiment, ranking blocks by the mean absolute gradient (MAG, SMT's statistic) inside ASAP with the same parameter budget, sharing mechanism and protection strategy. Table R1 shows that MAG is below ASAP on both datasets.

To separate the two steps of ASAP's score, we also tested the mean squared gradient norm (MSG) and the unstandardized factorial effect (UFE), which uses the factorial contrasts without the standardization.

Together, these experiments show two things. First, gradient magnitude does not explain the gains. Replacing ASAP's standardized effect score with MAG lowers Final.ACC and New.ACC by about 1.5 points on both datasets, and replacing it with MSG lowers them by about 1.4 points. Second, both steps of ASAP's score contribute. Removing only the standardization (UFE) lowers accuracy by about 0.8 points, and UFE in turn exceeds MAG by about 0.7 points. Avg.FOR is close for all selection scores, since they use the same protection mechanism.

MANOVA-guided selection is therefore necessary, and its contribution is clear from two standard statistical principles. The factorial contrast compares a block's gradients with and without a modality, which identifies the block's modality role. Standardization judges each effect relative to its variation across samples, which favors blocks whose effect is consistent. The General response gives the details, and its Table R2 gives the overlap between the selections.

Table R1. Selection scores with everything else in ASAP fixed (CMU-MOSI averaged over orders P1P_1, P3P_3 and P5P_5, MUSIC-AVQA on order P3P_3).

Selection scoreMOSI Final.ACCMOSI Avg.FORMOSI New.ACCAVQA Final.ACCAVQA Avg.FORAVQA New.ACC
Random selection80.350.2680.2866.042.1068.36
MAG81.440.2181.2067.261.8068.99
MSG81.550.1681.2867.431.7869.13
UFE82.120.1781.9167.991.8469.72
ASAP82.930.1782.6968.791.8270.50

[W1b] Direct evidence for the modality-specific block assignments

Thank you for asking for more direct evidence, which led us to verify the modality roles at the level of individual tasks and block sets. Table R8 below gives three pieces of evidence.

Evidence 1, per-task gains. In Table R8a, ASAP improves the audio and video tasks by about 2 points each relative to MAG, against less than 1 point for the audio–visual task. This is the pattern expected if the roles are meaningful. By Equation 5 (page 5), the roles decide which blocks a unimodal task uses, whereas the audio–visual task activates all four block sets whatever their roles.

Evidence 2, removing a block set. In Table R8b, removing the learned updates of the audio or the video block set lowers its own task, by about 3 points for audio and 8 points for video. Each unimodal task therefore relies on the set assigned to its modality, and not only on the shared set.

Evidence 3, swapping two block sets. Swapping the audio and video updates lowers the video task about as much as removing its updates, and the audio task by most of that loss. One set's updates therefore largely cannot stand in for the other's, so each set holds knowledge specific to its modality.

Together with the results in our response to W1a, we have shown the unique contribution of the MANOVA decomposition and the advantages of its modality-specific assignments.

Both comparisons are in the revised manuscript, in Section 5.3, Table 3 (page 9) and Appendix D.6, Tables 14 and 15 (page 35). The next part turns to your questions on the task setting.

Table R8. Per-task accuracy and block interventions on MUSIC-AVQA order P3P_3.

(a) Selection comparison, final accuracy (%)

Selection scoreAVAV
Random selection56.8075.4765.84
MAG58.1077.5466.13
MSG58.3577.7066.23
UFE59.1678.3666.46
ASAP60.2779.3166.78

(b) Interventions on the ASAP checkpoint, change in percentage points

InterventionΔ AΔ VΔ AV
Swap A/V groups−2.36−7.930.00
Remove audio-group updates−3.250.00−3.65
Remove video-group updates0.00−8.05−3.36
AuthorsOpenReview ↗

Response to Reviewer WS9A (2/3)

[W2 / Q2] Objectives with the same modality configuration, and protection in longer sequences

Thank you for these two questions, which identified a case the submission did not test and led us to extend ASAP and to measure how protection grows. Both depend on one design choice, how ASAP keeps its importance records, so we answer them together with one new experiment.

The two record rules. As submitted, ASAP identifies a task by its modality combination and keeps one importance record per task (ASAP-M). Two objectives with the same modality configuration then share a record, and because Equation 8 (page 6) excludes the current task's own record, the earlier objective is unprotected while the later one trains, which we believe is the scenario you describe. When objective identity is known during training, each objective can instead keep its own record, with only the current objective's record excluded (ASAP-O). Nothing else changes. The two objectives still use the same active blocks, selection and the budget are the same, and inference needs no task or objective identifier.

The experiment. We tested both rules and PCLR, the second-best method on MUSIC-AVQA by Final.ACC in Table 1 (page 7), on a new stream of nine objectives built from the MUSIC-AVQA question types, two audio, two visual and five audio–visual. For example, audio counting at stage S1S_1 is followed by audio comparison at stage S3S_3, which has the same modality configuration.

Result 1, retention. One record per objective protects the earlier objective. In Table R9, when a same-modality successor is learned, the earlier objective drops by about 0.2 points under ASAP-O, against 1.6 to 3.6 points under ASAP-M and a similar drop under PCLR. The whole stream benefits as well. In Table R5, ASAP-O has the highest Final.ACC and halves Avg.FOR (0.93 against 1.89 for ASAP-M and 2.03 for PCLR), while its New.ACC stays close to ASAP-M's and above PCLR's.

The next part reports the second result, on trainable capacity, and the trade-off between the two rules.

Table R9. Retention of each modality's first objective when a successor with the same modality is learned (Appendix D.9, Table 20, page 40).

ObjectiveMethodAfter own stageBefore successorAfter successorForgetting
A-Counting (S1S_1, successor S3S_3)ASAP-O66.56865.78265.5850.197
ASAP-M66.56865.78262.1443.638
PCLR65.29063.71760.2753.441
V-Counting (S2S_2, successor S4S_4)ASAP-O77.94577.69477.5270.167
ASAP-M77.94578.19576.6081.587
PCLR76.27477.61175.4392.172
AV-Existential (S5S_5, successor S6S_6)ASAP-O81.78181.78181.5790.202
ASAP-M81.57981.57979.8581.721
PCLR80.66880.66879.5551.113

Table R5. Nine-objective stream. S1S_1 to S9S_9 give the mean accuracy over the objectives learned so far, with S9S_9 as Final.ACC (Appendix D.9, Table 21, page 40).

MethodS1S_1 (A)S2S_2 (V)S3S_3 (A)S4S_4 (V)S5S_5 (AV)S6S_6 (AV)S7S_7 (AV)S8S_8 (AV)S9S_9 (AV), Final.ACCAvg.FORNew.ACC
PCLR65.29069.99566.95069.33571.90871.34570.01366.75167.1012.02868.739
ASAP-M66.56871.86368.10470.05072.52072.35570.90769.89968.9001.89470.281
ASAP-O66.56871.86369.19671.30272.54372.90671.37770.02069.5160.92870.179
AuthorsOpenReview ↗

Response to Reviewer WS9A (3/3)

[W2 / Q2, continued] Trainable capacity and the trade-off between the two record rules

Result 2, capacity. Table R10 shows that the frozen fraction of active blocks depends on the record rule.

With one record per task (ASAP-M), it is bounded. By Equation 8 (page 6), the blocks protected while a task trains are those of its active set that lie in other tasks' importance records, and a task's record is replaced, not accumulated, when the task recurs. The frozen fraction is therefore bounded by the union of the other tasks' records, however many stages follow. In the nine-stage stream it stays at 14.6% from the fifth stage on, because these stages all train the audio–visual task and the records in force no longer change. Over the thirty stages of the paper's six five-stage orders, where tasks recur, it is between about 5% and 23% at later stages (Appendix D.8, Figure 6, page 38).

With one record per objective (ASAP-O), each new objective adds a record, so the frozen fraction grows with the number of distinct objectives, as you anticipate. It grows slowly, because each record is the smallest set of blocks that covers the fraction η\eta of the objective's importance and the records overlap. The increments shrink from the sixth objective to the ninth, and the frozen fraction reaches about 45% after nine objectives, so more than half of the active blocks remain trainable. ASAP-O's new-objective accuracy stays within half a point of ASAP-M and above PCLR at every stage. A smaller η\eta protects fewer blocks for the same record, which is a direct way to slow this growth (Appendix D.8, page 38).

The trade-off. The two rules therefore trade retention against capacity. ASAP-M keeps the frozen fraction bounded but does not separate objectives that share a modality, whereas ASAP-O protects them at the cost of a frozen fraction that grows with the number of objectives and of objective identity during training. On the benchmarks of the paper each task has one objective, so the two rules coincide there and Table 1 is unchanged.

In the revised manuscript, Section 3.4 (page 6) now states that Equation 8 can use objective-specific records, Appendix D.8 (page 38) contains the record bound, the stage-wise frozen fractions and the η\eta diagnostic, Section 5.4 (page 10) states the bound, and the Limitations paragraph (page 10) notes the requirement of objective identity and reports the frozen fraction at nine objectives.

Table R10. Frozen fraction of active blocks and new-objective accuracy at each stage of the nine-objective stream (Appendix D.9, Table 21, page 40).

StageFrozen (%), ASAP-MFrozen (%), ASAP-ONew-obj. acc. (%), ASAP-MNew-obj. acc. (%), ASAP-ONew-obj. acc. (%), PCLR
S1S_1 A-Count0.000.0066.56866.56865.290
S2S_2 V-Count8.398.3977.94577.94576.274
S3S_3 A-Compare7.6023.6063.97364.31062.963
S4S_4 V-Loc8.2125.0978.20477.71476.163
S5S_5 AV-Exist14.6024.8081.57981.78180.668
S6S_6 AV-Count14.6032.4072.49072.41170.593
S7S_7 AV-Loc14.6037.6062.50062.06560.435
S8S_8 AV-Compare14.6041.2063.94263.85162.398
S9S_9 AV-Temporal14.6044.8065.32864.96463.869

We hope our responses have answered all your concerns. Please let us know if any point needs further detail, and we would be glad to discuss it.

Reviewer_fAeeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ASAP (MANOVA-guided selection and shared adaptation with selective protection), a parameter-efficient fine-tuning framework for continual multimodal learning. It mitigates the stability–plasticity trade-off for multimodal large language models learning sequential audio, visual and audio-visual tasks. The work derives a theoretical upper bound for old-task loss growth and proves the optimality of importance-driven block selection to minimize this bound under fixed settings. Evaluations on MUSIC-AVQA and CMU-MOSI with six task orders outperform continual learning baselines on new-task and end accuracy. Further ablations investigate module contributions, memory retention, computation overhead, and cross-backbone transfer.

Critical strengths
  1. A coherent framework that jointly addresses modality-aware parameter sharing and selective protection. The most compelling aspect of this work is the explicit separation between parameter activation and parameter trainability. By assigning selected blocks to shared, audio, video, and interaction components, ASAP enables knowledge reuse across different modality combinations while limiting interference through importance-based protection.

  2. Consistent empirical improvements accompanied by informative component ablations. The experimental results are encouraging across both evaluated benchmarks. In Table 1, ASAP improves Final.ACC over the strongest competing method from 67.46% to 69.46% on MUSIC-AVQA and from 80.37% to 82.82% on CMU-MOSI. It also achieves the highest New.ACC on both datasets while maintaining competitive forgetting performance.

Critical weaknesses
  1. The evaluation uses pre-accessed auxiliary data covering all modalities and tasks, restricting its applicability to general continual learning. ASAP needs a labeled pre-collected auxiliary dataset spanning audio, visual and audio-visual tasks for MANOVA gradient decomposition and importance estimation. This presupposes prior knowledge of future tasks, which is unavailable in fully online continual learning. Though the auxiliary set is small and the assumption is declared, the paper fails to test sensitivity to sample representativeness. It is unknown whether ASAP retains performance when auxiliary data diverges from later tasks or unseen modality combinations emerge. Since modality-specific fixed block allocation underpins ASAP, good results under this protocol cannot guarantee general effectiveness.
  2. Experiments insufficiently validate the necessity of MANOVA-based selection and the robustness of performance gains. The core contribution leverages standardized factorial gradient effects to pick modality-related parameter blocks. Yet its key ablation only compares against random block selection, lacking baselines of simpler gradient-ranking methods. The MANOVA selection demands gradients from four input conditions and extra statistical computation; without stronger comparisons, gains may stem merely from gradient-guided selection instead of the MANOVA statistical design. Moreover, results averaged over six task orders lack random-seed uncertainty estimates, and task orders cannot replace repeated training to assess variance. The efficiency test only measures post-merging forward latency and ignores selection and preprocessing cost. These gaps weaken evidence for the method’s core contribution and efficiency claims.
Questions for authors
  1. Is MANOVA-based selection essential compared with simpler gradient-based approaches? Could the authors provide controlled comparisons against gradient-magnitude ranking, squared-gradient-norm ranking, and factorial-effect ranking without covariance normalization, while keeping the adaptation budget, modality-based activation, and protection mechanism unchanged? These comparisons would help distinguish the benefit of the MANOVA decomposition from the more general benefit of allocating trainable blocks using gradient information.

  2. Statistical significance and computational overhead Could the authors report results across multiple independent random seeds, including standard deviations or confidence intervals for New.ACC, Final.ACC, and Avg.FOR?

5 comments
AuthorsOpenReview ↗

Response to Reviewer fAee (1/3)

We sincerely thank Reviewer fAee for the detailed review and for naming the specific baselines and statistics that would make the evaluation more convincing. Tables R4 and R5 below address W1, Table R1 in the second part of this reply addresses Q1, and Tables R6 and R7 in the third part address Q2.

[W1] Auxiliary data, sample representativeness and general effectiveness

Thank you for this thoughtful comment, which led us to test ASAP with a different selection dataset and a longer stream of objectives. We break it into four points and answer them in turn.

First, "presupposes prior knowledge of future tasks." Like ZSCL [1], SND [2] and other recent continual adaptation methods [3, 4, 5], which rely on reference data from outside the task stream, ASAP assumes a small auxiliary set that is available before training. In ASAP this set is labeled, has aligned audio and video, and amounts to about 1% of the training samples. It does not require separate examples for audio, visual and audio–visual tasks, because we construct all four input conditions from the same samples by including or omitting audio and video (Section 3.2, page 4). Importance estimation uses only samples of the task just learned, so it needs no future-task data.

Furthermore, examples of future tasks are not necessarily needed for selection. In Table R4 we select CMU-MOSI's block sets from MUSIC-AVQA auxiliary questions, which contain no CMU-MOSI examples. Final.ACC and New.ACC decrease by less than half a point, and the result remains 2.4 points above DGG, the second-best method on CMU-MOSI by Final.ACC in Table 1 (page 7). Therefore, when auxiliary samples from the future tasks are unavailable, the selection can use an existing dataset with the same modalities.

Second, "sensitivity to sample representativeness." Table R4 also tests a case like the one you describe, in which the auxiliary data diverge from the later tasks, because the MUSIC-AVQA samples differ from CMU-MOSI in both dataset and task. Selection therefore benefits only modestly from the native auxiliary set.

Third, "unseen modality combinations." A task can use audio, video or both, and the four block sets already cover all three combinations. A new sensory modality would require a new selection, and we now state this drawback in the Limitations paragraph (page 10). Any task that uses a combination of the known modalities, however, is compatible with the selection already made, because the selection assigns blocks to modalities and not to tasks.

Fourth, "general effectiveness" of the fixed block allocation. To test this, we ran a new experiment in Table R5, in which nine objectives built from the MUSIC-AVQA question types, such as audio counting, visual location and audio–visual temporal questions, arrive one at a time. Because every question type uses a known modality combination, the allocation is fixed once, before the first objective, and never revised. In Table R5, ASAP-M is ASAP as submitted, with one importance record per task. ASAP-O is an exploratory extension with one record per objective (Appendix D.9, page 40). ASAP-M reaches a Final.ACC of 68.90 against 67.10 for PCLR, the second-best method on MUSIC-AVQA by Final.ACC in Table 1 (page 7), with higher New.ACC and lower Avg.FOR, and it stays above PCLR after every one of the nine stages. Together with Table R4, this supports selecting modality-associated blocks once across datasets and learning objectives.

At the same time, we agree with you that fully online learning without auxiliary access, or with additional sensory modalities, is an important setting. We leave both cases to future work.

Table R4. CMU-MOSI results on order P3P_3 by source of the block selection (Appendix D.2, Table 4, page 26).

Source of the CMU-MOSI block setsFinal.ACCAvg.FORNew.ACC
CMU-MOSI clips (submission)83.430.0582.83
MUSIC-AVQA questions82.990.0482.48

Table R5. Nine-objective stream. S1S_1 to S9S_9 give the mean accuracy over the objectives learned so far, with S9S_9 as Final.ACC (Appendix D.9, Table 21, page 40).

MethodS1S_1 (A)S2S_2 (V)S3S_3 (A)S4S_4 (V)S5S_5 (AV)S6S_6 (AV)S7S_7 (AV)S8S_8 (AV)S9S_9 (AV), Final.ACCAvg.FORNew.ACC
PCLR65.29069.99566.95069.33571.90871.34570.01366.75167.1012.02868.739
ASAP-M66.56871.86368.10470.05072.52072.35570.90769.89968.9001.89470.281
ASAP-O66.56871.86369.19671.30272.54372.90671.37770.02069.5160.92870.179

[1] Z. Zheng et al. Preventing zero-shot transfer degradation. ICCV 2023.
[2] Y.-C. Yu et al. Select and distill. ECCV 2024.
[3] M. Zheng et al. Adapt without forgetting. ECCV 2024.
[4] Z. Gao et al. Stabilizing zero-shot prediction. NeurIPS 2024.
[5] Z. Chen et al. Forging a dynamic memory. CVPR 2026.

AuthorsOpenReview ↗

Response to Reviewer fAee (2/3)

[W2a / Q1] Is MANOVA-based selection essential compared with simpler gradient-based approaches?

Thank you for specifying the three comparisons so precisely, which let us separate the contribution of the factorial effect from that of its standardization. To show whether MANOVA-guided selection is essential, we ran all three as new experiments, ranking blocks by the mean absolute gradient (MAG), the mean squared gradient norm (MSG) and the unstandardized factorial effect (UFE), with the adaptation budget, modality-based activation and protection mechanism unchanged, as you ask. Table R1 shows that the gains do not stem merely from gradient-guided selection.

Together, these experiments show two things. First, gradient magnitude does not explain the gains. Replacing ASAP's standardized effect score with MAG lowers Final.ACC and New.ACC by about 1.5 points on both datasets, and replacing it with MSG lowers them by about 1.4 points. Second, both steps of ASAP's score contribute. Removing only the standardization (UFE) lowers accuracy by about 0.8 points, and UFE in turn exceeds MAG by about 0.7 points. Avg.FOR is close for all selection scores, since they use the same protection mechanism. The gains are consistent across orders. ASAP leads on every tested order of both datasets, and the step from UFE to ASAP also holds across the three training seeds of Table R6 in the third part of this reply. Simpler gradient-guided selection therefore does not reproduce ASAP's gains, and both steps of the MANOVA-guided score are needed.

These gains follow from two standard statistical principles. First, the factorial contrast takes the effect of a factor to be the difference between the conditions with and without it on the same samples. This identifies a block's modality role, whereas a gradient magnitude measures only how strongly the block responds overall. Second, standardization judges an effect relative to its standard deviation, not by its raw size, so a block with a large but inconsistent effect ranks below a block with a smaller but consistent one (Appendix A.2, page 18, and Proposition 1, page 17). The factorial contrast therefore decides the roles, and the standardization decides which blocks are reliable enough to select. Table R2 in the General response adds the overlap between the selections.

Based on all the evidence above, MANOVA-guided selection is essential to ASAP's gains, and the contribution of each of its steps is clear. Its added cost, from the four input conditions and the extra statistical computation you note, is under 5% of one order's wall-clock time on MUSIC-AVQA, as measured in Table R7 in the third part of this reply. This comparison is now in Section 5.3, Table 3 (page 9) of the revised manuscript, with the full tables and controls in Appendix D.6 (page 34). The next part reports the seed study and the cost measurements you ask for.

Table R1. Selection scores with everything else in ASAP fixed (CMU-MOSI averaged over orders P1P_1, P3P_3 and P5P_5, MUSIC-AVQA on order P3P_3).

Selection scoreMOSI Final.ACCMOSI Avg.FORMOSI New.ACCAVQA Final.ACCAVQA Avg.FORAVQA New.ACC
Random selection80.350.2680.2866.042.1068.36
MAG81.440.2181.2067.261.8068.99
MSG81.550.1681.2867.431.7869.13
UFE82.120.1781.9167.991.8469.72
ASAP82.930.1782.6968.791.8270.50
AuthorsOpenReview ↗

Response to Reviewer fAee (3/3)

[W2b / Q2] Random-seed uncertainty

Thank you for this comment, which led us to measure the seed variation that the six task orders could not show. We agree with you that task orders do not substitute for repeated training. We therefore trained ASAP and the second-best method of Table 1 (page 7) by Final.ACC on each dataset, DGG on CMU-MOSI and PCLR on MUSIC-AVQA, with three independent random seeds on order P3P_3, and Table R6 reports the three metrics you list with their standard deviations.

On both datasets, ASAP exceeds the second-best method on every seed and every metric, and the Final.ACC and New.ACC margins are many times larger than the seed standard deviations. The margin on this order is close to the six-order margin in Table 1 (page 7).

Table R6 also compares ASAP with UFE, the unstandardized score of Table R1. ASAP exceeds UFE by 0.65 ± 0.17 points in Final.ACC and by a similar margin in New.ACC, so the gain from standardization holds across seeds. The two differ in Avg.FOR by 0.055 ± 0.055, about one seed standard deviation, as expected from their shared protection rule.

Update: the three-seed study now covers all six orders of both datasets (Table R12 in our update to all reviewers), with the same conclusion.

The per-seed values are in Appendix D.4, Table 7 (page 28), and the caption of Table 1 (page 7) now points to them.

Table R6. Three training seeds on order P3P_3 (mean ± standard deviation). Δ is the within-seed difference between ASAP and the competitor.

ComparisonMetricASAPCompetitorΔ
CMU-MOSI P3P_3 vs. DGGFinal.ACC83.139 ± 0.25780.391 ± 0.1792.748 ± 0.092
Avg.FOR0.109 ± 0.0480.213 ± 0.038−0.103 ± 0.059
New.ACC82.789 ± 0.06779.796 ± 0.1622.993 ± 0.219
MUSIC-AVQA P3P_3 vs. PCLRFinal.ACC68.750 ± 0.25266.250 ± 0.1212.500 ± 0.185
Avg.FOR1.847 ± 0.1033.837 ± 0.102−1.990 ± 0.139
New.ACC70.443 ± 0.25068.637 ± 0.1971.807 ± 0.426
CMU-MOSI P3P_3 vs. UFEFinal.ACC83.139 ± 0.25782.491 ± 0.1010.648 ± 0.171
Avg.FOR0.109 ± 0.0480.055 ± 0.0180.055 ± 0.055
New.ACC82.789 ± 0.06781.973 ± 0.1310.816 ± 0.151

[W2c / Q2] Selection and preprocessing cost

Thank you for pointing this out, which made our efficiency comparison complete. We measured the preparation cost in Table R7. Block selection and importance estimation together add about 6% to the wall-clock time of one training run, that is, one task order of five stages, and about 2% per run when the one selection serves all six orders. These measurements are in Appendix D.7, Table 19 (pages 36–37), and Section 5.3 (page 9) now states the preparation cost.

Table R7. Time added by ASAP's two preparation steps on MUSIC-AVQA with one H100 GPU, as a share of the wall-clock time of one training run (one task order of five stages).

Preparation stepShare of one training run
Block selection (once per dataset)4.77%
Importance estimation (after each of the five stages)1.35%
Total for one training run6.12%

We hope our responses have answered all your concerns. Please let us know if any point needs further detail, and we would be glad to discuss it.

Reviewer_jPkdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper try to improve the catastrophic forgetting for MLLM in contiune learning. It addresses three input configurations: audio + text (A), video + text (V), and audio + video + text (AV). Instead of maintaining independent adapters for every task, ASAP selects sparse parameter blocks and 1) select: uses MANOVA-based gradient decomposition to identify four types of parameter blocks: shared, audio-specific, video-specific, and audio–video interaction blocks; 2) share: allows tasks with overlapping modalities to reuse and update relevant parameter blocks; 3) protect: uses squared loss-gradient importance scores to identify and freeze blocks important to previously learned tasks . The paper evaluates Gemma-4-E2B on MUSIC-AVQA and CMU-MOSI over six continual-learning orders.These are consistent improvements on the reported benchmarks.

Critical strengths
  1. The research problem that how to reuse parameters for new modalities without causing catastrophic forgetting is well motivated and clear.
  2. The three components architecture is interpretable.
  3. The experiments are informative and the improve is consistent.
Critical weaknesses
  1. The novelty is limited. The idea of parameter sharing and protection has been extensively studied, including HAT(ICML 2018) and PackNet(CVPR 2018) as cited in paper. The modality aware is also not novel as in PathWeave (NeurIPS 2024), MokA (NeurIPS 2025), and IMoD (CVPR 2026). The overall Select–Share–Protect framework largely builds upon established ideas.
  2. The necessity of MANOVA is not shown enough, the ablation only compare the selection with random selection. Comparisons with the other selection method like SMT (ICLR 2025) (not the whole design, just the selection design) would better justify the contribution of MANOVA.
  3. ASAP allocates an equal number of parameter blocks to the four groups (shared, audio-specific, video-specific, and interaction) without sufficient justification. An ablation study comparing different allocation proportions is needed to evaluate whether equal allocation is optimal.
  4. ASAP requires labeled examples from all future modality combinations before continual training begins in order to perform its MANOVA selection. This is inconsistent with the usual continual-learning setting, in which tasks arrive incrementally and future task examples are unavailable in advance.
Questions for authors

See weakness.

5 comments
AuthorsOpenReview ↗

Response to Reviewer jPkd (1/3)

We sincerely thank Reviewer jPkd for the careful review and for the constructive suggestions. Each of the four points you raised led to a change in the revised manuscript. We added a comparison table for the novelty concern in W1, ran the SMT-style selection comparison you asked for in W2, added the allocation ablation requested in W3, and stated and tested the assumption discussed in W4.

[W1] Novelty

Thank you for this comment and for the references, which helped us state the contribution more precisely. Unlike the works you mention, ASAP measures each block's modality role in the pretrained weights instead of fixing it by design, shares blocks according to the modalities a task uses instead of a task-specific mask, and protects only the smallest set of blocks that covers an earlier task's importance instead of everything the task used. The roles are measured with two standard statistical principles (Equations 1–2, page 4) in our MANOVA-guided selection. First, the factorial contrast takes a modality's effect on a block to be the difference between the block's gradients on the same sample with and without that modality, whereas a gradient magnitude alone shows that a block is sensitive but not to which modality. Second, standardization divides each effect by its standard deviation across samples, so an effect that is large on a few samples but unstable ranks below a smaller effect that is present on every sample. We detail the three differences below, one step at a time.

Select. Unlike PathWeave, MokA and IMoD, which add parameters whose modality role is fixed by design (adapters per modality, modality-specific low-rank factors, or a partition of one LoRA update), ASAP tests the existing pretrained blocks and assigns each selected block one of four roles (shared, audio, video or interaction) from the result (Equation 3, page 5). None of these three methods asks which pretrained weights already serve each modality. ASAP's roles therefore describe how the pretrained model itself uses each modality, and the adaptation budget goes to the blocks whose modality effect is strongest and most consistent. The roles are also built for continual tasks. MokA and IMoD are fine-tuning methods with no rule for what later tasks share or protect, and PathWeave continues over new modalities, whereas ASAP's roles decide sharing and protection for new tasks on known modalities.

Share. Unlike HAT and PackNet, where a mask learned or pruned for each task decides which parameters the task uses, ASAP lets the modalities present in the input decide (Equation 5, page 5). Two advantages follow. First, ASAP needs no task identifier at inference and no mask per task, both of which HAT and PackNet require, so a new task on known modalities needs nothing new to decide what it shares. Second, sharing is targeted by construction. A new task in PackNet reuses all earlier retained weights, and in HAT the reuse is left to the learned mask. In ASAP a task reuses exactly the blocks of the modalities it has in common with earlier tasks, and tasks without a common modality cannot interfere through the modality-specific sets.

Protect. Unlike PackNet, which freezes every weight a task retains, and HAT, which restricts every unit in a mask accumulated over all earlier tasks, ASAP protects only the smallest set of blocks that covers a chosen share of each earlier task's importance (Equation 8, page 6). This set is the task's importance record, and it differs further in two ways. First, the record is replaced, not accumulated, when the task recurs, whereas the protected sets of PackNet and HAT only grow, so most active blocks stay trainable for new tasks (Appendix D.8, Figure 6, page 38). Second, we show what partial protection costs. Under the stated assumptions, the earlier task's loss increase is bounded and no smaller set reaches the prescribed coverage (Theorem 1 and Corollary 1, page 7). HAT and PackNet give no comparable statement relating the amount of protection to the amount of forgetting.

To make these distinctions easy to check, we have added a comparison to the revised manuscript. Appendix E.1, Table 22 (page 41) compares ASAP with these methods, SMT and SPU on selection or allocation, activation, the earlier-task freeze rule, inference conditioning and stored state. Section 2 (page 3) points to this table, and Section 1 (page 2) now states the contribution more clearly. We turn to the necessity of this measurement, the subject of your W2, with new experiments in the next part of this reply.

AuthorsOpenReview ↗

Response to Reviewer jPkd (2/3)

[W2] The necessity of MANOVA

Thank you for this suggestion, which gave us a cleaner understanding of what the MANOVA-guided score contributes. To show its necessity and contribution, we ran the comparison you describe as a new experiment, ranking blocks by the mean absolute gradient (MAG, SMT's statistic) inside ASAP with everything else fixed. Table R1 shows that MAG is below ASAP on both datasets.

In addition, to separate the gain of the factorial effect from the gain of its standardization, we ran two further controlled experiments. One uses the mean squared gradient norm (MSG), a second gradient-magnitude score, and the other uses the unstandardized factorial effect (UFE), the factorial contrasts without the standardization.

Together, these experiments show two things. First, gradient magnitude does not explain the gains. Replacing ASAP's standardized effect score with MAG lowers Final.ACC and New.ACC by about 1.5 points on both datasets, and replacing it with MSG lowers them by about 1.4 points. Second, both steps of ASAP's score contribute. Removing only the standardization (UFE) lowers accuracy by about 0.8 points, and UFE in turn exceeds MAG by about 0.7 points. Avg.FOR is close for all selection scores, since they use the same protection mechanism. The gains are consistent across orders. ASAP leads on every tested order of both datasets, and the step from UFE to ASAP also holds across the three training seeds of Appendix D.4, Table 7 (page 28). Simpler gradient-guided selection therefore does not reproduce ASAP's gains, and both steps of the MANOVA-guided score are needed.

These gains follow from the two statistical principles described in our reply to W1. The factorial contrast compares a block's gradients with and without a modality, so it identifies the block's modality role, which a gradient magnitude cannot. Standardization judges each effect relative to its variation across samples, so it favors blocks whose effect is consistent (Appendix A.2, page 18, and Proposition 1, page 17).

Based on all the evidence above, MANOVA-guided selection is necessary for ASAP's gains, and its contribution is clear.

Table R1. Selection scores with everything else in ASAP fixed (CMU-MOSI averaged over orders P1P_1, P3P_3 and P5P_5, MUSIC-AVQA on order P3P_3).

Selection scoreMOSI Final.ACCMOSI Avg.FORMOSI New.ACCAVQA Final.ACCAVQA Avg.FORAVQA New.ACC
Random selection80.350.2680.2866.042.1068.36
MAG81.440.2181.2067.261.8068.99
MSG81.550.1681.2867.431.7869.13
UFE82.120.1781.9167.991.8469.72
ASAP82.930.1782.6968.791.8270.50

[W3] Equal allocation

Thank you for this suggestion, which led us to find that a larger shared quota may help new-task learning. Equal quotas are ASAP's untuned structural default, and we did not claim in the draft that they are optimal. We chose them because they need no knowledge of the task stream. Equal audio and video quotas give the two modalities symmetric capacity (Equation 5, page 5), and equal shared and interaction quotas avoid tuning to a particular future task mix. However, we agree that the budget allocation among the four block sets is an important issue, since it decides how much capacity each task can train, so we ran the ablation you request as a new experiment in Table R3.

The ablation points in two directions. Increasing the shared quota (40/20/20/20) improves New.ACC by about half a point, with the Final.ACC change within seed variation. Reducing the interaction quota (30/30/30/10) lowers Final.ACC and increases forgetting, and columns A and V of Table R3 show that the accuracy loss is concentrated on the audio and video tasks, consistent with interaction blocks limiting interference from audio–visual learning. Using factorial-effect strengths on auxiliary samples to guide the allocation before training could be a promising future direction.

For both points, the revised manuscript now includes the selection-score comparison (Section 5.3, Table 3, page 9, with the full tables and controls in Appendix D.6, page 34), the rationale for equal quotas (Section 3.2, page 5) and the allocation ablation (Section 5.3 and Appendix D.6, Table 16, page 36).

Table R3. Allocation ablation at a fixed total budget on CMU-MOSI order P3P_3. A, V and AV are per-task accuracies.

Quotas S/A/V/I (%)Final.ACCAvg.FORNew.ACCAVAV
25/25/25/25 (default)83.430.0582.8384.1183.6782.51
40/20/20/2083.620.0983.3884.4083.8282.65
30/30/30/1082.900.3883.1283.3883.0982.22
Seed SD of the default0.2570.0480.0670.0840.2920.421
AuthorsOpenReview ↗

Response to Reviewer jPkd (3/3)

[W4] Labeled examples from all future modality combinations

Thank you for raising this concern, which led us to test how much the selection depends on data from the future tasks. We break it into two parts, what the selection needs in advance and whether tasks can arrive incrementally.

First, what the selection needs. Like ZSCL [1], SND [2] and other recent continual adaptation methods [3, 4, 5], which rely on reference data from outside the task stream, ASAP assumes a small auxiliary set that is available before training. In ASAP this set is labeled, has aligned audio and video, and amounts to about 1% of the training samples. It does not require separate examples for each future modality combination, because we construct all four input conditions from the same samples by including or omitting audio and video while keeping the text and answer fixed (Section 3.2, page 4).

Furthermore, examples of future tasks are not necessarily needed for selection. In a new experiment in Table R4, we selected CMU-MOSI's block sets from MUSIC-AVQA auxiliary questions, which contain no CMU-MOSI examples. Final.ACC and New.ACC decrease by less than half a point, with little change in forgetting, and the result remains 2.4 points above DGG, the second-best method on CMU-MOSI by Final.ACC in Table 1 (page 7), on the same order and seed (Appendix D.4, Table 6, page 27). Therefore, when auxiliary samples from the future tasks are unavailable, the selection can be made from an existing dataset with the same modalities, as MUSIC-AVQA is used for CMU-MOSI here.

Second, tasks can arrive incrementally. A new sensory modality would require a new selection, but any task that uses a combination of the known modalities is compatible with the selection already made, because the selection assigns blocks to modalities and not to tasks. We tested this in a new experiment in Table R5, with nine objectives built from the MUSIC-AVQA question types, such as audio counting, visual location and audio–visual temporal questions, which arrive one at a time. Because every question type uses a known modality combination, we select the blocks only once, before the first objective, and each new objective is learned by activating the block sets of its modalities (Equation 5, page 5). In Table R5, ASAP-M is ASAP as submitted, with one importance record per task. ASAP-O is an exploratory extension with one record per objective (Appendix D.9, page 40). In the three metrics of Table 1 (page 7), ASAP-M reaches a Final.ACC of 68.90 against 67.10 for PCLR, the second-best method on MUSIC-AVQA by Final.ACC in that table, with higher New.ACC and lower Avg.FOR. It also stays above PCLR after every one of the nine stages.

At the same time, we agree that continual learning without any auxiliary access, or with additional sensory modalities, is an important setting. We leave both cases to future work. The Limitations paragraph (page 10) now states the auxiliary-data assumption and the restriction to known modalities, and the abstract (page 1) and Section 3.1 (page 4) make the auxiliary-data setting explicit.

Table R4. CMU-MOSI results on order P3P_3 by source of the block selection (Appendix D.2, Table 4, page 26).

Source of the CMU-MOSI block setsFinal.ACCAvg.FORNew.ACC
CMU-MOSI clips (submission)83.430.0582.83
MUSIC-AVQA questions82.990.0482.48

Table R5. Nine-objective stream. S1S_1 to S9S_9 give the mean accuracy over the objectives learned so far, with S9S_9 as Final.ACC (Appendix D.9, Table 21, page 40).

MethodS1S_1 (A)S2S_2 (V)S3S_3 (A)S4S_4 (V)S5S_5 (AV)S6S_6 (AV)S7S_7 (AV)S8S_8 (AV)S9S_9 (AV), Final.ACCAvg.FORNew.ACC
PCLR65.29069.99566.95069.33571.90871.34570.01366.75167.1012.02868.739
ASAP-M66.56871.86368.10470.05072.52072.35570.90769.89968.9001.89470.281
ASAP-O66.56871.86369.19671.30272.54372.90671.37770.02069.5160.92870.179

We hope our responses have answered all your concerns. Please let us know if any point needs further detail, and we would be glad to discuss it.

[1] Z. Zheng et al. Preventing zero-shot transfer degradation. ICCV 2023.
[2] Y.-C. Yu et al. Select and distill. ECCV 2024.
[3] M. Zheng et al. Adapt without forgetting. ECCV 2024.
[4] Z. Gao et al. Stabilizing zero-shot prediction. NeurIPS 2024.
[5] Z. Chen et al. Forging a dynamic memory. CVPR 2026.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response

We sincerely thank all reviewers for their thoughtful and detailed feedback. We are pleased that the reviewers found the problem well motivated and the framework coherent and interpretable (jPkd, fAee, WS9A), and recognized the consistent improvements on both benchmarks (jPkd, fAee). The constructive suggestions led to a clean analysis of the MANOVA-guided selection, to new experiments on auxiliary-data transfer and longer objective streams, and to a clearer statement of our assumptions.

All three reviewers raised the same central question, whether the MANOVA-guided selection is necessary or whether the gains could come from simpler gradient-guided selection. To show the necessity and contribution of MANOVA-guided selection, we ran new experiments that change only the selection score and keep every other step of ASAP fixed.

These experiments, reported in Table R1 below, show two things. First, gradient magnitude does not explain the gains. Replacing ASAP's standardized effect score with the mean absolute gradient (MAG, SMT's statistic) lowers Final.ACC and New.ACC by about 1.5 points on both datasets, and replacing it with the mean squared gradient norm (MSG) lowers them by about 1.4 points. Second, both steps of ASAP's score contribute. Removing only the standardization, which gives the unstandardized factorial effect (UFE), lowers accuracy by about 0.8 points, and UFE in turn exceeds MAG by about 0.7 points. Avg.FOR is close for all selection scores, since they use the same protection mechanism. The gains are consistent across orders. ASAP leads Final.ACC and New.ACC on every tested order of both datasets, and the step from UFE to ASAP also holds across the three training seeds of Table R6 in our reply to Reviewer fAee. Simpler gradient-guided selection therefore does not reproduce ASAP's gains, and both steps of the MANOVA-guided score are needed.

These gains follow from two standard statistical principles. First, the factorial contrast is the step from MAG to UFE. In a factorial design, the effect of a factor is the contrast between the conditions with and without it on the same samples. ASAP applies it to the four input conditions (Equation 1, page 4), which assigns a block's modality role, whereas a gradient magnitude measures only the block's overall loss sensitivity. Second, standardization is the step from UFE to ASAP. In statistical testing, an estimated effect is judged relative to its standard deviation under the hypothesis of no effect, not by its raw size (Equation 2, page 4). A block with a large but inconsistent effect therefore ranks below a block with a smaller but consistent one, and the scores are comparable across weight matrices of different scale (Appendix A.2, page 18, and Proposition 1, page 17).

Table R2 below shows that the two steps also change the selection in different ways. UFE and MAG select about half of the same blocks but agree on their roles no more often than chance (about 26% against 25%), so the factorial contrast mainly changes the roles. ASAP shares 8–15% of its blocks with UFE and under 10% with MAG, so the standardization mainly changes which blocks are selected.

Based on all the evidence above, MANOVA-guided selection is necessary for ASAP's gains, and its contribution is clear. These results are in Section 5.3, Table 3 (page 9) and Appendix D.6 (page 34) of the revised PDF. Update: the remaining orders have finished. The complete six-order results on both datasets are in Table R11 of our update below and support the same analysis.

Table R1. Selection scores with everything else in ASAP fixed (CMU-MOSI averaged over orders P1P_1, P3P_3 and P5P_5, MUSIC-AVQA on order P3P_3).

Selection scoreMOSI Final.ACCMOSI Avg.FORMOSI New.ACCAVQA Final.ACCAVQA Avg.FORAVQA New.ACC
Random selection80.350.2680.2866.042.1068.36
MAG81.440.2181.2067.261.8068.99
MSG81.550.1681.2867.431.7869.13
UFE82.120.1781.9167.991.8469.72
ASAP82.930.1782.6968.791.8270.50

Table R2. Overlap between the selections of Table R1 (%), as blocks selected by both and the share of those given the same role (Appendix D.6, Table 17, page 36).

Selections comparedMOSI BlocksMOSI RoleAVQA BlocksAVQA Role
MAG and MSG65.675.164.571.6
UFE and MAG51.726.452.325.5
ASAP and UFE8.462.215.166.1
ASAP and MAG7.324.89.821.6
ASAP and random selection2.725.92.129.0
AuthorsOpenReview ↗

Summary of revisions in the uploaded PDF

We thank all the reviewers again for their time and effort. We have incorporated the feedback and the additional experiments into our revised manuscript, with all changes marked in blue, and we will carefully review it once more for the final version.

The revision includes the following changes.

  • Comparison of selection scores, with per-task accuracy, block interventions, overlap between selections and the allocation ablation (Section 5.3, Table 3 and Appendix D.6, Tables 14–17)
  • The two statistical principles behind the selection score, a worked example of the four input conditions and the rationale for equal quotas (Section 3.2)
  • Auxiliary-data setting and selection from a different dataset (abstract, Section 3.1 and Appendix D.2, Table 4)
  • Results over three training seeds (Appendix D.4, Table 7) and the preparation cost (Section 5.3 and Appendix D.7, Table 19)
  • Nine-objective experiment and the frozen fraction in longer sequences (Section 5.4, Appendices D.8 and D.9, Tables 20–21), with exploratory objective-specific records in Appendix D.9
  • Comparison with related methods, applicability and limitations (Sections 1 and 2, Limitations paragraph and Appendix E, Table 22)

The complete selection-score and three-seed results on all six orders are now in the PDF (Tables 3, 7 and 14).

AuthorsOpenReview ↗

Update: full results

We sincerely thank all reviewers again for their time and constructive suggestions. As promised, the remaining runs have finished, and the revised PDF (changes in blue) now includes them.

1. Selection scores on all six orders of both datasets (Table R11). The complete results also support our previous analysis of Table R1. Gradient magnitude does not explain the gains, and both steps of ASAP's score, the factorial contrast and the standardization, contribute. ASAP leads Final.ACC and New.ACC on all 12 dataset–order combinations.

Table R11. Selection scores with everything else in ASAP fixed, six-order means (Section 5.3, Table 3 and Appendix D.6, Table 14).

DatasetSelection scoreFinal.ACCAvg.FORNew.ACC
CMU-MOSIRandom selection80.270.3580.25
MAG81.330.2981.22
MSG81.450.2681.31
UFE82.020.2481.93
ASAP82.820.2582.72
MUSIC-AVQARandom selection67.081.5167.29
MAG67.961.1067.98
MSG68.071.1268.10
UFE68.671.0968.67
ASAP69.461.0769.47

2. Three training seeds on all six orders of both datasets (Table R12). The complete results also support our previous analysis of Table R6. ASAP's margins over the second-best method in Final.ACC and New.ACC are several times their standard deviation across seeds, with lower Avg.FOR.

Table R12. Three training seeds on all six orders (six-order mean ± standard deviation across seeds). Δ is the within-seed difference between ASAP and the second-best method (Appendix D.4, Table 7).

ComparisonMetricASAPSecond-best methodΔ
CMU-MOSI vs. DGGFinal.ACC82.869 ± 0.11580.532 ± 0.143+2.337 ± 0.175
Avg.FOR0.178 ± 0.0680.480 ± 0.084−0.302 ± 0.131
New.ACC82.770 ± 0.07680.399 ± 0.187+2.370 ± 0.253
MUSIC-AVQA vs. PCLRFinal.ACC69.467 ± 0.11067.533 ± 0.518+1.934 ± 0.407
Avg.FOR1.046 ± 0.0392.228 ± 0.278−1.181 ± 0.306
New.ACC69.334 ± 0.19867.954 ± 0.155+1.380 ± 0.292

Section and table numbers in the revised PDF are unchanged. Page numbers after Appendix D.4 have shifted by one to two pages.

Please let us know if any point needs further detail, and we would be glad to discuss it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.