acceptodds
Under review as a conference paper at ICLR 2027

BiCLoRA: Two-Sided Constrained LoRA for Audio-Video-Text Continual Learning

Abstract

Pretrained audio-video-text models align complementary auditory, visual, and linguistic information within a shared representation space, enabling unified recognition of real-world events. In deployment, these models must continually acquire concepts beyond their pretraining data, often without retaining past observations because of privacy and storage constraints. We formulate this problem as exemplar-free Audio-Video-Text Continual Learning (AVT-CL), which extends an aligned multimodal space while preserving prior knowledge and cross-modal alignment. We identify two failure modes of continual low-rank adaptation: *historical interference*, where the input factor overlaps with activation subspaces occupied by previous sessions, and *alignment drift*, where a freely trained output factor writes new information into arbitrary shared-space directions, even when the input side is protected. To address both, we propose *Bilaterally Constrained Low-Rank Adaptation* (**BiCLoRA**), which fixes both factors before optimization. A history-safe input basis derived from a bounded eigensummary of past projection inputs restricts where updates may act, while a frozen output basis spanning the leading directions of current-session representations restricts where new information may be written. Only a small coupling kernel for each modality is optimized and subsequently merged into the current projection. Across three pretrained AVT backbones and three benchmarks, BiCLoRA improves macro final average accuracy from to and backward transfer from to over the strongest matched adaptation baseline. Ablations confirm that the input constraint protects historical representations, the output constraint preserves cross-modal alignment, and effective retention requires both.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Incremental Learning, Catastrophic Forgetting, Multi-modal Learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_MPzXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies exemplar-free Audio-Video-Text Continual Learning. The goal is to continually extend the shared representation space of a pretrained audio-video-text model without storing historical samples, while preserving previously acquired knowledge and cross-modal alignment. The authors argue that existing low-rank continual adaptation methods mainly constrain the input side, which still leaves room for alignment drift on the output side. To address this, the paper proposes BiCLoRA, which imposes two-sided constraints on low-rank updates through a history-safe input basis and a frozen output basis, while only training the coupling kernel in between. The experiments show that the method consistently outperforms the compared LoRA-based baselines on FAA and BWT. The paper also supports its main claim with ablations, visualizations, and forgetting case analysis.

Critical strengths
  • The problem setting is clear and well motivated. The paper focuses on exemplar-free AVT continual learning and clearly distinguishes this setting from visual CL, vision-language CL, and audio-visual CL.
  • The method design matches the diagnosis of the problem. The paper identifies two failure modes, historical interference on the input side and alignment drift on the output side, and addresses them with the history-safe input basis and the frozen output basis respectively.
  • The experiments and ablations are fairly convincing. Table 3 clearly shows that the input-side and output-side constraints are complementary. Removing either one leads to a noticeable drop in FAA and BWT, and removing both causes a large degradation. This provides solid support for the proposed method.
  • The method is parameter-efficient. BiCLoRA uses only 128 trainable adaptation parameters per session while keeping a bounded persistent state, making it lighter than methods such as InfLoRA and KeepLoRA.
Critical weaknesses
  • The novelty is more about a well-structured combination of mechanisms, while the theoretical support remains limited. The main contribution is to unify input-side protection and output-side freezing within one LoRA framework, but beyond the perturbation bound on the input side, the theoretical explanation for the output basis design is still relatively weak.
  • The presentation of the baselines could be clearer. The appendix states that some baselines use matched implementations rather than their original forms. This helps isolate variables, but it also makes the fairness of the main comparison less transparent.
  • The diversity of the continual learning tasks and the generalizability of the proposed method are somewhat limited. The current evaluation is limited to a classification-style AVT-CL setting, and the authors leave overlapping concepts, missing modalities, and broader distribution shifts as future work, so the generalizability of the method remains to be verified. And I feel that more forward-looking multimodal continual learning research should be conducted on MLLMs. Compared with classification tasks, validation in an MLLM setting would be more meaningful.
Questions for authors

See above

2 comments
AuthorsOpenReview ↗

Response to Reviewer MPzX

Thank you for the positive assessment of the setting, complementary constraints and parameter efficiency. Changes are marked in blue in the revised PDF. The revision adds output-side analysis and direct controls while clarifying the scope of the evidence.

W1 Output-side explanation and contribution

For a historical input h=Pα+ϵh=P\alpha+\epsilon and a unit semantic direction ww, Eq. (9) gives ∣w⊤ΔWh∣≤∥C⊤U⊤w∥2∥ϵ∥2|w^\top\Delta Wh|\le\lVert C^\top U^\top w\rVert_2\lVert\epsilon\rVert_2.

The factors describe directional sensitivity and the residual outside the retained history. Orthonormal VV makes the first coefficient equal to ∥ΔW⊤w∥2\lVert\Delta W^\top w\rVert_2, directly comparable with a freely trained output map. Eq. (12) includes the normalization term needed for cosine similarity. The bounds characterize the output-side sensitivity; the choice of basis is tested directly in Table 8.

Table 8 supplies the complementary empirical evidence. At identical mean audio/video Frobenius norms of 0.22/0.31 on ImageBind SSW60, BiCLoRA has drift 0.020 and audio-video cosine 0.25, compared with 0.058 and 0.18 for matched D1. Across the four amplitude controls, the method retains 0.47 to 0.90 macro FAA and 0.66 to 1.34 BWT.

The larger-update test increases the kernel learning rate eightfold. Macro FAA/BWT falls to 71.43/-9.09, but at mean norms 1.18/1.65 its ImageBind SSW60 cosine/drift are 0.20/0.055. Default D1 has norms 1.08/1.54 and cosine/drift -0.13/0.380. Thus the alignment advantage does not depend on the update being small.

The methodological contribution is the asymmetric construction of the bases from historical inputs and current-session outputs. At the same 128 adaptation parameters, the data-derived output basis adds 0.89 FAA and 1.24 BWT over the random basis in Table 3.

W2 Baseline presentation

Table 2 explicitly separates matched-history InfLoRA at 69.08/-13.17 from the AVT transfer restoring its original history rule at 69.75/-11.32. BiLoRA achieves 69.78/-13.33. BiCLoRA reaches 72.21/-7.72 and improves FAA over BiLoRA in all nine settings and over original-history InfLoRA in eight. Appendix C specifies the common adaptation sites, optimization budget and classifier.

The revised abstract distinguishes FAA gains of 2.46 and 2.43 over these transfers from the 3.13-point matched-history gain. Appendix B.1 labels its shared-MLP LADA variant as a LADA-inspired feature-space control rather than a native-method reproduction.

W3 Scope and MLLMs

The common-classifier protocol enables controlled study of representation adaptation across disjoint-class, exemplar-free AVT streams. The tested setting uses paired audio and video with class-name guidance. Sec. 5 explicitly identifies overlapping concepts, missing modalities, broader distribution shifts and instruction-following MLLM streams as unevaluated extensions.

A compatible linear projector is a natural place to investigate the bilateral parameterization in an MLLM. Token-level objectives and downstream language-model behavior require separate evaluation. The current contribution is the bounded-state adaptation rule and its controlled AVT analysis, not an untested claim of MLLM generalization.

AuthorsOpenReview ↗

Follow-up on our response, Reviewer MPzX

Thank you again for the positive assessment. Eq. 9 and Eq. 12 now give the output-side bound, Table 2 separates matched variants from original forms, and Sec. 5 states the scope of the evidence. As the discussion period closes today, we would be grateful for a short note on whether this addresses your concerns.

Reviewer_qxFSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies exemplar-free audio-video-text class-incremental learning and proposes BiCLoRA, a bilaterally constrained low-rank adaptation method. It combines a history-safe input basis with a frozen output basis to mitigate historical interference and cross-modal alignment drift, adapting modality projections through small coupling kernels. Experiments across three pretrained backbones and three benchmarks demonstrate improved classification accuracy and reduced forgetting.

Critical strengths
  1. A coherent and efficient approach to retaining multimodal representations. The method combines bounded historical state, history-aware input directions, and a fixed output subspace in a mergeable update. With rank 8 per modality, only 128 adaptation parameters are trained across the two perceptual branches, excluding the temporary classifier. 
  2. Consistent improvements with meaningful component-level evidence. BiCLoRA achieves the best BWT in all nine backbone-benchmark combinations and the best FAA in eight.
Critical weaknesses
  1. The main comparison does not establish the value of adapting the representations over keeping them frozen. Table 2 evaluates several adaptation methods under a common analytic classifier, but does not include a control that leaves both pretrained perceptual representations unchanged while updating exactly the same classifier. Consequently, the reported gains demonstrate an advantage over the evaluated adaptation alternatives, but do not establish an advantage over the simplest matched alternative. This is consequential because BiCLoRA strongly restricts representation changes, and the classifier accumulates statistics from representations produced at different stages. A matched frozen-representation control is needed to assess the practical benefit of the adaptation itself. 
  2. The output-side alignment explanation is not fully isolated from general restrictions on the update. Removing Design 2 changes both the admissible output directions and the number of trainable parameters, from an r-by-r kernel to a p-by-r output map. The random frozen-output-basis control in Table 3 is useful and reaches 71.32 macro FAA, compared with 72.21 for the proposed basis, showing that freezing the output subspace already accounts for a substantial part of the gain. However, these results and Figure 4 do not establish whether the alignment benefit persists when effective update magnitudes are comparable. Moreover, the reported geometry diagnostics primarily concern audio-video alignment, whereas the paper motivates preservation of an audio-video-text space. Eq. (7) bounds pre-normalization input perturbations, rather than cross-modal alignment. The interpretation would be stronger with update-norm-matched comparisons and direct measurements of historical audio-text and video-text alignment.
Questions for authors
  1. Matched frozen baseline: What are FAA and BWT across the nine settings when both perceptual representations remain frozen and the exact same analytic classifier, random projection, and sufficient-statistics update rule from Table 2 are used? Does BiCLoRA consistently outperform this control? 
  2. Output-side mechanism: How do the learned update norms compare among BiCLoRA, its random-output-basis variant, and the input-protected variant with a trainable output map? Do the accuracy and alignment advantages persist at comparable update magnitudes? Are historical audio-text and video-text relationships also preserved? 
  3. Contribution of text: What happens when text guidance is removed from both the loss and the temporary-classifier initialization? This control would clarify how much text contributes beyond the bilateral constraints on audio-video adaptation.
5 comments
AuthorsOpenReview ↗

Response to Reviewer qxFS, Part I

Thank you for the precise questions. Changes are marked in blue in the revised PDF. D1 denotes matched-history InfLoRA with a trainable output map. This comment answers Q1 and the magnitude component of Q2. The companion comment addresses historical alignment, the bound and Q3.

Q1 Matched frozen baseline

Table 2 includes frozen perceptual representations with the identical analytic classifier, a random projection of dimension 2,048, ridge coefficient of 1,000, class order and sufficient-statistics update. The realization of the random projection is paired across methods.

SettingFrozen FAABiCLoRA FAAFrozen BWTBiCLoRA BWT
IB K60069.1271.74-7.31-8.73
IB VGGSound61.4863.39-9.18-11.21
IB SSW6060.8461.67-4.14-4.96
LB K60078.3679.71-6.18-6.97
LB VGGSound67.4269.02-8.94-11.01
LB SSW6068.9169.94-6.89-7.92
EB K60083.1884.25-4.78-5.67
EB VGGSound66.8167.64-8.11-9.94
EB SSW6083.4282.51-2.31-3.05
Macro71.0672.21-6.43-7.72

BiCLoRA improves FAA in eight settings by 1.15 points on average. Frozen features lead by 0.91 on EBind SSW60 and have better macro BWT. With P=T−1∑kAk,kP=T^{-1}\sum_k A_{k,k} the mean acquisition accuracy, the definitions of FAA and BWT give the exact identity in Eq. (11): FAA=P+T−1TBWT\mathrm{FAA}=P+\frac{T-1}{T}\mathrm{BWT}.

Because FAA and BWT are computed from the same accuracy matrix, applying it with each setting's own TT gives PP exactly: 79.54 for BiCLoRA and 77.16 for frozen features. The higher acquisition accuracy leaves a positive final-accuracy gain after subsequent changes. Frozen representations still update their classifier, so their BWT reflects classifier evolution and candidate-set expansion without representation drift. The identity separates the two accuracy terms, rather than isolating a causal effect of drift.

Q2 Update magnitude and output directions

On ImageBind SSW60, Table 8 reports mean per-session audio/video update norms of 0.22/0.31 for BiCLoRA, 0.16/0.23 for the random basis and 1.08/1.54 for D1. The four controls give

D1 controlMacro FAAMacro BWTBiCLoRA FAA gainBiCLoRA BWT gain
Normalized-feature RMS matching71.74-8.380.470.66
Smaller adaptation learning rate71.58-8.640.630.92
Update Frobenius-norm matching71.45-8.860.761.14
Fixed merge multiplier of 0.2471.31-9.060.901.34

Frobenius matching is per session and modality. Feature matching uses RMS Euclidean displacement of normalized pooled features on current-session training inputs, relative to each method's own pre-session model. The smaller learning rate is 1.2×10−41.2\times10^{-4}. The fixed scale uses the reference mean-norm summaries but no session-specific reference during execution.

Each intervention reruns the full stream and accumulates classifier statistics from post-merge representations. No historical test data select the scales. Differences above use displayed macro values.

Magnitude control recovers much of the original accuracy gap. Random directions give 71.32 FAA, close to fixed-scale D1 at 71.31. At the same 128 trainable adaptation parameters, the data-derived basis adds 0.89 FAA and 1.24 BWT over random directions.

Q2 Larger-update controls

The reverse intervention rescales BiCLoRA to D1's per-session norm schedule and gives 71.47 FAA and -8.90 BWT. On ImageBind SSW60, its mean norms are 1.08/1.54, matching D1, yet drift is 0.041 versus 0.380 and audio-video cosine is 0.22 versus -0.13.

Increasing BiCLoRA's kernel learning rate eightfold is a separate full-stream experiment. Macro FAA/BWT decreases to 71.43/-9.09. On ImageBind SSW60, mean update norms are 1.18/1.65 and cosine/drift are 0.20/0.055. Default BiCLoRA has cosine/drift 0.25/0.020. Thus larger updates affect accuracy and geometry, but the alignment advantage over D1 persists at comparable mean parameter-update magnitudes.

AuthorsOpenReview ↗

Response to Reviewer qxFS, Part II

This companion comment completes Q2 and Q3. The same revisions are marked in blue in the revised PDF. Appendix D evaluates fixed first-session held-out samples after the complete stream. Table 8 reports ImageBind SSW60 diagnostics alongside nine-setting macro FAA/BWT. Text-prototype queries use all seen classes and are separate from the analytic classifier.

Q2 Historical audio-text and video-text relationships

Across nine settings, Appendix D reports mean audio-text/video-text cosine of 0.220/0.334 for BiCLoRA, 0.236/0.360 for frozen features and 0.140/0.228 for D1. The detailed ImageBind SSW60 comparison in Table 8 is

MethodAudio-textVideo-textAudio-videoJoint drift
Frozen0.180.360.320.000
BiCLoRA0.160.330.250.020
Random output basis0.170.350.310.007
D10.080.19-0.130.380
Frobenius-matched D10.140.290.180.058
Feature-matched D10.150.320.230.031
Original-history InfLoRA0.100.23-0.060.290

BiCLoRA stays within 0.02/0.03 of frozen audio-text/video-text cosine. Matching update magnitude improves D1 substantially, while the accuracy and BWT margins remain. The random basis preserves geometry particularly well. Its lower FAA separates near-frozen geometry from useful adaptation.

MethodAudio queryVideo queryJoint query
Frozen9.6273.4458.85
BiCLoRA9.1070.8357.29
D15.7745.8337.50
Frobenius-matched D18.3366.1552.08
Original-history InfLoRA6.4153.6544.79

BiCLoRA's video-query accuracy is 2.61 points below frozen features, but 25.00 above D1 and 17.18 above original-history InfLoRA. Its joint-query accuracy is 1.56 below frozen features. These report both the remaining loss and the historical compatibility retained relative to free output adaptation.

Q2 Scope of the theoretical bound

Eq. (7) concerns pre-normalization perturbations. The revision adds the directional bound in Eq. (9) and its normalized form in Eq. (12). For one modality, let h=Pα+ϵh=P\alpha+\epsilon and define, for a unit old-class text vector, qc=∥C⊤U⊤tc∥2=∥ΔW⊤tc∥2q_c=\lVert C^\top U^\top t_c\rVert_2=\lVert\Delta W^\top t_c\rVert_2.

Orthonormal VV gives the equality, and D1 has the comparable coefficient ∥B⊤tc∥2\lVert B^\top t_c\rVert_2. Table 8 reports 0.024/0.058 for BiCLoRA, 0.151/0.365 for default D1 and 0.031/0.073 for Frobenius-matched D1, in audio/video order. This coefficient contains both update magnitude and direction.

Eq. (9) bounds ∣tc⊤ΔWh∣|t_c^\top\Delta Wh| by qc∥ϵ∥2q_c\lVert\epsilon\rVert_2. To address normalization, let xx be the full pre-normalization output, δ=ΔWh\delta=\Delta Wh, z=x/∥x∥2z=x/\lVert x\rVert_2 and z′=(x+δ)/∥x+δ∥2z'=(x+\delta)/\lVert x+\delta\rVert_2. For nonzero xx and x+δx+\delta, Eq. (12) gives ∣tc⊤(z′−z)∣≤qc∥ϵ∥2+∣tc⊤z∣∥δ∥2∥x+δ∥2|t_c^\top(z'-z)|\le\frac{q_c\lVert\epsilon\rVert_2+|t_c^\top z|\lVert\delta\rVert_2}{\lVert x+\delta\rVert_2}.

The second term accounts for the normalization change. Both parameterizations obey the Cauchy-Schwarz bound. Numerical checks establish implementation consistency, not optimality of the chosen basis. The direct magnitude-controlled cosine and zero-shot comparisons provide the empirical evidence.

Q3 Contribution of text

Table 9 removes text from both the loss and temporary-classifier initialization. BiCLoRA gives 71.66 FAA and -8.22 BWT, versus 72.21/-7.72 by default. D1 under the same objective gives 68.73/-13.42. The remaining FAA advantage is 2.93 points using displayed values.

Random classifier rows have the same initial norm as text-initialized rows. Keeping text initialization but removing only the explicit text loss gives 72.20 FAA. Under this one-epoch protocol, initialization accounts for most of the 0.55-point contribution of continual text guidance. Table 9 contains all four supervision choices. None removes the backbone's original multimodal pretraining.

Thank you for adding the matched frozen baseline, the update-magnitude controls, and the text ablations. These experiments address my requests for additional comparisons.

My main reservation is the trade-off relative to the matched frozen baseline. BiCLoRA improves mean acquisition accuracy from 77.16 to 79.54 and final average accuracy from 71.06 to 72.21, with higher FAA in eight of nine settings. Its BWT is more negative, at −7.72 versus −6.43. I agree that BWT also reflects classifier updates and the expanding label set. Taken together, the results show better acquisition and a modest final accuracy gain, alongside a larger subsequent drop from the higher acquisition accuracy.

The matched-magnitude experiments also support an alignment benefit from the output constraint. The remaining gains of 0.47–0.90 points in FAA and 0.66–1.34 points in BWT over the magnitude-controlled variants are modest. I appreciate the stronger experimental evidence, but the limited final accuracy gain and the trade-off against the matched frozen baseline lead me to retain my initial rating.

Reviewer_SeAnOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes BiCLoRA for audio-video-text continual learning. It constrains LoRA updates on both the input and output sides to reduce interference with previous tasks and preserve cross-modal alignment, while training only a small core matrix. Combined with an analytic classifier, BiCLoRA achieves higher average accuracy and less forgetting than the compared baselines across most evaluated settings.

Critical strengths
  1. The method achieves good overall performance across the evaluated settings.
  2. The appendix provides detailed experimental settings and additional ablation results.
Critical weaknesses
  1. The main idea is close to earlier work. DNS [1] already uses two-sided projection for multimodal continual learning, while FM-LoRA [2] and BiLoRA [3] use fixed bases on both sides for continual adaptation. These papers are not discussed, leaving the difference between BiCLoRA and existing methods unclear.

  2. The main experiments do not include keeping the pretrained features frozen and updating only the same classifier. This leaves a basic question unanswered: how much does changing the features actually help? The no-adaptation results in the appendix use a different classifier, so they cannot answer this question.

  3. The output constraint also reduces the number of trainable adaptation parameters from 16,384 to 128. Giving the model much less freedom to change could itself explain why it forgets less. A random fixed output basis already reaches 71.32% accuracy, close to BiCLoRA’s 72.21%, so the results do not clearly separate this effect from the benefit of the proposed output directions.

  4. Most of the alignment results measure the relationship between audio and video. There is little direct evidence that their relationships with text are also preserved. The paper also does not show what happens when text guidance is removed completely, so the contribution of text remains unclear.

  5. Some baselines differ substantially from their original methods. For example, InfLoRA’s [4] history mechanism is changed, and LADA [5] is implemented as a shared MLP rather than its original label-specific memory design.

  6. It is recommended to reduce the proportion of AI usage. The typesetting format is incorrect (fig.5).

[1] Xiaohao Liu, Xiaobo Xia, See-Kiong Ng, and Tat-Seng Chua. Continual Multimodal Contrastive Learning. NeurIPS, 2025.

[2] Xiaobing Yu, Jin Yang, Xiao Wu, Peijie Qiu, and Xiaofeng Liu. FM-LoRA: Factorized Low-Rank Meta-Prompting for Continual Learning. CVPR Workshops, 2025.

[3] Hao Zhu, Yifei Zhang, Junhao Dong, and Piotr Koniusz. BiLoRA: Almost-Orthogonal Parameter Spaces for Continual Learning. CVPR, 2025.

[4] Yan-Shuo Liang and Wu-Jun Li. InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning. CVPR, 2024.

[5] Mao-Lin Luo, Zi-Hao Zhou, Tong Wei, and Min-Ling Zhang. LADA: Scalable Label-Specific CLIP Adapter for Continual Learning. ICML, 2025.

Questions for authors

See the above weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer SeAn

Thank you for the detailed review and the references. Changes are marked in blue in the revised PDF. The revision distinguishes the subspace constructions, tests optimization controls and clarifies which comparisons are transfers under a common AVT interface.

W1 Related methods and novelty

Appendix A now discusses DNS, FM-LoRA and BiLoRA. DNS projects the gradient of each modality projection out of the subspaces spanned by its own historical inputs and by the historical outputs of the paired modality. FM-LoRA learns shared input and output bases on the first incremental task, freezes them, and trains task-specific coefficients with dynamic rank selection and meta-prompting. BiLoRA fixes both factors to Fourier bases and trains a sparse set of task-specific frequency coefficients.

BiCLoRA reconstructs its bases at each session from different information. Historical inputs determine excluded input directions, while current-session outputs of the consolidated model determine permitted output directions. A dense kernel couples the two bases and is then merged, retaining only a fixed-budget history summary. This asymmetric, stream-dependent construction is the distinction, rather than bilateral factorization alone. Table 2 includes the BiLoRA transfer at 69.78 macro FAA and -13.33 BWT. BiCLoRA reaches 72.21/-7.72 and improves FAA in all nine settings.

W3 Parameter count and magnitude

A random fixed output basis uses the same 128 parameters and reaches 71.32/-8.96; the current-session basis adds 0.89 FAA and 1.24 BWT. Removing the history-safe input construction, also at 128 parameters, gives 70.39/-12.25 (Table 3). If the parameter count alone explained retention, these three would be equal.

Table 8 directly tests a smaller learning rate. Reducing D1's adaptation learning rate to 1.2×10−41.2\times10^{-4} gives 71.58/-8.64. Together with feature-displacement matching, Frobenius matching and a fixed merge multiplier of 0.24, BiCLoRA leads all four controls by 0.47 to 0.90 FAA and 0.66 to 1.34 BWT. The temporary classifier's learning rate is unchanged. Appendix D defines each intervention and recomputes classifier statistics from post-merge features throughout the stream.

At eight times BiCLoRA's kernel learning rate, macro FAA/BWT falls to 71.43/-9.09 and mean ImageBind SSW60 update norms reach 1.18/1.65, versus D1's 1.08/1.54, yet audio-video cosine is 0.20 versus -0.13 and drift is 0.055 versus 0.380. The alignment advantage remains at comparable mean update magnitudes.

W2 Matched frozen control

The frozen row of Table 2 uses the identical analytic classifier, random projection and sufficient-statistics rule. It achieves 71.06 FAA and -6.43 BWT. BiCLoRA improves FAA in eight settings by 1.15 points, with EBind SSW60 the exception. The exact identity in Eq. (11), applied per setting, gives mean acquisition accuracy of 79.54 versus 77.16. BWT is measured from each session's own peak, which adaptation raises by 2.38 points at acquisition; the FAA lead is the net gain that remains after the subsequent decline, and since the newest session contributes only 1/T of FAA, it is carried mainly by previously learned classes.

W4 Historical text alignment and removal

Across nine settings, Appendix D reports mean audio-text/video-text cosine of 0.220/0.334 for BiCLoRA, 0.236/0.360 for frozen features and 0.140/0.228 for D1. On ImageBind SSW60, BiCLoRA's zero-shot accuracy with video queries is 70.83, 2.61 below frozen features but 25.00 above D1 and 17.18 above original-history InfLoRA. These use held-out samples from the first session after the full stream.

Removing text from both continual-training channels gives BiCLoRA 71.66 FAA and D1 68.73 in Table 9, retaining a 2.93-point advantage from displayed values.

W5 Baseline fidelity

Table 2 explicitly labels matched-history InfLoRA, which equals D1. The separate transfer restoring the original history rule achieves 69.75 FAA and -11.32 BWT. The abstract and Introduction name its 2.46-point FAA gap and BiLoRA's 2.43-point gap separately from the 3.13-point matched-history comparison. Appendix C specifies the shared AVT adaptation sites, optimization budget and classifier.

Appendix B.1 describes the LADA-inspired control as a residual transformation of fused features using a shared MLP and a penalty of 0.05 times the induced cosine displacement on previous class prototypes. It explicitly distinguishes this from original LADA's label-specific memory and feature distillation. Its evidence concerns the matched frozen text-prototype head, not a native-method reproduction or the Table 2 classifier.

Presentation

Figure 5 now uses a conventional layout, with panel labels above the panel titles, titles outside the heat maps and larger tick labels. Appendices A to C separate related work, feature-space transfer controls and baseline implementations, and Sec. 5 explicitly lists MLLM streams as unevaluated.

AuthorsOpenReview ↗

Follow-up on our response, Reviewer SeAn

Thank you again for the detailed review. As the discussion period closes today, we would appreciate a brief indication of whether the response addresses W1 to W5. Appendix A now distinguishes DNS, FM-LoRA and BiLoRA by where their bases come from, and BiLoRA is evaluated under the common interface at 69.78 versus 72.21 FAA. Table 2 adds the frozen control and the original InfLoRA history rule. Table 8 holds the update magnitude fixed in four ways, and BiCLoRA leads each by 0.47 to 0.90 FAA and 0.66 to 1.34 BWT. Table 9 removes text from both the loss and the classifier initialization, and the output constraint still adds 2.93 FAA. Figure 5 is re-typeset.

I thank the authors for their responses. I have also reviewed their replies to the other reviewers, which have addressed most of my concerns. However, I remain unconvinced by the paper’s novelty, and Table 2 in the PDF still has formatting issues. Taking the clarifications into account, I have decided to raise my score from 1 to 2.

AuthorsOpenReview ↗

Thank you for the updated assessment

Thank you for revisiting the paper and for the updated score. Table 2 has been re-typeset in the revised PDF. The frozen control is now set as a reference row within each backbone block, every block lists the same methods with KeepLoRA marked as not applicable on EBind, and the caption states which entries carry a standard deviation. On novelty, Appendix A now places BiCLoRA next to DNS, FM-LoRA and BiLoRA by where each method obtains its bases, and the distinction we draw is that BiCLoRA rebuilds both bases at every session from the stream itself, historical inputs on one side and current-session outputs of the consolidated model on the other. We appreciate the time you spent on the paper and on the responses to the other reviewers.

Reviewer_UBVwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies replay-free continual learning and introduces an audio-visual-text continual learning setting (AVT-CL). During each session, the model is trained with a temporary classifier together with a similarity loss to fixed class-text embeddings, and predictions are made by a cumulative analytic classifier. The main method, BiCLoRA, builds on InfLoRA. InfLoRA only protects the input side, so historical inputs that fall outside the protected subspace still leave residual interference. BiCLoRA adds a second constraint on the output side: the low-rank update is written as UCV⊤UCV^\top, and its output directions are restricted to the highest-energy directions of the pre-adaptation embeddings of the current session. Experiments with several pretrained models and benchmarks show improvements in both accuracy and retention over the compared methods.

Critical strengths
  1. The replay-free continual learning setting is highly relevant in practice.
  2. The experimental results are exciting, showing improvements in accuracy and retention across multiple pretrained models and benchmarks.
  3. The experiments are extensive, and the experimental design is reasonable.
Critical weaknesses
  1. I find the current contributions rather incremental. The AVT-CL setting adds supervision based on class-text embedding similarity to audio-visual continual learning, following the idea of CLIP. I'm not convinced this is a new setting rather than an existing task with one more supervision signal. On the method side, BiCLoRA adds a matrix that restricts the output directions to an input-protected low-rank update. It is unclear why this additional constraint is effective.
  2. The link between the motivation and the output constraint is weak. This is my main concern. The motivation identifies residual interference caused by an incomplete historical input subspace, but the method restricts updates to the leading directions of current-session output embeddings. The paper does not explain why these directions should address that residual interference or help preserve old knowledge. More analysis is needed to link this design to the problem it is meant to solve.
  3. The experiments do not cleanly separate the contributions of the supervision scheme and the adaptation method. I'd like to see classifier-only supervision, class-text similarity supervision alone, and their combination compared under the same adaptation method, followed by a comparison of the LoRA methods under the same supervision. Without these controls, it is hard to tell how much each part (the combined supervision and the output-direction constraint) contributes.
  4. The main comparison lacks a fully frozen representation baseline using the same cumulative analytic classifier. The zero-adaptation controls in the appendix use a different prediction interface, so they do not show the benefit of adaptation under the main evaluation setup. Several baselines, including InfLoRA, have also been modified, so the paper should clearly separate controlled component comparisons from comparisons against the original methods.
Questions for authors
  1. Could you clarify the novelty of AVT-CL beyond adding CLIP-style text-similarity supervision, and of BiCLoRA beyond adding an output-direction constraint?
  2. Why does restricting updates to the leading current-session output directions reduce forgetting caused by residual historical inputs? Please provide analysis linking this constraint to the motivation.
  3. Could you provide clean ablations separating the gains from classifier supervision, text-similarity supervision, their combination, and the additional output-direction constraint under matched supervision?

If these concerns are addressed, I will consider raising my score.

2 comments
AuthorsOpenReview ↗

Response to Reviewer UBVw

Thank you for the clear questions. Changes are marked in blue in the revised PDF. We address the connection between residual interference and the output constraint first. D1 denotes the matched-history input-protected variant.

Q2 Connecting the motivation and the output constraint

For one modality, write a historical input as h=Pα+ϵh=P\alpha+\epsilon, with V⊤h=V⊤ϵV^\top h=V^\top\epsilon. Eq. (9) gives, for any unit direction ww, ∣w⊤ΔWh∣≤q(w)∥ϵ∥2|w^\top\Delta Wh|\le q(w)\lVert\epsilon\rVert_2, where q(w)=∥C⊤U⊤w∥2=∥ΔW⊤w∥2q(w)=\lVert C^\top U^\top w\rVert_2=\lVert\Delta W^\top w\rVert_2.

The equality uses orthonormal VV. This connects the historical residual to the update's sensitivity along an old semantic direction. D1 has the directly comparable coefficient ∥B⊤w∥2\lVert B^\top w\rVert_2. The output span is fixed in BiCLoRA, while the kernel learns the mapping within that span. Eq. (12) additionally accounts for feature normalization.

On ImageBind SSW60, Table 8 reports audio/video text-direction coefficients of 0.024/0.058 for BiCLoRA and 0.151/0.365 for default D1. At matched Frobenius norms of 0.22/0.31, D1’s coefficients are still 29% and 26% higher, its drift is 2.9 times larger at 0.058 versus 0.020, and its audio-video cosine is 0.18 versus 0.25.

The reverse comparison is also in Table 8. Rescaling BiCLoRA to D1’s norm schedule keeps drift at 0.041 and audio-video cosine at 0.22. Raising the kernel learning rate eightfold brings mean norms to 1.18/1.65, above D1’s 1.08/1.54, yet cosine remains 0.20 versus D1’s -0.13. Both experiments show that the alignment advantage comes from the fixed output span rather than from a smaller update.

Q1 Method novelty

The contribution is the asymmetric, stream-dependent construction of the two fixed bases, not the factorization UCV⊤UCV^\top alone. Current activations outside the bounded historical summary define the input basis. Current-session representations of the consolidated model define the output basis. Only the coupling kernel is trained, and the update is merged without retaining task-specific adapters.

The random-output and no-Design-1 variants also use 128 adaptation parameters. They achieve 71.32/-8.96 and 70.39/-12.25, compared with 72.21/-7.72 for BiCLoRA in Table 3. Thus basis construction affects the outcome even at the same parameter count. Table 8 shows that this margin persists when D1 is matched to BiCLoRA's update magnitude.

Q1 Setting novelty

AVT-CL jointly adapts two perceptual branches from paired audio-video observations and class names under an exemplar-free, disjoint-concept stream. Text guides training, but inference uses fused audio-video representations and a common analytic classifier. The question is how both branches acquire concepts while preserving historical representations and their shared geometry with text. Table 9 and the text-alignment columns of Table 8 distinguish this representation-learning problem from merely adding a text loss. The contribution is the controlled protocol, its failure analysis and the corresponding adaptation rule.

Q3 Supervision controls

Table 9 crosses four objectives with both adaptation rules. Entries are macro FAA / BWT under the same inference classifier.

SupervisionD1BiCLoRA
CE and random classifier initialization68.73 / -13.4271.66 / -8.22
CE and text initialization, no text loss69.08 / -13.1672.20 / -7.73
Text-similarity CE only68.06 / -14.3271.40 / -8.51
Both terms69.08 / -13.1772.21 / -7.72

The output constraint adds 2.93 to 3.34 FAA across these objectives using displayed values. Text guidance adds 0.55 FAA and 0.50 BWT on top of the constraint, mostly through classifier initialization, so the output constraint, not the supervision, accounts for the gain. All rows retain the backbone's multimodal pretraining. Table 2 compares the adaptation methods under the common default objective.

W4 Frozen control and baseline forms

Table 2 includes frozen representations at 71.06 macro FAA, original-history InfLoRA at 69.75 and BiLoRA at 69.78. BiCLoRA improves on frozen representations in eight settings and exceeds the two transfers by 2.46 and 2.43 macro FAA. Appendix C distinguishes the matched-history comparison from restoring the original history rule under the common AVT interface.

AuthorsOpenReview ↗

Follow-up on our response, Reviewer UBVw

Thank you again for the clear questions. With the discussion period closing today, we would appreciate a short note on whether the three points are addressed. For Q2, Eq. 9 links the historical residual to the update’s sensitivity along old directions, and Table 8 reports coefficients of 0.024/0.058 for BiCLoRA versus 0.151/0.365 for the trainable output map. For Q3, Table 9 crosses four supervisions with both adaptation rules, and the output constraint adds 2.93 to 3.34 FAA under each. For W4, Table 2 now includes the frozen control at 71.06 versus 72.21 FAA, together with the original forms of InfLoRA and BiLoRA. We are glad to clarify anything further today.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response on the frozen control, the output constraint, text supervision and baselines

We thank all reviewers for their constructive feedback. Reviewers found the exemplar-free setting practically relevant (UBVw, MPzX), the experiments extensive with consistent gains (UBVw, SeAn, qxFS, MPzX), and the method coherent and parameter-efficient (qxFS, MPzX). Changes are marked in blue in the revised PDF. The revision adds the frozen and transferred-method controls in Table 2, magnitude and geometry diagnostics in Table 8, and supervision controls in Table 9. D1 denotes matched-history InfLoRA, which protects the input side while learning a free output map. References below use the revised numbering.

Summary of revisions and findings

Question (reviewers)Revision (where)Main finding
Frozen control (UBVw, SeAn, qxFS)Matched classifier, Table 2 and Eq. (11)BiCLoRA reaches 72.21 FAA versus 71.06, leads in 8/9 settings and obtains 79.54 versus 77.16 acquisition accuracy.
Output constraint and theory (UBVw, qxFS, MPzX)Directional and normalized bounds, Eqs. (9), (12) and Table 8The bounds connect output sensitivity to historical residuals; FAA, BWT and alignment margins over D1 persist at matched update norms.
Update magnitude and bases (SeAn, qxFS, MPzX)Four D1 controls and random-basis ablation, Tables 3 and 8FAA/BWT gains remain 0.47-0.90/0.66-1.34; the output basis adds 0.89 FAA over random directions.
Text supervision and alignment (UBVw, SeAn, qxFS)Alignment probes and supervision grid, Tables 8, 9 and Appendix DRemoving text loss and initialization gives 71.66 FAA versus D1's 68.73; video-query accuracy is 70.83.
Baselines and related work (UBVw, SeAn, MPzX)DNS/FM-LoRA/BiLoRA discussion and original-history transfers, Table 2 and Appendices A-CFAA gains are 2.46 over original-history InfLoRA and 2.43 over BiLoRA.
Scope and presentation (SeAn, MPzX)Revised Figure 5 and Sec. 5Figure readability is improved; MLLM streams are explicitly listed as unevaluated.

1. Adaptation beyond frozen representations

With the identical analytic classifier, random projection and sufficient-statistics update, frozen representations achieve 71.06 macro FAA and -6.43 BWT. BiCLoRA achieves 72.21 and -7.72, improving FAA in eight of nine settings by 1.15 points on average. EBind SSW60 is the exception. Applying Eq. (11) per setting gives mean acquisition accuracies of 79.54 for BiCLoRA and 77.16 for the frozen control, separating acquisition from subsequent accuracy changes. BWT is measured from each session's own peak, which adaptation raises by 2.38 points at acquisition; the FAA lead is the net gain that remains after the subsequent decline, and since the newest session contributes only 1/T of FAA, it is carried mainly by previously learned classes.

2. Update magnitude and output directions

At the same 128 parameters, a random frozen output basis reaches 71.32/-8.96; the current-session basis adds 0.89 FAA and 1.24 BWT.

BiCLoRA leads all four D1 controls by 0.47 to 0.90 macro FAA and 0.66 to 1.34 BWT, and by 0.02 to 0.08 in audio-video cosine on ImageBind SSW60. Table 8 also tests larger BiCLoRA updates. At eight times the kernel learning rate, macro FAA/BWT decreases to 71.43/-9.09. On ImageBind SSW60, mean audio/video update norms become 1.18/1.65, versus D1's 1.08/1.54, but audio-video cosine is 0.20 versus -0.13 and joint drift is 0.055 versus 0.380. Larger updates reduce performance, yet the geometry advantage remains at comparable mean parameter-update magnitudes.

3. Historical text relationships and supervision

Across nine settings, mean audio-text/video-text cosine is 0.220/0.334 for BiCLoRA, 0.236/0.360 for frozen features and 0.140/0.228 for D1. The probes evaluate first-session held-out samples after the complete stream. On ImageBind SSW60, video-query zero-shot accuracy is 70.83, 2.61 points below frozen features but 25.00 above D1 and 17.18 above original-history InfLoRA.

Removing text from both the loss and temporary-classifier initialization gives 71.66 FAA for BiCLoRA and 68.73 for D1. Across all four supervision choices in Table 9, the output constraint adds 2.93 to 3.34 FAA, calculated from the displayed values.

4. Baseline transparency

Table 2 separates matched-history InfLoRA from the transfer using its original history rule. The latter achieves 69.75 FAA and -11.32 BWT, while BiLoRA achieves 69.78 and -13.33. BiCLoRA exceeds them by 2.46 and 2.43 macro FAA, respectively. These are the comparisons named in the revised abstract and Introduction. Appendices A to C explain the related methods and the common AVT interface, including the distinction between the LADA-inspired control and native LADA.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.