acceptodds
Under review as a conference paper at ICLR 2027

Beyond One-Geometry-Fits-All: State-Conditioned Dual-Path Reconstruction for Missing-Modality PEFT

Abstract

Parameter-efficient fine-tuning (PEFT) for multimodal missing-modality adaptation typically reuses a single low-rank adaptation structure across all availability states. However, independently optimized adapters exhibit systematically larger cross-state than same-state, cross-seed subspace separation. Assigning independent adapters per state enhances state-specific expressiveness but scales linearly with the number of states and may weaken cross-state knowledge transfer through parameter isolation; a fixed shared structure is more efficient, yet its limited rank budget cannot simultaneously cover all state-preferred update directions. We formalize this Capacity-Transfer Tension geometrically, establishing a constructive representational separation between CODAR and fixed shared rank-r geometry via subspace coverage and reconstruction analyses. To address it, we propose Conditional Dual-Path Adapter Reconstruction (CODAR), which reconstructs both low-rank factors from reusable dictionaries while conditioning their composition on modality availability. A modality-specific pathway captures state-dependent variation, whereas a transferable pathway reuses shared input-side directions across towers. Across MM-IMDb, UPMC-Food101, and Hateful Memes, CODAR consistently improves over representative PEFT baselines with 101,116 trainable adapter parameters. Controlled comparisons and component ablations further show that the gains cannot be explained by availability conditioning or parameter count alone, while geometric analyses show that CODAR realizes state-dependent update subspaces while retaining shared primitives. Code is available at https://anonymous.4open.science/r/anonymous_code_-48D6/.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Parameter-Efficient Fine-Tuning, Missing-Modality Learning, Multimodal Learning, State-Conditioned Adaptation, Low-Rank Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_UoFyOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Conditional Dual-Path Adapter Reconstruction (CODAR) for parameter-efficient adaptation of image-text models under known modality-availability states. It reconstructs both low-rank factors from reusable dictionaries through state-conditioned routing, combining modality-specific dictionaries with a cross-tower shared input-side dictionary. A CP-based generator amortizes routing parameters across modules. The paper analyzes cross-state update geometry, derives a reconstruction lower bound for fixed shared rank-r geometry, and establishes a constructive representational separation for CODAR. Evaluation on three classification benchmarks with CLIP ViT-B/16 reports consistent improvements over reproduced PEFT baselines and a parameter-matched conditional control, using 101,116 trainable adapter parameters. Ablations, geometric diagnostics, and cross-scenario experiments provide additional validation.

Critical strengths

A focused problem formulation supported by geometric evidence. The paper asks whether modality availability should change the row and column subspaces of a PEFT update. Cross-state versus cross-seed comparisons and energy-weighted spectral deficits provide an informative motivation. The constructive theorem connects this motivation to the proposed parameterization while explicitly limiting its scope to representation. This is a useful contribution to understanding missing-modality adaptation.

Critical weaknesses
  1. Limited methodological novelty and significance. The methodological contribution appears incremental. CODAR combines state conditioning, dictionary-based low-rank factor reconstruction, and modality-specific/shared pathways, building closely on existing PEFT mechanisms such as VB-LoRA and MoRA. The availability-dependent geometric perspective is useful, but the theoretical separation is established against fixed shared geometry and does not distinguish CODAR from other conditional low-rank parameterizations. Although the controlled experiments report consistent improvements, the gains over the parameter-matched conditional baseline are relatively modest, and the main evaluation remains concentrated on one CLIP backbone and three classification benchmarks. Consequently, the current evidence provides limited support for the broader significance of this particular architectural combination. 
  2. The supervision protocol limits the interpretation of training-time missing-modality robustness. Algorithm A2 computes complete-input teacher predictions for examples assigned a missing state. Thus, training retains access to the modality hidden from the student. This is a valid possible protocol for artificial modality dropout, but its applicability to genuinely incomplete training data requires qualification. StateCondLoRA receives the same distillation objective, and Table A11 includes a no-distillation ablation; these are useful controls. However, equivalent training-information access for the other principal baselines is not explicit, leaving uncertainty about the scope and interpretation of some comparisons. The discrepancy between minibatch state sampling in Algorithm A2 and fixed masks in Appendix C further needs resolution.
Questions for authors
  1. Clarification of the principal baseline: Please provide the exact StateCondLoRA factor-generation equations, output dimensions, parameter-count breakdown, and initialization. Does a nonzero base factor or offset prevent both A and B from being zero? Please verify that the adapter receives a nonzero learning signal at the start of training. This answer directly affects my confidence in the parameter-matched comparison. 
  2. Assessment-critical supervision comparison: Given the complete-input teacher access in Algorithm A2, please state which reproduced baselines receive equivalent information and objectives. Building on the existing no-distillation ablation, can you report a matched comparison of CODAR, StateCondLoRA, and MoRA in which originally hidden modalities are unavailable to all methods during training? Alternatively, please explicitly restrict the relevant claims to training with recoverable complete pairs. This would clarify both the applicability and attribution of the reported gains. 
  3. Clarification of state assignment: Appendix C states that missing masks remain fixed across epochs, whereas Algorithm A2 samples a state for each example within each minibatch. Which protocol produced the reported results? Please specify the protocol for the main comparisons and the StateSpecificLoRA diagnostic, since it determines state-specific data exposure.
Reviewer_43YbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CODAR for parameter-efficient adaptation under missing modalities. It reconstructs both low-rank factors using availability-conditioned routing over modality-specific and shared dictionaries. The authors provide a representational analysis and report improvements over PEFT baselines on different benchmarks using CLIP ViT-B/16.

Critical strengths

1.CODAR conditions both low-rank factors on modality availability, allowing both row and column subspaces to vary across states.

2.The dual-path design combines modality-specific dictionaries with shared input-side primitives, enabling state-dependent adaptation without separate adapter copies for each state.

Critical weaknesses
  1. In Table 1, CODAR’s standard deviations on Hateful Memes are 0.49, 0.47, and 0.52 across the three missing-modality settings. Mathematically, the standard deviation of the average score cannot exceed the largest component standard deviation, 0.52. However, the reported standard deviation is 0.53, violating this mathematical constraint and raising concerns about the reliability of the reported experimental results.

  2. The evaluation is restricted to CLIP ViT-B/16, leaving CODAR’s generalizability and scalability to other architectures unclear.

Questions for authors

1.Could the authors provide the individual scores for each random seed under all three missing-modality settings for CODAR on Hateful Memes in Table 1? Please explain how the reported standard deviation of 0.53 was obtained.

2.Could the authors provide results on additional backbone models, particularly larger models, to assess CODAR’s generalizability and scalability?

Reviewer_mwy6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CODAR for missing-modality PEFT, reconstructing both low-rank factors through availability-conditioned routing over shared and modality-specific dictionaries. It provides an expressivity separation from fixed rank-r geometry. Experiments on three CLIP-based benchmarks report consistent gains, supported by ablations and geometric analyses.

Critical strengths
  • The geometric analysis provides useful evidence that update subspaces vary across availability states.

  • Reported improvements are consistent across three datasets, with matched seeds, missing masks, and extensive ablations.

Critical weaknesses
  1. Baseline implementations require clarification. StateCondLoRA’s initialization may yield zero gradients if both generated factors start at zero. VB-LoRA’s stated configuration does not reconcile with its reported parameter count, and the described Uni-LoRA implementation differs from the cited method.

  2. Architectural gains are insufficiently isolated. CODAR includes auxiliary mechanisms absent from StateCondLoRA. Removing routing consistency reduces its remaining advantage to only 0.21/0.21/0.69 points, without corresponding uncertainty estimates.

Questions for authors
  • Please clarify StateCondLoRA’s factor parameterization and initialization, and reconcile the VB-LoRA and Uni-LoRA implementation discrepancies.

  • Does CODAR retain a statistically reliable advantage when applicable auxiliary mechanisms are matched? Please provide paired differences and confidence intervals.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.