acceptodds
Withdrawn from ICLR 2027

Architecture-Dependent Fusion Pathways in MLLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal Large Language Models (MLLMs) achieve strong performance across vision-language tasks, yet the internal mechanisms by which visual and textual information are fused across layers remain insufficiently understood. We investigate representative MLLMs from two architectural paradigms: concatenation architectures and native multimodal architectures. We conduct three progressively connected analyses: alignment decoupling identifies which modality changes, attention routing and entropy characterize how cross-modal information is distributed, and intrinsic dimensionality examines how fusion reshapes feature spaces. Separately, we perform causal intervention experiments as a validation of the resulting interpretation. As a supplementary analysis, we use visual CKA to examine the Platonic Representation Hypothesis. Together, these analyses reveal two distinct fusion pathways: concatenation models follow a text-first, vision-later pathway, whereas native models exhibit earlier visual-textual co-adaptation and feature-space reorganization. This work provides a mechanistic perspective for understanding multimodal fusion and supports architecture-aware diagnostics of multimodal representations.

Primary area:
interpretability and explainable AI
Keywords:
MLLMs, fusion, mechanism

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wecaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper compares layer-wise visual-textual fusion in two MLLM families: concatenation models (LLaVA-NeXT Vicuna/Mistral, Cambrian) and native models (SOLO, EVE, SAIL). The authors analyze cross-modal and intra-modal CKA, attention routing and entropy, and intrinsic dimensionality. Two interventions (i.e., masking text tokens on POPE and injecting layer-specific noise on MMBench) further validate their findings. They conclude that concatenation models follow a text-first, vision-later pathway, whereas native models exhibit earlier cross-modal co-adaptation. Cross-model visual CKA is also examined in relation to the Platonic Representation Hypothesis.

Critical strengths
  • [S1] The research question is clear and relevant. Most work on MLLM internals focuses on concatenation architectures, and comparing native, encoder-free models against them within a single layer-wise framework is a less explored and worthwhile direction.

  • [S2] The analysis design is methodologically reasonable in principle. The paper combines representation similarity, attention statistics, and intrinsic dimensionality rather than relying on a single score, is explicit about what each measure can and cannot reveal, and attempts to move from representational observations to interventions, which is appropriate for a mechanistic claim.

  • [S3] The paper includes robustness checks that are often omitted: bootstrap resampling with standard errors and relative bias (Appendix A.6), intrinsic-dimensionality results under multiple variance thresholds (Appendix A.4), and a Mutual KNN analysis as a local-structure complement to CKA (Appendix A.1). The writing is clear, and the authors are candid about the limits of their comparison and about SAIL as an exception.

Critical weaknesses

[W1] The evidence for the central claim is confounded and lacks controls.

  • (a) Asymmetric reference representations. The conclusion that native models “change their visual representations earlier” (Figures 3 and 5) rests on comparisons against the layer-0 representation. For concatenation models, layer 0 already contains semantic features produced by CLIP (or CLIP+DINO) and the projector, whereas for encoder-free native models it is close to a raw patch embedding. Figure 5 makes this explicit: the visual intrinsic dimension starts at 6 for EVE and at 30 for Cambrian. The early change observed in native models may therefore reflect the construction, inside the LLM, of visual features that CLIP already provides, rather than a difference in fusion pathway. The paper does not rule out this simpler explanation. One way to address it would be to re-align native models to the layer at which their visual representations reach a CLIP-level semantic structure and compare from there.

  • (b) No control for the POPE masking experiment. In Table 1, masked F1 scores fall to 36.3 to 53.8%, which is at or below chance for a binary task. This suggests that the question inputs themselves are damaged. Without a control that masks the same number of randomly chosen text tokens, the observed drop cannot be attributed to the functional importance of “concentrated textual anchors” rather than to input corruption in general.

  • (c) Imbalanced perturbation ranges. In Table 2, the early range covers layers 0 to 7 (8 layers) and the late range covers layers 16 to 32 (17 layers), so the total perturbation differs by more than a factor of two. The noise magnitude and the injection protocol (independent injection at each layer, or a single injection propagated forward) are not specified. Directly comparing early and late drops under these conditions is not well founded.

  • (d) Architecture is confounded with other factors. Each family contains three models, and LLM backbone, training data, resolution, and the number of visual tokens all vary with the family. The authors acknowledge in Section 3.1 that a controlled comparison is infeasible, but in that case the causal framing of the title (“Architecture-Dependent”) and of the conclusions goes beyond what the evidence supports.

[W2] The attention analysis and the causal experiment are not distinguished from attention sinks.

In Figure 9(e), SAIL’s normalized V→T entropy is exactly 0 at every layer while its V→T routing ratio is 0.4 to 0.7. By definition, this means every visual query attends to exactly one text key, and that single token absorbs more than half of the visual attention mass. If visual tokens could not see any text token at all, the routing ratio would be 0, so this token must be a text token positioned before the image in the sequence, most plausibly BOS or a prompt token. Viewed in the same way, the V→T entropies of 0.03 to 0.05 for Cambrian and LLaVA-Mistral also indicate mass concentrated on one or two tokens. This is the characteristic signature of attention sinks, which are well documented in LLMs.

If this possibility is not excluded, the Section 3.3 finding that concatenation models “attend to a small set of textual keys” becomes a rediscovery of the sink phenomenon rather than evidence of cross-modal information flow, and the Section 3.5 POPE experiment, which masks the tokens receiving concentrated attention, becomes an experiment that removes sink tokens, which would naturally explain the collapse in performance. I ask the authors to report (i) which text tokens, by sequence position and content, actually receive the visual attention mass, (ii) whether BOS is among the special tokens masked according to Section 3.1, (iii) whether system-prompt or question-template tokens are counted as “textual tokens,” and (iv) the identity of the anchor tokens masked in Table 1. Two controls would resolve this concern: masking only BOS, and masking randomly selected text tokens.

[W3] Experimental details needed to interpret the results are missing.

The following are not specified anywhere in the main text or appendix, which makes the results difficult to reproduce or interpret. (i) The input template and token order (whether the image precedes the text). Under causal attention this determines which text tokens the visual tokens can see and is directly tied to W2. (ii) Whether the cross-modal CKA in Figure 1 uses the same sample-level mean pooling described in Section 3.2, and whether it uses 10,000 or 5,000 samples. (iii) Whether the matrix used for intrinsic dimensionality is token-level or sample-level pooled. If token-level, the differing numbers of visual tokens across models affect the estimates. (iv) The noise variance and injection protocol for Table 2. (v) What exactly the layer-0 visual representation of native models is in Section 3.2 (raw patch embedding, or the output of a first projection).

[W4] Sample-level and model-level uncertainty are not shown in the main figures.

Figures 1 and 3 report point estimates per layer with no indication of variability. Two kinds of uncertainty are needed.

  • (a) Sample-level. CKA yields a single value per layer over the whole sample set, so sample-level uncertainty must come from resampling. The authors already perform bootstrap resampling in Appendix A.6, but report it only for Cambrian at layers 1, 15, and 30 (Table 3), and it is not reflected in the main figures. Given that the appendix itself reports a relative bias of 12.6% at n=500, I ask the authors to extend the bootstrap to all layers and all models and to draw confidence bands around each curve. This is particularly important for judging whether the transition-range dip in Figure 1 (roughly 0.47 to 0.37) exceeds resampling variability.

  • (b) Model-level. The family averages are arithmetic means of three models, and the individual model curves are shown only as faint lines whose colors are hard to tell apart. In Figure 1, the early peak near layer 3 (about 0.67) and the sharp drop at the final layer (about 0.42) each appear to be produced by a single model, so it is not possible to tell whether the shape of the family average reflects a family-wide property or a single model. I ask the authors to add a band showing the across-model standard deviation or min-max range around each family average, and to state in the text at which layers the two families’ bands overlap and at which they separate. With only three models per family these bands are likely to be wide, and if so the strength of the “family-level difference” claim should be adjusted accordingly.

Figure 4 already reports medians with IQR and meets this standard, so applying the same treatment to Figures 1, 3, and 5 should be straightforward. Separately, the introduction states that CKA is robust to “common linear transformations.” Linear CKA is invariant only to orthogonal transformations and isotropic scaling, and this statement should be corrected.

[W5] Scope of the contribution and unvalidated practical implications.

  • (a) It is unclear whether the three analyses function as independent tests. When routing ratios fail to separate the families (Δr is 0.3 to 0.6 in both), the comparison shifts to entropy. SAIL is excluded from the attention analysis and treated as a “structural exception.” Intrinsic dimensionality is shown in the main text for only one pair of models. Because results that do not fit the narrative are handled as exceptions, it is not clear what outcome would have falsified the proposed two-pathway hypothesis.

  • (b) The core conclusion, namely that concatenation models preserve the visual encoder’s features in early layers while native models construct visual features from scratch, follows to a large extent from the definitions of the two architectures. The paper should state more clearly what it finds beyond this expectation.

  • (c) Section 5 proposes concrete practical implications, including transition-band layer-selective LoRA, anchor regularization, and earlier cross-modal feedback, but none is tested. Since the per-layer intervention infrastructure of Section 3.5 is already in place, an experiment comparing LoRA applied only to the transition band against uniform application across all layers would not have been difficult, and it would have directly demonstrated the practical value of the analysis.

  • (d) The Platonic Representation Hypothesis analysis in Section 4 is only loosely connected to the two-pathway claim. Moreover, most evaluated models share Vicuna, Mistral, or Llama backbones, so deep-layer convergence cannot be distinguished from shared structure in the LLM residual stream.

Due to the aforementioned weaknesses, my initial score is 1. However, I am willing to raise it to 2 if the authors can convincingly address at least three of these weaknesses.

Questions for authors
  • How exactly is the cross-modal CKA in Figure 1 computed? The sample-level mean pooling is described only in Section 3.2 for the intra-modality CKA, and it is not stated whether the same procedure applies to Figure 1. Please specify the two matrices being compared at each layer, including their dimensions and the number of samples used.
  • Is there a formal definition of the transition layers? The ranges early (0 to 7), transition (8 to 15), and late (16 to 32) appear only as labels in Figure 1, and the caption’s reference to Section 3.1 concerns the model classification rather than the layer ranges. Please state the criterion used to define these ranges and whether they were fixed a priori or chosen after observing Figure 1.
Reviewer_177iOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to study feature similarity interaction patterns between text and image tokens in transformer-based multimodal LLMs across the depth of the network layers. In particular it considers how these vary across encoder-free multi-modal architectures and concatenation-based approaches. The study relies on a numner of representative models from both architecture families. It fowards 10k image-caption pairs through these models to study the modality interaction patterns. It compares the interaction patterns within each of the different models under consideration. Additionally, it considers the similarities of activations between different models across layer depth.

Critical strengths
  • Rather than analyzing fusion patterns in a single model, this study investigates fusion patterns across multiple architectures, including both concatenation-based models as well as encoder-free models.
Critical weaknesses
  • It is not clear which problem this paper is trying to solve, and whether it succeeded, or how we could measure if it did. The paper presents a number of analyses of attention patterns in MLLMs, but does not deliver clear actionable insights, and does not demonstrate how the analyses be leveraged to e.g. improve models or unlock other impacts. The conclusion gives some ideas, but these are not delivered upon in the current paper, significantly limiting its impact.

  • As also indicated by the authors in section 3.1, the comparisons between models and Paradigms are not fully controlled. It is unclear to what extent the results are determined by changes in training data, compute, architectures, and vision encoder pre-taining strategies.

  • The introduction fails to clearly (even if briefly) define the central concept of (CKA) layer similarity. It is not clear what is compared with what by the similarity: individual token features, aggregated features,…. Nor how the similarity is computed: a distance metric, distributional metic, … Only in section 3.2 this is made explicit, while a brief non-technical description early on would ground the discussion in the preceding sections.

  • In section 3.2 it is not clear why representations are averaged across the tokens in a particular layer. Moreover, why is averaging a good way for pooling? It only retains the first order moments of features.

  • It is not clear what the authors mean with “stacked visual representations” in section 4.

Questions for authors
  • Please add clear definitions of what the features Z and H represent, and how they are related.
Reviewer_1rQLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how visual and textual information interact across layers in multimodal large language models (MLLMs). It compares two broad architectural paradigms: models that concatenate visual and textual tokens and native multimodal models. The analysis combines layer-wise representation alignment, attention routing and entropy, intrinsic dimensionality, and causal interventions. The central claim is that the two paradigms exhibit different fusion pathways: concatenation-based models tend to follow a text-first, vision-later trajectory, whereas native models show earlier visual-textual co-adaptation and feature-space reorganization. A supplementary visual CKA analysis also examines whether deeper representations become more similar across models. Overall, the paper addresses an important interpretability question and offers a useful multi-view empirical framework, but the evidence does not yet fully isolate architecture as the cause of the reported differences.

Critical strengths

Strengths

  1. The paper asks a timely and meaningful question about the internal mechanisms of multimodal fusion rather than focusing only on end-task performance.

  2. Comparing concatenation-based and native multimodal architectures is valuable, and the proposed distinction between text-first/vision-later fusion and early co-adaptation is intuitive and potentially useful for future model analysis.

  3. The study combines several complementary measurements—representation similarity, attention statistics, intrinsic dimensionality, and intervention-based validation—instead of relying on a single diagnostic.

  4. The layer-wise analysis provides a clear organizing perspective and helps connect observed representational changes with architecture-specific processing patterns.

  5. The paper is generally well motivated, and the figures and narrative make the main empirical pattern reasonably easy to follow.

Critical weaknesses
  1. Architecture is not cleanly separated from training data, objectives, visual encoders, parameterization, and other model-specific choices. The results therefore support an architecture-associated difference more strongly than an architecture-caused difference.

  2. The evaluated model set is still limited, and variation within each architectural family may weaken a simple two-pathway taxonomy. Broader coverage or controlled model pairs would make the generalization more convincing.

  3. CKA, attention mass, attention entropy, and intrinsic dimensionality are informative but indirect proxies for fusion. Their precise relationships to multimodal information integration and task-relevant computation require stronger justification.

  4. The causal interventions are promising, but their scope appears narrower than the descriptive analysis. More intervention types, tasks, layers, and robustness checks would better establish the proposed mechanisms.

  5. The paper would benefit from clearer statistical reporting and sensitivity analyses, including uncertainty across samples, prompt or dataset effects, metric hyperparameters, and the stability of qualitative conclusions.

Questions for authors
  1. Can the authors more directly disentangle architectural effects from differences in pretraining, visual backbones, alignment objectives, and model scale? Even a carefully matched subset or explicit confound analysis would help.

  2. How stable are the identified transition layers and pathway labels across datasets, prompt formats, sequence lengths, image complexity, and different intrinsic-dimension thresholds?

  3. Do the representation and attention diagnostics predict model behavior at the individual-example level, or are the reported relationships mainly aggregate correlations?

  4. Can the authors explain how heterogeneous cases within the native family fit the proposed taxonomy and specify what empirical result would falsify the two-pathway interpretation?

  5. How sensitive are the intervention results to the intervention magnitude and selection rule, and are the observed performance changes specific to multimodal fusion rather than general layer sensitivity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.