- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes DepthRoute, a text-conditioned framework for adaptive visual-depth selection in multimodal sentiment analysis. It studies three routing strategies over individual layers, late layers, and contiguous layer groups, and evaluates them on MVSA-Single and TumEmo with frozen SigLIP2 and CLIP backbones. The experiments on two datasets validate the effectiveness of the proposed method.
Critical strengths
- The motivation is clear and well grounded: rather than focusing on cross-modal fusion, the paper studies how visual representations from different encoder depths should be adaptively utilized. The unified comparison of full, late, and group-wise routing makes this question easy to follow.
Critical weaknesses
-
The experimental scope is somewhat limited, as the evaluation is conducted only on MVSA-Single and TumEmo. I suggest adding experiments on MVSA-Multiple to provide stronger evidence of the method's generalizability. This is particularly important given that the reported performance on TumEmo is lower than that of CoDe [1] and MVSA-Single is lower than that of SPP=SCL [2].
-
The citation of references is problematic, for example:
"including multiview calibration Wei et al. (2023) " should be "including multiview calibration (Wei et al. ,2023) "
Questions for authors
- The current experiments do not clearly isolate the effect of adaptive routing from the group-wise representation design. Adding a fixed Group baseline with the same group construction and projections but without input-dependent routing will be helpful.
- The current routing analysis does not clearly establish whether the learned depth selection is genuinely image-dependent, since the router is also conditioned on text. Controlled analyses with fixed text and varied images, or text permutation, would provide stronger evidence for adaptive visual-depth selection.
- The experimental scope is somewhat limited, as the evaluation is conducted only on MVSA-Single and TumEmo. I suggest adding experiments on MVSA-Multiple to provide stronger evidence of the method's generalizability. This is particularly important given that the reported performance on TumEmo is lower than that of CoDe [1] and MVSA-Single is lower than that of SPP=SCL [2].
[1] WU D, YANG D, SHEN H, et al. Resolving sentiment discrepancy for multimodal sentiment detection via semantics completion and decomposition[J]. Pattern Recognition, 2026, 172: 112719.URL https://doi.org/10.1016/j.patcog.2025.112719.
[2] Wu J, Li S. SPP-SCL: Semi-Push-Pull Supervised Contrastive Learning for Image-Text Sentiment Analysis and Beyond[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2026, 40(3): 2173-2181.URL https://doi.org/10.1609/aaai.v40i3.37200.
3 comments
Q3: Siglip results for MVSA-Multiple datasets
Thank you for suggesting MVSA-Multiple. We added this dataset to evaluate adaptive visual layer selection on a third image–text sentiment benchmark. All methods use the same SigLIP visual backbone. We report Accuracy and Weighted-F1 as mean ± standard deviation over three seeds.
| Method | Accuracy | Weighted-F1 |
|---|---|---|
| Final-Layer | 0.6943 ± 0.0011 | 0.6829 ± 0.0022 |
| All-Layer Concat | 0.6972 ± 0.0041 | 0.6886 ± 0.0039 |
| DepthRoute-Layer | 0.7122 ± 0.0027 | 0.6903 ± 0.0075 |
| DepthRoute-Group | 0.7031 ± 0.0019 | 0.6916 ± 0.0010 |
| DepthRoute-Late | 0.7075 ± 0.0040 | 0.6941 ± 0.0036 |
All three DepthRoute variants achieve higher mean Accuracy and Weighted-F1 than both fixed baselines. Relative to All-Layer Concat, DepthRoute-Layer improves mean Accuracy by 1.50 percentage points, and DepthRoute-Late improves mean Weighted-F1 by 0.55 percentage points. DepthRoute-Group achieves the highest mean Weighted-F1 on MVSA-Single and TumEmo, while DepthRoute-Late achieves the highest mean on MVSA-Multiple. These results suggest that the most effective visual-depth selection strategy can vary across datasets.
I appreciate the authors' efforts in addressing my concerns. However, the results on the additional datasets presented in the rebuttal are still quite poor. Therefore, I remain highly skeptical about the effectiveness of the proposed method and do not believe the paper meets the acceptance bar for ICLR.
Thank you for recognizing the motivation for adaptive visual-depth selection and for suggesting controls to isolate the role of input-dependent routing.
1. Fixed Group baselines. We compared DepthRoute-Group with two input-independent controls that use the same depth groups, group projections, shared expert pool, and classifier. Uniform Group assigns equal weights to all group-expert pairs. Learned Static Group learns a single set of routing weights shared across examples. The table reports Weighted-F1 with SigLIP2 as mean ± standard deviation over three seeds.
DepthRoute-Group improves mean Weighted-F1 over Learned Static Group by 2.60 percentage points on MVSA-Single and 2.25 points on TumEmo. The gains over Uniform Group are 2.78 and 2.09 points, respectively.
2. Sensitivity to image and text inputs. We varied one modality at a time at inference and measured how the routing weights changed across visual depth groups. We summed the expert weights within each group and used Jensen–Shannon divergence (JSD) to compare the resulting depth-group distributions. For the three-group checkpoint trained with seed 42, the mean JSD was 0.00150 when we held the text fixed and changed the image, 0.06353 when we held the image fixed and varied the text, and 0.05546 under text permutation. Under this perturbation protocol, depth allocation showed modest sensitivity to image changes and substantially greater sensitivity to text changes.
3. MVSA-Multiple and comparison with recent methods. We agree that evaluation on MVSA-Multiple would strengthen the evidence for generalization. Our current findings are limited to MVSA-Single and TumEmo.
We acknowledge that SPP-SCL and CoDe report higher scores on MVSA-Single and TumEmo, respectively. DepthRoute examines visual encoder depth as a design choice for image–text sentiment classification. The fixed-routing comparisons above show that adapting depth weights to each image–text pair improves mean Weighted-F1 on both datasets with the frozen visual backbone and grouped architecture held fixed. We will clarify this contribution in the revision and discuss the published scores of SPP-SCL and CoDe as reference results.
4. Citation formatting. We will revise the example to “including multiview calibration (Wei et al., 2023)” and use parenthetical citations in similar contexts throughout the manuscript.