acceptodds
Under review as a conference paper at ICLR 2027

PROBABILISTIC MULTIMODAL REPRESENTATION LEARNING VIA PRODUCT-OF-EXPERTS AND QUANTUM TENSOR NETWORKS

Abstract

Multimodal representation learning must fuse heterogeneous signals while accounting for differences in modality reliability. Deterministic fusion typically does not represent this uncertainty, whereas generative fusion can be constrained by compact latent spaces and posterior collapse. We introduce QuTA (Quantum Tensor Autoencoder), which combines a Product-of-Experts posterior with a quantum Tree Tensor Network (TTN). The PoE forms a shared probabilistic latent from modality-specific Gaussian experts, and a 16-qubit TTN transforms it into nonlinear quantum features using only 75 circuit parameters. A classical skip and an objective combining task, reconstruction, and KL losses make the hybrid model trainable end-to-end. The architecture defines a deployment path from local simulator-based training to remote quantum-processor inference. We further evaluate the frozen circuit with noisy density-matrix simulation and observe stable predictive performance. Across four benchmarks, QuTA achieves the best result in the matched-protocol CH-SIMS comparison and remains competitive on MOSI, MELD, and MOSEI, while using substantially fewer trainable parameters than fine-tuned-BERT systems.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
quantum computing, machine learning, multimodal learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xfjiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose a multimodal representation learning framework combining, Product-of-Experts (PoE) and Quantum Tree Tensor Network (TTN). The method fuses text, audio, and visual representations into a shared 64-dimensional probabilistic latent space, then transforms the shared latent representation using a 16-qubit variational circuit with 75 trainable circuit parameters. The method is evaluated on three multimodal sentiment and emotion recognition benchmarks: CH-SIMS, MOSI, and MELD.

Critical strengths
  • The combination of probabilistic fusion, a quantum latent transformation, reconstruction objectives, and a classical skip connection is clearly described.
  • This work explicitly compares the TTN with an MLP, removes the skip connection, and replaces PoE with concatenation.
  • The experiments address a practically important problem in multimodal representation learning and demonstrate that the architecture can perform inference using subsets of available modalities.
Critical weaknesses
  • The main novelty is the integration of a quantum TTN into a probabilistic multimodal architecture. Based on the ablation table provided this difference is smaller than the reported standard deviation of the full model. Hence the method doesnt convincingly establish that it is meaningfully better than a comparable classical transformation

  • The authors state that their classical replacement uses a 64–1–3 MLP with 71 trainable parameters, an MLP with just one hidden neuron is extremely constrained.

  • QuTA uses nearly twice as many trainable parameters as CmIR on CH-SIMS for a relatively small improvement.

Questions for authors
  • While the paper evaluates missing-modality and input-noise robustness, these experiments are conducted only on the full QuTA architecture. Could the authors provide details on performance of PoE against concatenation-based fusion under identical missing-modality and corruption settings?

  • why do the authors think QuTA uses nearly twice as many trainable parameters as CmIR on CH-SIMS for a relatively small improvement ?

Reviewer_tx5uOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

QuTA combines Gaussian product-of-experts (PoE) fusion with a 16-qubit tree tensor network (TTN) for multimodal sentiment analysis. The reported gains are modest, and the experiments do not establish a meaningful advantage over classical alternatives.

Critical strengths
  • Clear presentation and architecture description.
  • Includes ablations, missing-modality tests, and simulated noise experiments.
  • Reports trainable parameter counts and acknowledges the use of simulation.
Critical weaknesses
  1. The paper does not establish that the TTN offers a computational advantage over classical simulation.
  2. Replacing the TTN with an MLP changes CH-SIMS Acc-3 only modestly, while removing the skip connection leaves Acc-3 unchanged. Stronger, matched classical baselines are needed.
  3. CH-SIMS uses three seeds, while MOSI and MELD use one. The Welch p-value should be omitted unless comparable seed-level results are available for both methods.
  4. The MVAE reproduction requires clarification. MVAE and CmIR should also be evaluated under the same missing-modality and input-corruption conditions.
  5. The noiseless and noisy results use different simulators. Report DM1 performance at zero noise before attributing changes to gate noise.
  6. Report results without reconstruction and KL losses, with the quantum branch alone, and without the skip connection. Clarify the scope of the MLP replacement and the contribution of each logit branch.
  7. Parameter counts alone do not establish computational efficiency. Runtime, simulation cost, and realistic hardware deployment costs are not reported.
Questions for authors

1. What is the classical simulation complexity of the TTN, and does its structure support the claimed O(n) computation?

2. How much does the quantum branch contribute to the final predictions, and what are the results when it is removed or used alone?

3. How was MVAE reproduced, and can the MOSI and MELD results be reported over at least five seeds?

4. Do MVAE and CmIR show comparable robustness under missing modalities and input corruption, and does the noise result hold with DM1 at zero noise?

Reviewer_goodOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces QuTA, a hybrid quantum-classical framework for probabilistic multimodal representation learning. It combines Product-of-Experts (PoE) fusion with a 16-qubit quantum tree tensor network (TTN) to learn compact representations from text, audio, and visual inputs. Experiments on CH-SIMS, MOSI, and MELD show competitive performance, along with additional studies on missing modalities, input noise, and simulated quantum hardware noise.

Critical strengths
  • Interesting research direction. Combining probabilistic multimodal fusion with quantum representation learning is an interesting way to explore compact, uncertainty-aware representations.

  • Compact architecture. The proposed quantum TTN uses only 75 trainable circuit parameters and provides a lightweight nonlinear transformation within the multimodal learning framework.

Critical weaknesses
  • Limited methodological novelty. The framework mainly combines existing techniques, including a Gaussian Product of Experts, a variational autoencoder, and a quantum tree tensor network. Its main contribution seems to be integrating established components rather than introducing a fundamentally new learning approach.

  • Unclear benefit of the quantum component. Table 4 shows that replacing the quantum TTN with a small classical MLP only lowers Acc-3 from 75.47% to 74.76%, and the reported standard deviations overlap. This gain is marginal, and stronger classical baselines are needed to show that the quantum computation is actually necessary.

Questions for authors
  • Limited and inconsistent performance gains. On CH-SIMS, the Acc-3 improvement over CMIR is only 0.47 percentage points. On MOSI, the method still falls behind stronger fine-tuned baselines, and on MELD its weighted F1 of 59.30 is well below COSMIC's 65.21. These results do not fully support the claimed effectiveness across different multimodal tasks.

  • No validation on real quantum hardware. The experiments rely on classical quantum simulators and report no results on actual quantum hardware. The real computational cost, measurement overhead, and inference latency are not evaluated. As a result, the practical advantage of the proposed quantum architecture remains unclear.

  • Insufficient validation of uncertainty modeling and robustness. Although the method uses probabilistic fusion, the paper does not directly evaluate the calibration or quality of the learned uncertainty estimates. The experiments on missing modalities and input noise are conducted mainly on CH-SIMS, with too few comparisons against other robust multimodal fusion methods. This limits the evidence for the claimed robustness advantage.

1 comment

Dear Authors,

Will you provide a rebuttal for your paper? I think you can try to make it better.

Reviewer

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.