acceptodds
Under review as a conference paper at ICLR 2027

Understanding and Mitigating Bias in MLLM-based Recommender Systems

Abstract

Multimodal large language models (MLLMs) are increasingly adopted as backbones for recommender systems, giving rise to MLLM-based recommender systems (MLLM-RSs). However, bias issues in these systems remain largely underexplored, undermining stakeholders' trust and interests. Two key challenges to solve: How to (1) identify the source of residing bias and (2) mitigate bias without costly retraining. We propose TRACE (Tracing and Removing bias via cAusal Concept Effects), a novel causally grounded framework to identify and mitigate bias in MLLM-RSs. First, it traces the propagation of bias through three connected levels: from the training sample level, to the modality-specific concept level that captures higher-level factors potentially driving bias, and finally to the recommendation outcome level that models how these concepts causally affect bias. We devise a causally-mediated chain influence to connect the three levels, attributing measured bias effects in recommendation outcomes to individual training samples. Second, the resulting chain influence directly guides three efficient bias mitigation operators: a removal operator suppresses high-influence bias-driving samples along identified bias directions; a corrective operator reinforces genuine preference signals from corrective pairs within the same directions; and a composed operator integrates both to balance bias removal and preference preservation during parameter updates. Experiments on three datasets and two MLLM backbones under different debiasing cases demonstrate that our proposed methods achieve a superior balance between recommendation fairness and accuracy.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Recommender system, multimodal large language model, fairness

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Af63OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TRACE, a post-hoc framework for identifying and mitigating bias in recommenders built by LoRA fine-tuning a multimodal LLM. Identification proceeds in three levels: modality-specific hidden-state views are encoded by an "identifiability-oriented" sparse autoencoder whose prior is conditioned on a bias-relevant auxiliary variable; a linear SCM with cross-modal bilinear terms is fitted over selected concepts and a bias outcome to obtain per-concept effects; and influence functions link training samples to concept activations, giving a chain influence equal to the τ-weighted sum of sample-to-concept influences. Mitigation applies a one-step, closed-form update on the LoRA adapter projected onto the span of the τ-weighted concept gradients: a removal operator unlearns the top-|chain-influence| samples, a corrective operator pushes up preferred-but-less-popular items in fixed-norm steps, and a composed operator combines both. Experiments on MicroLens, Amazon Baby, and MovieLens-1M with Qwen2.5-VL-3B cover popularity, gender/age/occupation, co-existing, and modality-dominance bias.

Critical strengths
  1. Relevant problem and general formulation. Bias in MLLM-based recommenders is underexplored. The paper defines bias through a generic variable ρ\rho and an aggregation Φρ\Phi_\rho, which lets one framework cover popularity, user-attribute, co-existing and modality-dominance bias.
  2. Identification and mitigation are connected, without retraining. The chain influence links training samples, multimodal concepts and the bias outcome, with an invariance property under concept rescaling and permutation. The removal and corrective operators reuse the identified subspace and come with first-order guarantees. The edit is applied directly to the LoRA adapter and takes about 4–6 h, compared with about 97 h for retraining-based methods such as RW on MicroLens.
  3. Broad evaluation and interpretable analysis. Experiments cover three datasets, two backbones, a wide range of baselines, ablations and sensitivity studies. The concept analysis shows semantically coherent concepts whose activations vary systematically with popularity, together with interpretable cross-modal pathways in the learned SCM.
Critical weaknesses
  1. The identification claim at the sample level is not validated. The central contribution is that IchainI_{chain} attributes measured bias to individual training samples. Influence estimates for fine-tuned large models with a damped diagonal Fisher are known to be unreliable (Appendix B acknowledges this), yet the paper never checks whether the predicted change in JcenJ_{cen} or in BB from removing high-∣Ichain∣|I_{chain}| samples matches the actual change. Without such checks, the improvements in Section 5 could come from the projected update direction rather than from accurate sample attribution. Relatedly, DbiasD_{bias} is selected by ∣Ichain∣|I_{chain}|, which includes samples whose upweighting decreases bias; unlearning those should increase bias. The paper should explain why the absolute value, rather than a signed criterion, is used.
  2. Ranking uses one positive against 20 sampled negatives. TailP@K over sampled candidates depends on the popularity distribution of the negative sampler, which is not described; Figure 1(a) suggests candidates are themselves heavily tail-weighted. FpopF_{pop} is the harmonic mean of NDCG (around 50 to 60) and TailP (around 10 to 20), so it is dominated by TailP.
Questions for authors

In addition to the weaknesses, I have one further question for the authors: TRACE-R reports identical accuracy results, standard deviations included, across different bias settings. On MovieLens, popularity debiasing (Table 1) and gender debiasing (Table 2) both give AUC 84.08 ± 0.17, NDCG@3 50.28 ± 0.73 and NDCG@5 55.75 ± 1.29. On MicroLens, popularity debiasing (Table 1) and modality-dominance debiasing (Table 8) both give AUC 84.66 ± 0.08, NDCG@3 49.49 ± 0.13 and NDCG@5 55.17 ± 0.30. Since the concepts, SCM, Dbias{D}_{bias} and PbiasP_{bias} all depend on the bias variable, the updates should differ. Were these separate runs? Please explain.

5 comments
AuthorsOpenReview ↗

Response to Reviewer Af63 - W1

We thank the reviewer for the careful assessment. We directly address the concerns regarding sample-level attribution, fairness evaluation, and reporting consistency.

W1.

The identification claim at the sample level is not validated. The central contribution is that $I_{chain}$ attributes measured bias to individual training samples. Influence estimates for fine-tuned large models with a damped diagonal Fisher are known to be unreliable (Appendix B acknowledges this), yet the paper never checks whether the predicted change in $J_{cen}$ or in $B$ from removing high-$|I_{chain}|$ samples matches the actual change. Without such checks, the improvements in Section 5 could come from the projected update direction rather than from accurate sample attribution. Relatedly, $D_{bias}$ is selected by $|I_{chain}|$, which includes samples whose upweighting decreases bias; unlearning those should increase bias. The paper should explain why the absolute value, rather than a signed criterion, is used.

Response:

Thank you for the valuable suggestions. We provide direct validation of sample-level attribution and clarify the rationale for magnitude-based selection.

(1) Does the identification predict actual bias change? We conduct a controlled leave-group-out test on MovieLens. We select the top/bottom 128 and 256 training samples by signed IchainI_{\mathrm{chain}}. Starting from the same fine-tuned adapter, we perform one additional epoch of warm-start training, with and without each selected group, and measure the resulting difference in Jcen\mathcal{J}_{\mathrm{cen}}. This test is motivated by the warm-start influence analysis of Bae et al. (2022). Table R8 shows that the predicted and observed changes agree in sign for all four groups. Removing positive-influence samples decreases Jcen\mathcal{J}_{\mathrm{cen}}, whereas removing negative-influence samples increases it. This directly supports the directional validity of sample attribution. As expected for local influence approximations, the magnitudes are less precisely matched.

(2) Why the absolute value? Magnitude ranking was chosen to capture the samples that most strongly engage the bias-relevant concepts. Then, TRACE-R applies an aggregated, bias-subspace-projected update rather than deletion of each sample, and the resulting effect depends on the combined influences. To evaluate this choice, we compare absolute selection with signed selection (Table R9). The absolute variant yields higher TailP@3 and NPCG@3, while the signed variant achieves higher AUC and NDCG@3. This shows a fairness-utility trade-off rather than a consistent advantage of either criterion.

Thank you for your valuable comments. We have clarified the signed interpretation of IchainI_{\mathrm{chain}}, the selection rationale, and added the new validation protocol in Section 4.4 and Appendix E.10.6.

Table R8: Predicted vs. observed change of Jcen\mathcal{J}_{cen} after removing sample groups on MovieLens.

MetricTop-128Top-256Bottom-128Bottom-256
Predicted ΔJ\Delta \mathcal{J}-22.45-22.83+13.50+13.72
Observed ΔJ\Delta \mathcal{J}-7.06-7.62+6.89+8.06

Table R9: Comparison of absolute and signed TRACE-R (popularity debiasing on MovieLens).

VariantAUC ↑NDCG@3 ↑TailP@3 ↑NPCG@3 ↑FpopF_{pop}@3 ↑
Absolute84.8953.7914.0740.5122.30
Signed85.0954.6813.6040.4821.78
AuthorsOpenReview ↗

Response to Reviewer Af63 - W2, Q1

W2.

Ranking uses one positive against 20 sampled negatives. TailP@K over sampled candidates depends on the popularity distribution of the negative sampler, which is not described; Figure 1(a) suggests candidates are themselves heavily tail-weighted. $F_{pop}$ is the harmonic mean of NDCG (around 50 to 60) and TailP (around 10 to 20), so it is dominated by TailP.

Response:

We clarify the negative-sampling protocol and the interpretation of our fairness metrics.

Consistent candidate sampling. As described in E.1.1, we follow the commonly used protocol in MLLM-RS (Ye et al., 2025; Wang et al., 2026) to sample negative candidates. Specifically, for each test positive, 20 negatives are drawn uniformly at random with a fixed seed. All methods are evaluated on the same candidate sets for a fair and controlled comparison. On MicroLens, Baby, and MovieLens, tail-tier (T1) and popular-tier (T5) items account for 35.51% and 18.47%, 32.99% and 21.24%, and 48.02% and 15.62% of the sampled candidates, respectively. We have explicitly reported the sampling procedure and candidate distributions in Appendix E.1.1.

Fairness is not assessed using FpopF_{\mathrm{pop}} alone. We agree that, because TailP is smaller than NDCG numerically, the harmonic mean Fpop is more sensitive to changes in TailP. Therefore, we report NDCG, TailP, and NPCG separately, rather than relying on FpopF_{\mathrm{pop}} alone. Specifically, NPCG measures ranking accuracy for non-popular target items, distinguishing preference-consistent debiasing from blind tail promotion. For example, on MicroLens, TRACE-RC improves both TailP@3 and NPCG@3, while maintaining NDCG@3. Similar joint improvements in TailP and NPCG are observed on Baby and MovieLens (Table 1).

Q1.

In addition to the weaknesses, I have one further question for the authors: TRACE-R reports identical accuracy results, standard deviations included, across different bias settings. On MovieLens, popularity debiasing (Table 1) and gender debiasing (Table 2) both give AUC 84.08 ± 0.17, NDCG@3 50.28 ± 0.73 and NDCG@5 55.75 ± 1.29. On MicroLens, popularity debiasing (Table 1) and modality-dominance debiasing (Table 8) both give AUC 84.66 ± 0.08, NDCG@3 49.49 ± 0.13 and NDCG@5 55.17 ± 0.30. Since the concepts, SCM, $D_{bias}$ and $P_{bias}$ all depend on the bias variable, the updates should differ. Were these separate runs? Please explain.

Response:

Thank you for pointing this out.

Source of the results. Yes, TRACE-R was independently instantiated for each bias setting, with bias-specific concepts, SCM, sample selection, and projection subspace. However, the evaluation script mistakenly used checkpoints from another bias setting on the same dataset: MovieLens popularity reused gender-derived results, and MicroLens modality debiasing reused popularity-derived results. We traced the affected rows to their respective checkpoints and reported the corrected results in Tables R10 and R11, and corrected the rows in Tables 1 and 8.

Observations. On MovieLens popularity debiasing, the corrected TRACE-R improves NDCG@3 from 50.28 to 53.79 and NPCG@3 from 37.47 to 40.51, with a modestly lower TailP@3 (14.40 to 14.07). On MicroLens modality debiasing, NDCG@3 increases from 49.49 to 54.27 and WI-N@3 from 42.33 to 48.40. The corrected results substantially reduce the previously reported accuracy degradation.

Verification and revision. We additionally audited the results of TRACE-R, TRACE-C, and TRACE-RC against their respective evaluation records and confirmed that the remaining reported rows are correct. We have corrected the affected tables and corresponding discussion. The results and conclusions for the full TRACE-RC framework remain unchanged.

Table R10: Popularity debias of TRACE-R on MovieLens.

MethodAUC ↑NDCG@3 ↑NDCG@5 ↑TailP@3 ↑TailP@5 ↑NPCG@3 ↑NPCG@5 ↑FpopF_{pop}@3 ↑FpopF_{pop}@5 ↑
Submitted84.0850.2855.7514.4018.0237.4743.2022.3927.24
Corrected84.8953.7958.9614.0716.9240.5145.9822.3026.29

Table R11: Modality debias of TRACE-R on MicroLens.

MethodAUC ↑NDCG@3 ↑NDCG@5 ↑WI-N@3 ↑WI-N@5 ↑FmodF_{mod}@3 ↑FmodF_{mod}@5 ↑
Submitted84.6649.4955.1742.3349.3245.6352.08
Corrected86.4254.2759.5548.4054.3651.1656.83

References

Juhan Bae, Nathan Ng, et al. If influence functions are the answer, then what is the question? NeurIPS, 2022.

Yu Wang, Yonghui Yang, et al. Multimodal large language models with adaptive preference optimization for sequential recommendation. SIGIR, pp. 1903–1913, 2026.

Yuyang Ye, Zhi Zheng, et al. Harnessing multimodal large language models for multimodal sequential recommendation. AAAI, pp. 13069–13077, 2025.

Dear Reviewer,

Thank you again for your detailed and constructive feedback. In response to your comments, we have provided additional validation of sample-level attribution, further analysis of influence-based sample selection, and clarification of the evaluation protocol. We have also corrected the inconsistencies and verified the remaining results in the revised manuscript.

We sincerely hope these clarifications and additional results are helpful, and we would be glad to answer any remaining questions or provide further clarification.

Thank you again for your valuable suggestions and consideration.

Best regards,

Authors

Reviewer_xvhwOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript examines the long-overlooked issue of bias in MLLM-RS and proposes a causal analysis framework called TRACE. It makes three key contributions: First, it introduces the “Causal Mediation Chain Effect” method, which decomposes bias tracing into three levels; second, it transforms the identified bias pathways into a one-step closed-form parameter update for the LoRA adapter without the need for retraining, thereby eliminating bias while preserving recommendation personalization; third, the framework is bias-type-agnostic, requiring only a single measurable bias-related variable, and can jointly address multiple coexisting biases.

Critical strengths
  1. By incorporating an iVAE-style prior conditioned on bias-related auxiliary variables into a sparse autoencoder, this approach addresses the issues of unidentifiable latent variables and semantic instability in standard SAEs.

  2. Simply erasing the bias direction would also remove the associated true preference signals; therefore, a correction operator was designed.

  3. All updates are closed-form, one-step operations that act solely on the LoRA adapter, eliminating the need to retrain the large model backbone.

Critical weaknesses
  1. SCM's assumptions are too strong and it relies heavily on heuristics.

  2. MovieLens does not contain images itself; it relies on movie posters to construct a visual modality, and visual signals have very weak relevance to recommendations.

3.The baseline was adapted from a traditional RS or LLM-RS design. Whether this adaptation was sufficiently optimized cannot be determined.

Questions for authors
  1. How stable is the concept-bias subspace across seeds? The paper acknowledges in Remark 1 that the formal distinguishability condition is not satisfied, and the entire mitigation scheme relies on the identified subspace P_bias.

  2. What modalities primarily drive the bias pathways in MovieLens and Baby?

3.Is it necessary to include sensitive attributes during deployment for attribute depolarization?

6 comments
AuthorsOpenReview ↗

Response to Reviewer xvhw - W1

We thank the reviewer for the detailed review. We clarify the scope of the causal assumptions, the role of multimodal information, and the fairness of our baseline adaptation. We also outline additional analyses directly addressing the questions about stability and modality contributions.

W1.

SCM's assumptions are too strong and it relies heavily on heuristics.

Response:

Thank you. We clarify that TRACE adopts a common and tractable causal approximation rather than assuming exact recovery of the MLLM's mechanisms. Its causal pathways and effects are estimated from data rather than heuristic attribution, and we further validate the impact of modelling choices through ablations and new robustness experiments.

Assumption 1 is a common choice. Assumption 1 builds on the well-established linear acyclic SCM with independent noise, as commonly studied in causal discovery (Shimizu et al., 2006; Peters et al., 2014; Zheng et al., 2018). We extend its bias-outcome equation with bilinear terms to capture cross-modal interactions. This provides a tractable approximation rather than assuming arbitrary nonlinear MLLM mechanisms can be exactly recovered.

Data-driven rather than heuristic attribution. Discovering a DAG over all sparse concepts is computationally expensive and prone to spurious dependencies. We therefore use ridge-based screening to retain bias-relevant concepts before discovery, following the principle of high-dimensional feature screening (Fan & Lv, 2008). Importantly, ridge is used only for screening, not for determining causal dependencies. Sensitivity to the number of retained concepts K′K' is evaluated in Figs. 4(c) and 5(c).

Evidence for the effectiveness of SCM. Replacing the SCM with ridge weights ("w/o SCM") degrades NDCG@3 (53.17 to 51.55) and NPCG@3 (41.15 to 38.47) on MovieLens popularity bias (Table 3). On gender bias (Table 11), NDCG@3 drops from 53.47 to 48.55 and FattrF_{\mathrm{attr}}@3 from 68.25 to 62.52. This supports explicitly modelling concept dependencies rather than relying solely on bias correlations, which can conflate bias-related and useful preference signals.

New robustness evidence. We further vary the discovery algorithm, GES (Chickering, 2002), DirectLiNGAM (Shimizu et al., 2011), model specification (nonlinear SEM), and random seed (rerun). As shown in Table R4, the resulting causal-effect rankings retain Spearman correlations of 0.7044-0.8373 with our original estimates, while salient-concept overlap remains 0.7500-0.8750. These results indicate that the identified major bias pathways are reasonably consistent across alternative discovery methods and functional forms. These results are included in Appendix E.10.2.

Table R4: Robustness of the estimated causal effects (popularity debiasing on MovieLens).

MetricsRerunGESDirectLiNGAMNonlinear SEM
Spearman of τ\tau ↑0.83730.72210.79060.7044
Concept overlap ↑0.87500.75000.87500.8333
AuthorsOpenReview ↗

Response to Reviewer xvhw - W2&Q2, W3

W2 & Q2.

MovieLens does not contain images itself; it relies on movie posters to construct a visual modality, and visual signals have very weak relevance to recommendations.
What modalities primarily drive the bias pathways in MovieLens and Baby?

Response:

Thank you. We clarify the MovieLens visual setting and provide a new modality-level analysis of the identified bias pathways.

Dataset choice. We follow the established use of visual side information in multimodal recommendation (Wang et al., 2026; Chen et al., 2025; Lu et al., 2024) to augment MovieLens with movie posters, as posters can provide complementary visual cues (e.g., genre and style). We additionally evaluate MicroLens and Amazon Baby with native visual content. All compared methods use the same multimodal inputs for a fair comparison.

Modality-specific bias contributions. TRACE does not assume equal modality importance. Instead, it adaptively estimates concept-to-bias effects, including direct and mediated pathways. A modality with little contribution to bias receives a small τ\tau. On MovieLens, TRACE-RC still has the best NPCG and FpopF_{pop} (Table 1) and the best FattrF_{attr} (Table 2).

We further quantify the share of causal effect originating in each modality mm. (Table R5). On MovieLens and Baby, text dominates the identified popularity-bias pathways (53.7% and 70.4%), while images also contribute clearly (36.1% and 26.1%). MicroLens exhibits a more distributed pattern across item and preference views. These findings demonstrate dataset-dependent, multimodal bias pathways, supporting TRACE's adaptive identification rather than assuming fixed modality contributions. These results and analysis are included in Appendix E.10.4.

Table R5: Magnitude of causal effect from each modality.

DatasetTextImageInvPrefTmpPref
MovieLens0.53670.36050.05180.0510
Amazon Baby0.70390.26100.02370.0114
MicroLens0.35770.27470.23660.1311

W3.

The baseline was adapted from a traditional RS or LLM-RS design. Whether this adaptation was sufficiently optimized cannot be determined.

Response:

Thank you. We clarify the baseline adaptation and tuning protocols to ensure fair comparisons.

Consistent adaptation. As existing debiasing methods were developed for different recommendation architectures, we adapt them to the MLLM-RS setting while preserving their core debiasing mechanisms. All methods share the same backbone (Appendix E.1.4), data splits (Appendix E.1.1), candidate sets (Appendix E.1.1), and evaluation protocol (Appendix E.1.3).

Systematic hyperparameter tuning. We start from the original parameter setting described in each baseline's paper and then tune their key hyperparameters using the validation set, with search spaces reported in Table R6. Methods without additional hyperparameter search (RW, FairLRM, and LLM-FairRec) follow their prescribed configurations.

We have expanded Appendix E.1.2 to document the method-specific configurations and hyperparameter search spaces for reproducibility.

Table R6: Hyperparameter search spaces for adapted baselines.

BaselineHyperparameterSearch space
RR/RWRRReranking coefficient{0.01, 0.025, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5}
SPRecIterations[0, 4]
FUDLRRecalibrated step{0.01, 0.05, 0.1, 0.5, 1, 5}
PDAPopularity coefficient{0.025, 0.05, 0.1, 0.15, 0.2, 0.3, 0.4}
MACRCounterfactual constant{0.01, 0.0125, 0.025, 0.05, 0.075, 0.1, 0.15, 0.25, 0.5}
CFPAdversary weight[1, 5]
AuthorsOpenReview ↗

Response to Reviewer xvhw - Q1, Q3

Q1.

How stable is the concept-bias subspace across seeds? The paper acknowledges in Remark 1 that the formal distinguishability condition is not satisfied, and the entire mitigation scheme relies on the identified subspace $P_{bias}$.

Response:

Thank you. We follow your suggestion and validate the stability of concept-bias subspace across seeds.

Theoretical robustness. Remark 1 clarifies that formal identifiability of individual concepts is not guaranteed. However, TRACE operates on the causally weighted bias subspace rather than relying on fixed concept identities. Proposition 1 proves that IchainI_{\mathrm{chain}} and Gbias\mathbf{G}_{\mathrm{bias}} are invariant (up to column permutation) to concept permutation and nonzero rescaling, leaving Pbias\mathbf{P}_{\mathrm{bias}} unchanged. Thus, these coordinate ambiguities across seeds do not affect mitigation.

New cross-seed evidence. We directly measure the overlap between independently identified rank-rr bias subspaces using ∥Ur⊤Ur′∥F2/r\Vert\mathbf{U}_r^\top\mathbf{U}'_r\Vert_F^2/r. As shown in Table R7, the overlap reaches 0.95 for popularity bias and 0.97 for gender bias on MovieLens. These results demonstrate that TRACE recovers highly consistent mitigation subspaces across seeds. These results are included in Appendix E.10.5.

Table R7: Cross-seed bias-subspace stability on MovieLens.

MetricPopularityGender
Subspace overlap ↑0.94970.9697

Q3.

Is it necessary to include sensitive attributes during deployment for attribute depolarization?

Response:

No. Attributes are used only during offline identification; the deployed model is an edited LoRA adapter that receives exactly the same input as the backbone.

References

Weixin Chen, Li Chen, Yongxin Ni, and Yuhan Zhao. Causality-inspired fair representation learn- ing for multimodal recommendation. ACM Transactions on Information Systems, 43(6):153:1– 153:29, 2025.

David Maxwell Chickering. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3:507–554, 2002.

Jianqing Fan and Jinchi Lv. Sure independence screening for ultrahigh dimensional feature space. Journal of the Royal Statistical Society Series B: Statistical Methodology, 70(5):849–911, 2008.

Dan Lu, Xu Chen, Rui Chen, Shiqing Wu, and Guandong Xu. Fairness-aware mutual information for multimodal recommendation. In BESC, pp. 1–9, 2024.

Jonas Peters, Joris M. Mooij, Dominik Janzing, and Bernhard Sch¨olkopf. Causal discovery with continuous additive noise models. JMLR, 15(1):2009–2053, 2014.

Shohei Shimizu, Patrik O. Hoyer, Aapo Hyv¨arinen, and Antti J. Kerminen. A linear non-gaussian acyclic model for causal discovery. JMLR, 7:2003–2030, 2006.

Shohei Shimizu, Takanori Inazumi, Yasuhiro Sogawa, Aapo Hyv¨arinen, Yoshinobu Kawahara, Takashi Washio, Patrik O. Hoyer, and Kenneth Bollen. Directlingam: A direct method for learn- ing a linear non-gaussian structural equation model. Journal of Machine Learning Research, 12: 1225–1248, 2011.

Yu Wang, Yonghui Yang, Le Wu, Yi Zhang, Fei Liu, and Richang Hong. Multimodal large language models with adaptive preference optimization for sequential recommendation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR, pp. 1903–1913, 2026.

Xun Zheng, Bryon Aragam, Pradeep Ravikumar, and Eric P. Xing. Dags with NO TEARS: contin- uous optimization for structure learning. In NeurIPS, pp. 9492–9503, 2018.

Reviewer_1VpyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TRACE, a framework for diagnosing and mitigating bias in MLLM-based recommender systems. TRACE combines bias removal with preference-preserving correction during LoRA fine-tuning. Its bias investigation identifies interpretable concepts associated with recommendation disparities and traces their influence on the final recommendation outcomes, aiming to distinguish bias-related signals that should be removed from preference-related information that should be retained. The framework aggregates observed bias factors into an overall bias score using a weighted sum. Extensive experiments on widely adopted recommendation datasets compare TRACE against 11 baselines across multiple metrics, with results demonstrating its effectiveness in alleviating bias while maintaining recommendation performance.

Critical strengths
  1. The paper aims to address an important and relatively underexplored problem and goes beyond conventional popularity debiasing by also investigating user-attribute disparities and modality-related bias.
  2. The paper introduces a three-level framework connecting multimodal representations, interpretable concepts, and recommendation bias outcomes. Unlike approaches that only measure recommendation disparities or identify influential training samples, TRACE aims to explain the pathway through which multimodal information contributes to bias.
  3. It proposed the chain influence measure that combines sample-to-concept influence with concept-to-bias effects, allowing the estimated bias contribution of individual training samples to be quantified. This is a potentially useful bridge between influence-function-based data attribution and concept-level causal analysis.
  4. The experiments cover multiple recommendation datasets, baselines, and evaluation metrics. Comparisons against 11 baselines provide useful evidence of the framework's effectiveness across different settings.
Critical weaknesses
  1. Although the paper cites Ma et al.'s work and explain the reason of why it was not included as a baseline, it lacks a sufficiently detailed comparison and analysis of that methodology. A more direct discussion of methodological similarities and differences, would help establish the novelty of TRACE and identify the specific advantages of its proposed components.
  2. The paper's causal claims and concept-identification procedure rely on assumptions that may not hold reliably in practical MLLM-based recommendation settings. The framework would benefit from stronger validation of these assumptions and more comprehensive comparisons with standard sparse autoencoder (SAE) baselines. In particular, the paper should clarify whether its identified concepts and estimated causal effects are more reliable or interpretable than those obtained using conventional SAE-based methods.
  3. Biases observed in offline datasets may not fully reflect the disparities or user responses encountered in deployed recommendation systems. Consequently, improvements on offline fairness metrics do not necessarily establish that TRACE will improve fairness in online settings. The paper should discuss this limitation more explicitly.
  4. The sample influence estimates rely on local influence-function assumptions and a damped diagonal Fisher approximation, which may compromise ranking reliability for larger or nonlinear model updates. Sensitivity analyses on curvature approximations and ranking stability are needed to validate robustness.
Questions for authors

As I described in the weaknesses section. Besides, what are \mu_u in eq.(4) and \mu_B in eq.(5) represent respectively?

8 comments
AuthorsOpenReview ↗

Response to Reviewer 1Vpy - W1

We thank the reviewer for the constructive feedback. We follow your suggestions and clarify TRACE's methodological novelty, the scope of its causal assumptions, and the reliability and practical implications of its bias mitigation.

W1.

Although the paper cites Ma et al.'s work and explain the reason of why it was not included as a baseline, it lacks a sufficiently detailed comparison and analysis of that methodology. A more direct discussion of methodological similarities and differences, would help establish the novelty of TRACE and identify the specific advantages of its proposed components.

Response:

While both works employ causal reasoning for debiasing, they address distinct problems through fundamentally different bias identification and mitigation mechanisms.

Different problem. This preprint work by Ma et al. (2025) aims to build an end-to-end recommender framework with existing debiasing modules integrated, whereas TRACE aims to identify why a fine-tuned MLLM-RS is biased and correct its underlying sources causing bias without retraining.

Different technical solutions. The differences are twofold. (1) Bias identification. Ma et al. rely on a predefined causal structure without discovering the bias mechanisms within learned multimodal representations. Such a fixed structure cannot reveal unmodelled, dataset-specific or cross-modal bias pathways, limiting interpretability and targeted correction. In contrast, TRACE discovers causal dependencies among learned multimodal concepts and estimates bias effects through the novel chain influence. This provides interpretable bias attribution and source-guided mitigation. (2) Bias mitigation. Ma et al. incorporate propensity-weighted losses and adversarial fairness objectives, relying on optimisation-based debiasing. In contrast, TRACE translates the identified bias pathways into a one-step correction without retraining. Moreover, while Ma et al. do not explicitly address preference distortion caused by bias removal, TRACE introduces a corrective operator that reinforces genuine preference signals within the identified subspace.

Table R1: Methodological comparison with Ma et al. (2025).

AspectMa et al. (2025)TRACETRACE's advantage
ProblemBuild an end-to-end recommender framework with existing debiasing modules.Bias-source identification and efficient mitigation for MLLM-RS.Corrects existing MLLM-RSs without redesign.
Causal structurePredefined bias factors and adjustment.Discovered among multimodal concepts.Adapts to data-specific and cross-modal bias pathways.
Bias sourceNot identified.Three-level chain influence.Interpretable; guides mitigation.
MitigationLoss optimisation.Targeted one-step correction.Avoids retraining.
Preference preservationNo explicit design.Corrective operator.Preserves genuine preference.

Baseline coverage. As explained in Appendix E.1.2, Ma et al. is not directly included. Nevertheless, its main debiasing principles are covered by chosen baselines: its popularity-based backdoor adjustment is covered by PDA, while its demographic term mirrors the adversarial objective of CFP. Against these representative approaches, TRACE-RC achieves higher NPCG than PDA on all three datasets (Table 1) and lower Δ\DeltaNDCG than CFP across user attributes (Table 6).

Overall, the novelty of TRACE lies in identifying sources of bias and using them to guide targeted, preference-aware model editing through the proposed three-level chain influence and composed mitigation operators.

Thanks to your suggestions, we have included a discussion comparing with Ma et al. in Section 2 and Appendix E.1.2.

AuthorsOpenReview ↗

Response to Reviewer 1Vpy - W2

W2.

The paper's causal claims and concept-identification procedure rely on assumptions that may not hold reliably in practical MLLM-based recommendation settings. The framework would benefit from stronger validation of these assumptions and more comprehensive comparisons with standard sparse autoencoder (SAE) baselines. In particular, the paper should clarify whether its identified concepts and estimated causal effects are more reliable or interpretable than those obtained using conventional SAE-based methods.

Response:

Thank you for your suggestion. We clarify the assumptions and provide further evidence of concept and causal-effect reliability.

Assumptions and theoretical scope. (1) Concept identification: As stated in Remark 1 and Limitation (Appendix B), we do not claim formal identifiability under limited auxiliary information. Instead, the conditional prior provides an identifiability-oriented constraint. Appendix E.8 illustrates interpretable, bias-relevant concepts, while the ablations in Table 3 and Appendix E.6 demonstrate their practical utility for debiasing. (2) Causal modelling: Assumption 1 adopts the well-established linear acyclic SCM with independent noise, as commonly studied in causal discovery (Shimizu et al., 2006; Peters et al., 2014; Zheng et al., 2018). We extend its bias-outcome equation with bilinear terms to capture cross-modal interactions. This provides a tractable approximation rather than assuming arbitrary nonlinear MLLM mechanisms can be exactly recovered.

Existing evidence against standard SAEs. Tables 3 and 11 already compare AE, SAE, and iSAE. On MovieLens popularity debiasing, SAE increases tail exposure but reduces accuracy, indicating less effective separation of bias-related and preference-relevant information. For gender bias, iSAE reduces Δ\DeltaNDCG@3 from 3.25 (SAE) to 0.56 and improves FattrF_{\mathrm{attr}}@3 from 58.34 to 68.25. These results demonstrate the benefit of iSAE for preference-aware debiasing.

Built in protection. Prop. 1 shows that IchainI_{chain} and the columns of Gbias\mathbf{G}_{bias} are invariant to permutation and rescaling of concept coordinates, which is exactly the indeterminacy left by iVAE-type identifiability. Mitigation therefore depends on the bias subspace, not on the identity of individual concepts.

Additional results. To investigate representation reliability more directly, we compare SAE and iSAE in Table R2. Specifically, we use the mean correlation of aligned concept activations across seed pairs to measure the concept stability, and use Pearson between the causal signals across seeds to measure the causal effect stability. Compared with SAE, iSAE improves concept stability from 0.8015 to 0.8475 and effect stability from 0.6978 to 0.8527. The gains, particularly in causal-effect stability, indicate more robust bias-related representations and effect estimates. These results are added in Appendix E.10.1.

Table R2: Stability of concept representations and estimated causal effects.

MetricAESAEiSAE (ours)
Concept stability ↑0.8387 ± 0.010.8015 ± 0.010.8475 ± 0.01
Effect stability ↑0.7491 ± 0.070.6978 ± 0.070.8527 ± 0.03
AuthorsOpenReview ↗

Response to Reviewer 1Vpy - W3, W4

W3.

Biases observed in offline datasets may not fully reflect the disparities or user responses encountered in deployed recommendation systems. Consequently, improvements on offline fairness metrics do not necessarily establish that TRACE will improve fairness in online settings. The paper should discuss this limitation more explicitly.

Response:

Thank you for pointing this out, and we have now stated the following limitation in Appendix B:

Offline-to-online generalisation. Our evaluation measures recommendation utility and fairness on held-out interactions under fixed sampled candidate sets. These metrics do not capture online patterns, e.g., provider exposure, popularity feedback loops, or long-term welfare effects following deployment. Although NPCG evaluates ranking accuracy for observed non-popular positive interactions, it does not directly measure user satisfaction or online fairness. Future work could establish a public online testbed for MLLM-RS and examine TRACE through controlled online evaluations that measure user responses, exposure dynamics, and potential feedback effects over time.

Although this is a field-wide gap and no public online testbed for MLLM-RSs currently exists, TRACE addresses two important considerations to narrow this gap.

(1) Preference-aware fairness evaluation. Beyond tail exposure, NPCG measures ranking accuracy for genuinely preferred non-popular items, discouraging blind tail promotion (Appendix E.1.3). TRACE-RC improves NPCG across all three datasets (Table 1), while case studies (Tables 12–13) illustrate preference-consistent corrections rather than arbitrary exposure changes.

(2) Efficient adaptation to online setting. TRACE offers targeted post-training correction without retraining. On MicroLens and Baby, TRACE-RC requires about 6 hours, versus 97-142 hours for reweighting-based retraining (Table 1). This makes periodic bias re-identification and correction more feasible as interaction data evolve.

W4.

The sample influence estimates rely on local influence-function assumptions and a damped diagonal Fisher approximation, which may compromise ranking reliability for larger or nonlinear model updates. Sensitivity analyses on curvature approximations and ranking stability are needed to validate robustness.

Response:

While local influence approximations can become inaccurate for large or nonlinear updates, our new sensitivity analysis (Table R3) shows that TRACE's influence rankings and debiasing performance remain stable across the tested curvature approximations. We provide both empirical and theoretical evidence below.

(1) Empirical ranking and performance stability. We conduct a new sensitivity analysis (Table R3), comparing diagonal Fisher across damping strengths λ∈{10−4,10−3,10−2,10−1}\lambda\in\{10^{-4},10^{-3},10^{-2},10^{-1}\}, DataInf (Kwon et al., 2024), a closed-form influence approximation designed for LoRA, and the full-matrix empirical Fisher as a reference.

The diagonal Fisher achieves consistently high Spearman correlations (0.8606-0.8636) for the magnitude ranking of IchainI_{\mathrm{chain}} relative to the reference; DataInf also shows substantial agreement (0.7991). More importantly, Fpop@3F_{\mathrm{pop}}@3 remains within 0.41 points of the reference (23.04) across all approximations. These results demonstrate that TRACE's influence ranking and downstream debiasing performance are relatively insensitive to the tested curvature approximations.

(2) Subspace-projected update. TRACE further restricts updates to the identified bias subspace Sbias\mathcal{S}_{\mathrm{bias}}. By Proposition 2.1, projection removes curvature-approximation errors orthogonal to this subspace without amplifying the remaining error. Moreover, it preserves the full update's first-order effect on functionals whose gradients lie in Sbias\mathcal{S}_{\mathrm{bias}}. This provides a safeguard against updates in irrelevant parameter directions, complementing the empirical ranking stability.

Together, these findings support the robustness of TRACE under different practical curvature approximations and are included in Appendix E.10.3.

Table R3: Sensitivity of sample curvature approximation (popularity debiasing on MovieLens).

Metricsλ=10⁻⁴λ=10⁻³λ=10⁻²λ=10⁻¹DataInfReference
Spearman ↑0.86360.86180.86200.86060.79911.0000
FpopF_{pop}@3 ↑23.0522.9223.0522.6322.9123.04

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.