- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TRACE, a post-hoc framework for identifying and mitigating bias in recommenders built by LoRA fine-tuning a multimodal LLM. Identification proceeds in three levels: modality-specific hidden-state views are encoded by an "identifiability-oriented" sparse autoencoder whose prior is conditioned on a bias-relevant auxiliary variable; a linear SCM with cross-modal bilinear terms is fitted over selected concepts and a bias outcome to obtain per-concept effects; and influence functions link training samples to concept activations, giving a chain influence equal to the τ-weighted sum of sample-to-concept influences. Mitigation applies a one-step, closed-form update on the LoRA adapter projected onto the span of the τ-weighted concept gradients: a removal operator unlearns the top-|chain-influence| samples, a corrective operator pushes up preferred-but-less-popular items in fixed-norm steps, and a composed operator combines both. Experiments on MicroLens, Amazon Baby, and MovieLens-1M with Qwen2.5-VL-3B cover popularity, gender/age/occupation, co-existing, and modality-dominance bias.
Critical strengths
- Relevant problem and general formulation. Bias in MLLM-based recommenders is underexplored. The paper defines bias through a generic variable and an aggregation , which lets one framework cover popularity, user-attribute, co-existing and modality-dominance bias.
- Identification and mitigation are connected, without retraining. The chain influence links training samples, multimodal concepts and the bias outcome, with an invariance property under concept rescaling and permutation. The removal and corrective operators reuse the identified subspace and come with first-order guarantees. The edit is applied directly to the LoRA adapter and takes about 4–6 h, compared with about 97 h for retraining-based methods such as RW on MicroLens.
- Broad evaluation and interpretable analysis. Experiments cover three datasets, two backbones, a wide range of baselines, ablations and sensitivity studies. The concept analysis shows semantically coherent concepts whose activations vary systematically with popularity, together with interpretable cross-modal pathways in the learned SCM.
Critical weaknesses
- The identification claim at the sample level is not validated. The central contribution is that attributes measured bias to individual training samples. Influence estimates for fine-tuned large models with a damped diagonal Fisher are known to be unreliable (Appendix B acknowledges this), yet the paper never checks whether the predicted change in or in from removing high- samples matches the actual change. Without such checks, the improvements in Section 5 could come from the projected update direction rather than from accurate sample attribution. Relatedly, is selected by , which includes samples whose upweighting decreases bias; unlearning those should increase bias. The paper should explain why the absolute value, rather than a signed criterion, is used.
- Ranking uses one positive against 20 sampled negatives. TailP@K over sampled candidates depends on the popularity distribution of the negative sampler, which is not described; Figure 1(a) suggests candidates are themselves heavily tail-weighted. is the harmonic mean of NDCG (around 50 to 60) and TailP (around 10 to 20), so it is dominated by TailP.
Questions for authors
In addition to the weaknesses, I have one further question for the authors: TRACE-R reports identical accuracy results, standard deviations included, across different bias settings. On MovieLens, popularity debiasing (Table 1) and gender debiasing (Table 2) both give AUC 84.08 ± 0.17, NDCG@3 50.28 ± 0.73 and NDCG@5 55.75 ± 1.29. On MicroLens, popularity debiasing (Table 1) and modality-dominance debiasing (Table 8) both give AUC 84.66 ± 0.08, NDCG@3 49.49 ± 0.13 and NDCG@5 55.17 ± 0.30. Since the concepts, SCM, and all depend on the bias variable, the updates should differ. Were these separate runs? Please explain.
5 comments
Response to Reviewer Af63 - W2, Q1
W2.
Ranking uses one positive against 20 sampled negatives. TailP@K over sampled candidates depends on the popularity distribution of the negative sampler, which is not described; Figure 1(a) suggests candidates are themselves heavily tail-weighted. $F_{pop}$ is the harmonic mean of NDCG (around 50 to 60) and TailP (around 10 to 20), so it is dominated by TailP.
Response:
We clarify the negative-sampling protocol and the interpretation of our fairness metrics.
Consistent candidate sampling. As described in E.1.1, we follow the commonly used protocol in MLLM-RS (Ye et al., 2025; Wang et al., 2026) to sample negative candidates. Specifically, for each test positive, 20 negatives are drawn uniformly at random with a fixed seed. All methods are evaluated on the same candidate sets for a fair and controlled comparison. On MicroLens, Baby, and MovieLens, tail-tier (T1) and popular-tier (T5) items account for 35.51% and 18.47%, 32.99% and 21.24%, and 48.02% and 15.62% of the sampled candidates, respectively. We have explicitly reported the sampling procedure and candidate distributions in Appendix E.1.1.
Fairness is not assessed using alone. We agree that, because TailP is smaller than NDCG numerically, the harmonic mean Fpop is more sensitive to changes in TailP. Therefore, we report NDCG, TailP, and NPCG separately, rather than relying on alone. Specifically, NPCG measures ranking accuracy for non-popular target items, distinguishing preference-consistent debiasing from blind tail promotion. For example, on MicroLens, TRACE-RC improves both TailP@3 and NPCG@3, while maintaining NDCG@3. Similar joint improvements in TailP and NPCG are observed on Baby and MovieLens (Table 1).
Q1.
In addition to the weaknesses, I have one further question for the authors: TRACE-R reports identical accuracy results, standard deviations included, across different bias settings. On MovieLens, popularity debiasing (Table 1) and gender debiasing (Table 2) both give AUC 84.08 ± 0.17, NDCG@3 50.28 ± 0.73 and NDCG@5 55.75 ± 1.29. On MicroLens, popularity debiasing (Table 1) and modality-dominance debiasing (Table 8) both give AUC 84.66 ± 0.08, NDCG@3 49.49 ± 0.13 and NDCG@5 55.17 ± 0.30. Since the concepts, SCM, $D_{bias}$ and $P_{bias}$ all depend on the bias variable, the updates should differ. Were these separate runs? Please explain.
Response:
Thank you for pointing this out.
Source of the results. Yes, TRACE-R was independently instantiated for each bias setting, with bias-specific concepts, SCM, sample selection, and projection subspace. However, the evaluation script mistakenly used checkpoints from another bias setting on the same dataset: MovieLens popularity reused gender-derived results, and MicroLens modality debiasing reused popularity-derived results. We traced the affected rows to their respective checkpoints and reported the corrected results in Tables R10 and R11, and corrected the rows in Tables 1 and 8.
Observations. On MovieLens popularity debiasing, the corrected TRACE-R improves NDCG@3 from 50.28 to 53.79 and NPCG@3 from 37.47 to 40.51, with a modestly lower TailP@3 (14.40 to 14.07). On MicroLens modality debiasing, NDCG@3 increases from 49.49 to 54.27 and WI-N@3 from 42.33 to 48.40. The corrected results substantially reduce the previously reported accuracy degradation.
Verification and revision. We additionally audited the results of TRACE-R, TRACE-C, and TRACE-RC against their respective evaluation records and confirmed that the remaining reported rows are correct. We have corrected the affected tables and corresponding discussion. The results and conclusions for the full TRACE-RC framework remain unchanged.
Table R10: Popularity debias of TRACE-R on MovieLens.
| Method | AUC ↑ | NDCG@3 ↑ | NDCG@5 ↑ | TailP@3 ↑ | TailP@5 ↑ | NPCG@3 ↑ | NPCG@5 ↑ | @3 ↑ | @5 ↑ |
|---|---|---|---|---|---|---|---|---|---|
| Submitted | 84.08 | 50.28 | 55.75 | 14.40 | 18.02 | 37.47 | 43.20 | 22.39 | 27.24 |
| Corrected | 84.89 | 53.79 | 58.96 | 14.07 | 16.92 | 40.51 | 45.98 | 22.30 | 26.29 |
Table R11: Modality debias of TRACE-R on MicroLens.
| Method | AUC ↑ | NDCG@3 ↑ | NDCG@5 ↑ | WI-N@3 ↑ | WI-N@5 ↑ | @3 ↑ | @5 ↑ |
|---|---|---|---|---|---|---|---|
| Submitted | 84.66 | 49.49 | 55.17 | 42.33 | 49.32 | 45.63 | 52.08 |
| Corrected | 86.42 | 54.27 | 59.55 | 48.40 | 54.36 | 51.16 | 56.83 |
References
Juhan Bae, Nathan Ng, et al. If influence functions are the answer, then what is the question? NeurIPS, 2022.
Yu Wang, Yonghui Yang, et al. Multimodal large language models with adaptive preference optimization for sequential recommendation. SIGIR, pp. 1903–1913, 2026.
Yuyang Ye, Zhi Zheng, et al. Harnessing multimodal large language models for multimodal sequential recommendation. AAAI, pp. 13069–13077, 2025.
Dear Reviewer,
Thank you again for your detailed and constructive feedback. In response to your comments, we have provided additional validation of sample-level attribution, further analysis of influence-based sample selection, and clarification of the evaluation protocol. We have also corrected the inconsistencies and verified the remaining results in the revised manuscript.
We sincerely hope these clarifications and additional results are helpful, and we would be glad to answer any remaining questions or provide further clarification.
Thank you again for your valuable suggestions and consideration.
Best regards,
Authors
Response to Reviewer Af63 - W1
We thank the reviewer for the careful assessment. We directly address the concerns regarding sample-level attribution, fairness evaluation, and reporting consistency.
W1.
Response:
Thank you for the valuable suggestions. We provide direct validation of sample-level attribution and clarify the rationale for magnitude-based selection.
(1) Does the identification predict actual bias change? We conduct a controlled leave-group-out test on MovieLens. We select the top/bottom 128 and 256 training samples by signed Ichain. Starting from the same fine-tuned adapter, we perform one additional epoch of warm-start training, with and without each selected group, and measure the resulting difference in Jcen. This test is motivated by the warm-start influence analysis of Bae et al. (2022). Table R8 shows that the predicted and observed changes agree in sign for all four groups. Removing positive-influence samples decreases Jcen, whereas removing negative-influence samples increases it. This directly supports the directional validity of sample attribution. As expected for local influence approximations, the magnitudes are less precisely matched.
(2) Why the absolute value? Magnitude ranking was chosen to capture the samples that most strongly engage the bias-relevant concepts. Then, TRACE-R applies an aggregated, bias-subspace-projected update rather than deletion of each sample, and the resulting effect depends on the combined influences. To evaluate this choice, we compare absolute selection with signed selection (Table R9). The absolute variant yields higher TailP@3 and NPCG@3, while the signed variant achieves higher AUC and NDCG@3. This shows a fairness-utility trade-off rather than a consistent advantage of either criterion.
Thank you for your valuable comments. We have clarified the signed interpretation of Ichain, the selection rationale, and added the new validation protocol in Section 4.4 and Appendix E.10.6.
Table R8: Predicted vs. observed change of Jcen after removing sample groups on MovieLens.
Table R9: Comparison of absolute and signed TRACE-R (popularity debiasing on MovieLens).