- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes a multi-agent semantic enrichment framework to address the lack of semantic information. The framework generates structured user and merchant features and descriptions from transaction behaviors and preserves the original transaction structure. Based on the framework, the authors introduce a new multimodal financial fraud dataset MS-FFSD. The experiments evaluate semantic quality through statistical distribution alignment, demonstrate the benefits of enriched semantics for fraud detection across multiple GNN models and provide qualitative examples of LLM-based contextual reasoning.
Critical strengths
- Fraud datasets are usually hard to obtain since most institutions can not release this kind of dataset due to compliance requirements. A new fraud dataset (generation method) somehow narrows this gap so that the academic community can engage more with the problem. Existing LLM work on fraud detection has proven the importance of learning from semantic information. This work is a timely complement to support the methodology research.
- Good engineering design. The authors consider multiple sides of a transaction, including the user, merchant, and transaction itself. The use of demographic information is a bonus since it is good for downstream use in response AI studies, but it is also important to consider how to protect PII and avoid data misuse.
- The paper is well-written and easy to follow and the codebase looks good to the reviewer.
Critical weaknesses
- As a paper submitted to the dataset & benchmark area, the quantitative evaluation is limited. The paper emphasizes that enriched semantics could be used for LLM-based financial fraud reasoning. However, the full manuscript provides only two qualitative examples without any quantitative evaluation. It remains unclear whether the generated semantics actually improve LLM reasoning capability/accuracy on fraud detection, especially given that there are already some works letting LLMs leverage semantic information for fraud detection. Quantitative comparisons are needed to support the claim.
- While introducing a new financial fraud dataset is valuable, the experiments provide limited insights into what we can actually learn from it. Beyond modest performance improvements across GNN models, it would be helpful to know when and why semantic enrichment helps fraud detection, and what practical implications these findings offer. I would expect a more insightful analysis that goes beyond benchmarking existing models and provides actionable findings to fight fraud.
- The paper aims to address semantic scarcity through behavior-grounded enrichment, but 69.56% of users have only a single transaction and are enhanced by deterministic templates. Does this make those examples degrade to the regular user simulator like previous work? If yes, this raises a concern about how much of the enriched semantics is actually grounded in observed behavior.
Questions for authors
Check the above weaknesses
- You use IEEE-CIS to simulate the temporal distributions. However, this dataset is based on US consumers, and the proposed dataset is based on Chinese consumers with different payment patterns. If such migration would lead to a shifted distribution and how to prevent that?