- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposed the SecBFF framework which tries to address the temporal security event detection when graph coverage is not complete by combining graph structure with LLMs. The proposed method is topology-injected masked fine-tuning. It replaces the text embeddings of entities which are covered by pre-trained GFMs with graph embeddings with linear projection and layered normalization. For entities without graph embedding, the original text embedding will be kept. Authors evaluated the method on three datasets with five detection tasks. Authors claimed that using SecBFF, it outperformed text-only models and prefix-based graph LLMs in all tasks.
Critical strengths
Even though the major proposed methodology sounds not novel enough, the paper also included dive deep analysis of the method's effectiveness. The paper conducted analysis on entity embedding norms, entity-specialized attendtion heads, entity-to-entity attention to explore how topology injection affects the model's internal representations and attention behavior which is useful research.
Critical weaknesses
The proposed framework and suggested fine-tuning technique fit into general mechanisms of combining graph embeddings with LLMs which doesn't suggest enough novelty. Besides, the testing datasets none of which include more than millions of events or million-level of nodes and edges are actually small scale in practice. The effectiveness of the method upon real world large scale of data and on complex large graphs is unclear.
Questions for authors
Regarding this claim from the paper "Experiments across enterprise security logs, blockchain transactions, and cybersecurity events demonstrate that SECBFF consistently outperforms text-only and graph-text baselines on classification tasks, with the largest gains in low-context settings such as zero-shot and one-event detection where topological priors are most valuable.", I would suggest to benchmark the fine-tuned model against the latest cutting edge LLMs, i.e. Opus 5.5 etc., to suggest its effectiveness.