- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work addresses the problem of model lineage tracing. Output similarity typically only determines whether models are related or reconstructs undirected structures; reconstructing directed model lineages usually relies on weighted access. The authors aim to recover internal nodes and parent-child directions using only teacher-forced log-likelihood under gray-box conditions. TRACE-Edge designs directed edge scores based on the likelihood differences across tasks, and then uses the Chu–Liu/Edmonds algorithm to find the maximum weighted directed spanning tree.
Critical strengths
The problem is meaningful, and the distinction between "model similarity" and "direct parent-child relationship" is worth investigating. The authors also clearly demonstrate that gray-box access is superior to black-box access that only observes the output. The experiment included five configurations, five seeds for each, totaling 25 trees. In Table 2, when using training samples as probes, TRACE-Edge's parent-match accuracy was 0.740, better than MST's 0.620; TRACE-Set was even higher at 0.837. The paper does not conceal the fact that another method yielded better results in this setting.
Figures 3 and 4 are helpful in explaining the reasons for the failure. Under the training sample probe, most of the ancestor task signals were preserved; after switching to held-out samples, these signals were significantly weakened. The positive ancestor task recognition rate given in the text decreased from 0.917 to 0.458, which is consistent with the decline in tree recovery performance.
Critical weaknesses
W1: The advantage under realistic probe conditions is small and unstable. The held-out accuracy in Table 2 is 0.558, and the MST is 0.507; in the HET-8B configuration, the TRACE-Edge is only 0.233, lower than the MST of 0.567. Therefore, the method cannot yet be considered to stably recover the lineage of different models.
W2: Full parameter fine-tuning results significantly weaken the generalization claim. In Appendix D.4 and Table 6, under the held-out condition, TRACE-Edge is 0.500, but the MST reaches 0.867. This is not a "minor degradation" in the general sense, but a reversal of the main advantage.
W3: All nodes are observed, the root node is usually known, the task set is fully known, and nodes share the fine-tuning configuration. Real-world model source tracing often lacks intermediate checkpoints and includes unknown tasks, mixed data, and different hyperparameters.
W4: 1. For Case (4), the fraction in the table is written as , but Equation (7) should actually take the maximum value of . For example, when , the fraction in the formula is 1, but the table shows 9. This row needs to be corrected unless additional conditions are added.
Questions for authors
Q1 Why does it significantly lose to MST in a Full-FT held-out setup?
Q2 How does Simple Distance MST perform under the same log-likelihood access conditions?
Q3 Please correct Table 1 Case (4) and clarify when edge scores can actually distinguish between parent nodes and more distant ancestors.
Response
Thank you very much for your detailed review and for recognizing the problem setting. We ran additional experiments following the suggestions from you and the other reviewers. Their results require substantial revisions to both the content and the claims of the paper. We have therefore decided to withdraw the submission and resubmit it after revision. Thank you again for your time and effort.