- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This controlled pilot examines three distinct issues in multimodal knowledge graph completion: transparent-image preprocessing, positive-label filtering with model scores held fixed, and visual-package reassignment in saved MyGO models. Its central observation is that a small mean MRR change can coexist with changes to many individual query ranks.
Critical strengths
-
The paper separates changes to input preprocessing, evaluation labels, and fixed-model visual assignments. This makes its diagnostic measurements easier to interpret.
-
The query-level absolute changes and zero-visual control reveal behavior that aggregate MRR alone conceals. The limitations are stated candidly.
Critical weaknesses
-
The main MyGO comparison uses one training seed and 100 epochs, with no evidence of convergence; the referenced implementation uses a much longer training budget. Five new input mappings do not substitute for independent training runs. Consequently, the central difference between training conditions remains preliminary.
-
The three experiments examine image preprocessing, evaluation rules, and visual inputs, but use different experimental settings and do not yet support a unified, convincing conclusion. It remains unclear whether these findings generalize to other models and datasets.
Questions for authors
Do the original-, permuted-, and zero-visual patterns persist across multiple training seeds, matched execution conditions, a sufficiently long training budget, and an independently selected evaluation split?