- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies document-graph construction without autoregressive serialization. It proposes Invariant Graph Readout, combining direct graph prediction from frozen VLM visual features with graph-edit contracts that preserve designated state during correction and protected completion. It evaluates graph quality and latency against strict JSON generation on OracleGraph, and instantiates the completion interface with target-domain supervision on GraphDoc.
Critical strengths
-
The preservation contracts are specified concretely and are actually enforced rather than merely described informally.
-
The paper evaluates practical system behavior rather than reporting only static graph scores.
-
The paper is easy to follow.
Critical weaknesses
-
The methodological novelty appears relatively limited. Non-autoregressive graph prediction is already well established, while the proposed preservation contracts primarily enforce explicit constraints on graph-editing operations.
-
Despite the framework's complexity, the improvements remain modest. On OracleGraph, Full improves Graph F1 by only 1.16 percentage points over JSON Generation, while Edge F1 actually decreases. Moreover, protected completion contributes only 0.19 points over Guarded Readout. On GraphDoc, the final Graph F1 remains just 0.1069, raising concerns about the framework's practical effectiveness.
-
Although Section 4.2 describes the 1,000 GraphDoc evaluation pages as held out from training, Appendix D.1 acknowledges that these pages had been exposed during earlier system design. This potentially compromises the independence of the evaluation, particularly since GraphDoc provides the primary evidence for cross-domain reuse. Evaluation on a genuinely untouched test split would strengthen this claim.
-
The OracleGraph comparison does not adequately control for training resources or baseline strength. Full uses 5,758 fitting sources, compared with 4,700 for Strict JSON Generation. Furthermore, the JSON baseline disables grammar-constrained decoding and JSON repair, with parsing failures treated as empty graphs. Supervision-matched experiments and stronger structured-generation baselines are needed to establish whether the reported gains genuinely arise from the proposed graph-native interface.
Questions for authors
-
For the GraphDoc pages described in Appendix D.1 as having been exposed during earlier design work, what information from those pages was available during that process? Is there a fully untouched GraphDoc split on which the frozen revised pipelines can be evaluated?
-
For Strict JSON Generation in Appendix B, what fraction of OracleGraph outputs fail strict parsing, and how would Graph F1 change under grammar-constrained decoding or a validity-preserving JSON decoder? This would determine whether the quality difference in Tables 3 and 4 primarily reflects structural prediction quality or parser-induced empty outputs.