- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents FLOWREADER for evidence selection and assembly in long multimodal document question answering. The method computes query relevance over an offline content graph, applies spectral decomposition to relevance-weighted node embeddings, and assigns flow quotas to latent aspects in proportion to their spectral energy. A single capacitated minimum-cost flow optimization produces candidate evidence chains, which are subsequently selected, read in parallel by a vision-language model, and reconciled by a reasoner. The theoretical result guarantees that the complete routed flow satisfies the prescribed aspect allocation, but does not guarantee semantic coverage or preservation of that allocation after path selection. On VisDoMBench, the system achieves a macro accuracy of 68.9, exceeding the strongest reported baseline by 2.7 percentage points. Controlled ablations indicate that cost design contributes substantially to accuracy, while capacity constraints reduce prompt-token consumption by approximately 27% relative to the uncapacitated shortest-path configuration at similar accuracy.
Critical strengths
1.The paper formulates multimodal evidence assembly as a minimum-cost flow problem with aspect quotas, providing an alternative to online LLM-based sub-question planning. On VisDoMBench, the system achieves a macro accuracy of 68.9, improving over the strongest reported baseline by 2.7 percentage points, leading on three of five subsets, and achieving the highest worst-subset accuracy. Accuracy remains at 68.0 when the average measured number of distinct nodes read is reduced to 12.9. These results demonstrate practical utility in the evaluated setting, with improvements extending beyond a single subset 2.The paper compares multiple selection strategies, routing methods, and cost components while holding the graph, node scores, reader, reasoner, and judge fixed. The shortest-path comparison is particularly informative: it indicates that the main benefit of capacity constraints is reduced token consumption at similar accuracy, rather than an additional accuracy improvement. The authors also explicitly disclose the scope of the coverage guarantee, its limitations after path selection, and confounding factors in some ablations and evaluation procedures. These analyses and disclosures help identify which parts of the method are effective and where its conclusions apply
Critical weaknesses
1.The theoretical guarantee constrains only the numerical allocation of the complete flow. Final path selection need not preserve that allocation, and coverage of the actual semantic facets required by the question is not directly evaluated. The single-sink ablation substantially changes the termination scale when removing aspect nodes, so its performance degradation cannot be attributed specifically to the removal of spectral aspects. The 0.45-percentage-point difference between spectral aspects and LLM-generated sub-questions, with , establishes only that the experiment did not detect a significant difference; it does not establish semantic alignment or statistical equivalence.Meanwhile, the shortest-path configuration achieves similar accuracy, but the paper does not provide a comparison with matched token consumption or constrained chain length, nor does it isolate the respective effects of content-arc and aspect capacities. Because spectral allocation and capacity constraints are central contributions, these gaps limit assessment of their independent value and necessity
2.Figure 2 compares G²-Reader’s configured node upper bounds with FLOWREADER’s measured distinct-node counts, which does not quantify actual cross-system cost savings. Moreover, relative to G²-Reader, the system changes the embedding model, visual-node representation, and reading prompts, so the overall 2.7-percentage-point improvement cannot be attributed solely to the routing method. Appendix D discloses that some evaluation examples were used for parameter checks and prompt development, and reports limited score changes and unchanged rankings after excluding these 250 examples. This is meaningful mitigating evidence, but complete results on the remaining examples and clarification of whether failure analysis was strictly confined to development examples are still needed. The manuscript also insufficiently specifies residual gain, bridge, , and the selector’s annealing and pruning settings, hindering independent reproduction. Together, these issues limit assessment of the sources of improvement, evaluation independence, and practical efficiency
Questions for authors
1.Could you compare the spectral-aspect and single-sink configurations with a matched termination scale , and report aspect coverage or allocation retention before and after path selection? This would help distinguish the contribution of spectral allocation from that of termination-cost calibration.
2.Could you isolate the effects of content-arc and aspect capacities, and provide a shortest-path comparison with constrained chain length or matched measured token consumption? The current results do not establish which capacities primarily produce the savings, or whether simpler cost controls can achieve a similar accuracy–cost trade-off.
3 comments
Response to Reviewer 4HH3 (2/3): capacity isolation, comparison with G²-Reader, and evaluation independence (W2, Q2)
Q2: Which capacities produce the saving, and can a simpler cost control match it?
We lifted each family of capacities separately on the ablation set, with κ, the costs, the total flow, the number of chains, and the selector unchanged. We also ran the simpler control the reviewer suggests: shortest path over the same network with every chain capped at 2 nodes.
| Variant (κ = 1.3) | FlowReader | Variant | Chain length | Reader tokens | Every worker refuses | p |
|---|---|---|---|---|---|---|
| FlowReader, all capacities | 69.2 | — | 1.39 | 45.7k | 7.6% | — |
| Aspect quotas lifted | 69.2 | 69.6 | 1.29 | 43.9k | 7.6% | 1.0 |
| Content capacities lifted | 69.0 | 69.4 | 2.10 | 54.0k | 12.9% | 1.0 |
| Shortest path, chains ≤ 2 nodes | 69.1 | 63.0 | 1.22 | 43.6k | — | 0.014 |
The saving comes from the unit content capacities. Lifting them keeps accuracy but lengthens chains from 1.39 to 2.10 nodes, raises reader tokens by 18%, and narrows the evidence from 17.3 to 13.6 distinct nodes. Lifting the aspect quotas changes neither accuracy nor cost; they guarantee allocation and facet coverage.
Capping chain length does not reproduce the trade-off. The capped shortest path matches FlowReader's token cost (43.6k against 45.7k) but loses 6.1 points (63.0 against 69.1, p = 0.014). The capacitated flow achieves the lower cost while maintaining accuracy. These results are in Section 4.3(b) and Appendix J of the revised paper.
W2: Figure 2 and the comparison with G²-Reader
We now measure both systems under the same conditions. We ran G²-Reader's released code, unmodified, on our content graphs, using the same embedder, visual-node representation, backbone, and judge; the only difference is the reasoning workflow.
| System | Accuracy | Distinct nodes | Model calls per question | Reader tokens | Latency (mean, median) |
|---|---|---|---|---|---|
| FlowReader | 71.0 | 17.1 | 18 (16 in parallel) | 45.9k | 20 s, 15 s |
| G²-Reader, planning graph | 62.5 | 8.4 | 8.5 (sequential) | 41.6k | 118 s, 70 s |
FlowReader is 8.5 points ahead on the ablation set (38 vs 17 discordant, p = 0.006), reads twice as many distinct nodes at a similar token cost, and answers about 6 times faster. Figure 2 now shows G²-Reader's measured node count alongside the configured maximum.
W2: Evaluation independence
We follow G²-Reader's protocol: its released ablation set, data, and parses; the same backbone, judge, and decoding; and its evaluation prompt, byte for byte. We ran all our ablations, the constants sweep, and the prompt-wording sweep on the ablation set only.
Three earlier choices were made by comparing two or three alternatives on the full sets: the worker prompt family (+2.5 points on two subsets), the reasoner prompt (+0.6 macro points), and a refusal fallback added after an audit of one full-set run. The FetaTab example we disclosed came from that audit, and its effect is bounded by 0.2 macro points. Appendix D now lists these with their effect sizes.
Results on the questions outside the ablation set, as requested:
| Configuration | All questions | Outside the ablation set |
|---|---|---|
| FlowReader (reported) | 68.6 | 68.3 |
| FlowReader, κ = 1.3 | 69.3 | 69.7 |
| Single sink, κ = 1.3 | 68.6 | 68.5 |
| Even split, κ = 1.3 | 69.2 | 69.4 |
| Centered SVD, κ = 1.3 | 68.3 | 68.5 |
| MMR-16 | 66.7 | 66.7 |
Each configuration moves by at most 0.35 points, and each comparison made in the paper keeps its direction or moves toward FlowReader: the single sink, the even split, and the centered SVD are each compared with the κ = 1.3 control, and MMR with the reported configuration. Per-subset results are in Appendix N.
Response to Reviewer 4HH3 (3/3): reproducibility (W2)
W2: Reproducibility
The revised paper now defines each of these, as implemented in the released code (Appendix A.2 for the costs, Appendix C for the selector). Here u, v are unit node embeddings, q is the query, and N is the number of nodes.
- Residual gain. : how much of the query not covered by u is explained by v. It is conditioned only on the immediate predecessor, which keeps arc costs additive.
- Bridge. , normalized by its maximum over arcs. are the keyword tokens of v (content tokens if it has none), , and only tokens with count.
- Edge score. , where if the two nodes differ in modality and 0 otherwise. This is Eq. 9 with β = 0.30 and γ = 0.20.
- Agreement. divided by its relevance-weighted mean over arcs. The traversal cost is then , as in Eq. 2.
- Replicator selector. Chain quality is , with payoff and , where is the Jaccard overlap of the node sets. Shares start at and are updated by .
- Annealing and pruning. After each update, the shares are blended towards the uniform distribution over the n surviving chains with weight . The lift of a chain is its share above the blend's uniform level, . A chain is removed when . Iteration stops at convergence (190–305 iterations in practice) or after 1,000.
Response to Reviewer 4HH3 (1/3): guarantee after selection, facets, equivalence, and single sink at matched κ (W1, Q1)
We thank the reviewer for carefully assessing the guarantee and the ablations. We ran experiments for both questions and report the results below; all are in the revised paper, marked in blue.
How to read the results below. We use the evaluation sets and the ablation set released by G²-Reader. Accuracy is macro accuracy (the mean of the five subset accuracies), judged by GPT-4o-mini with G²-Reader's prompt. Comparisons are paired on the questions both systems answered, with an exact McNemar test. FlowReader appears in two forms: its reported configuration and a control with κ fixed at 1.3, against which we compare every aspect and capacity ablation.
W1: Does the guarantee survive chain selection, are the question's facets reached, and are spectral aspects equivalent to sub-questions?
After selection. We added a result that extends the guarantee from the routed flow to the chains that are read. Because source and content-to-aspect arcs have unit capacity, the flow decomposes into chains with flows f1≥⋯≥fK in (0,1], with ∑kfk=F and the chains ending at aspect a carrying da.
Proposition 2 (retention under selection). Let F=F0. For any selected set S of chains, let Ra(S) be the flow of the selected chains ending at aspect a. (i) The unmet allocation equals the flow of the chains not selected, ∑a(da−Ra(S))=∑k∈/Sfk. (ii) If S is the B chains of largest flow, the unmet allocation is at most (1 − B/K)F, and zero when B ≥ K. (iii) Every aspect with da>∑k∈/Sfk receives at least one selected chain. With the 1% margin on the routed demand used in the paper, (i)–(iii) hold up to an additive 0.01F0.
Facets. The guarantee is stated over spectral directions, so we tested directly whether they are reached as facets of the question. FlowReader reaches 83.3% of the secondary facets against 73.0% for relevance ranking, and reaches more facets than ranking on 140 queries against 7 (p = 3×10⁻³³; Appendix M). We also state the directions: ϕ1 is the unit vector maximizing ∑vhv2cos2(ev,ϕ) over the relevant content, da/F is the share of that energy along direction a, and the retained directions are the best rank-A approximation (Eckart–Young).
Equivalence. We agree that p = 0.35 shows only that no difference was detected, so we now test equivalence formally. Two one-sided tests on the same paired data from the evaluation sets give a 90% interval of [−0.47, +1.93] points for sub-questions minus spectral aspects. The two are equivalent within ±2 points (p = 0.04), so spectral aspects match LLM-generated sub-questions without a planning call. Section 4.3 now states the result in these terms.
Q1: Spectral aspects against a single sink at matched κ, and allocation retained after selection
We held κ at 1.3 and collapsed the aspect nodes into a single sink, with everything else unchanged. We ran an even split across the aspects as a second control.
This separates the two contributions as the reviewer asked. The 44.8 in Table 3 reflects the effect of the fivefold change in κ, and the revised paper presents it only as a termination-calibration result. At matched κ, the single sink is equivalent in accuracy within ±2 points (paired 90% interval [−1.9, +0.5] on the evaluation sets). The aspects add coverage: with a single sink, the share of questions on which every worker finds nothing answerable rises from 6.7% to 8.3%, and the aspects carry the allocation guarantee of Proposition 2.
The spectral weighting is also the cheaper allocation. An even split reaches the same accuracy on the evaluation sets only with 11% more node visits and 5% more reader tokens.
On retention, as reported under W1: after selection, 90.0% of the allocation is met unit for unit, and aspects allotted at least 1 unit are read in 99.8% of cases and receive 98.5% of their allocation (per-aspect table in Appendix K). These results are in Section 4.3(d) and Appendix J of the revised paper.