- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Sheet-Raptor, which answers questions over messy spreadsheets where the answer requires combining tables from different sheets or files.
Critical strengths
The problem is real and not much studied. Answering queries often needs data from other files, and the three failure cases mentioned in Figure 1 show this clearly.
Critical weaknesses
Sections 1–2 (Introduction)
- W1. A lot of terms are inconsistent and symbols are reused. "Spreadsheet", "file" and "sheet" are used for different things, and several symbols are reused (ex. calligraphic F is the file collection in 2.1 but the set of folded regions in Algorithm 1, m is both the number of sheets and the number of columns in Eq. 1, etc.).
- W2. M = (F, S, R) is not explained where it is first introduced (2.1). It becomes clear only in 3.1 that M is the table's address (file, sheet, range). Also, how header cells H are separated from data cells D is never stated.
- W3. Figure 1 is hard to read. What do the S=0.9 / 0.85 / 0.8 labels mean, and what are the Step1–Step5 arrows? Also, the caption does not walk through the running example used in Section 2.2.
Section 3.1 (Preprocessing)
- W4. The table representation is described only through words. There is no worked example of a serialized sheet and the table ranges the LLM predicts. Also, the snippet in Figure 2 is unreadable. Further, the paper never reports which LLM detects the tables, what prompts are used, or how accurate the detection is.
- W5. Folding is unclear, and several parts in it are not specified properly. For example, what counts as a "repeated structure" becomes concrete only with the 36-row monthly example at the end. Similarly, W_max gets just an upper bound (< 10), and ρ_min is never given a value, why?
Section 3.2 (Retrieval)
- W6. Eq. 2 uses L^f, L^s_i, L^t_{i,j} and L^{rc}_{i,j,k} without explaining them. Also, how are the summaries produced? LLM? prompt?
- W7. The training data for the retriever is not clear. The paper only says "about five million task–context pairs drawn from open and enterprise QA data", but no dataset is named.
- W8. The paragraph about question rewriting is long and hard to follow.
Section 3.3 (Agentic Analysis)
- W9. There is no overview of the agents and operations. Except the routing agent, the other agents (QA, manipulation, visualization) are introduced only in 3.3. And the whole section reads like a list of separate facts.
- W10. Further, claims are stated without examples. For example, "a structural edit can change the ranges needed by a subsequent edit" sounds plausible, but it is never shown.
Section 4.1 (Benchmark)
- W11. The new benchmark is not well motivated or described. Why the existing benchmarks are not enough is covered in only a few lines, in 4.2. Who annotated it, and how? How were the context files chosen, and are the relevant tables labeled?
Section 4.2 (Main Results)
The headline gains are probably real, at least in direction. But several numbers don't add up, and the comparison setup favors the proposed system.
- W12. The baseline numbers contradict each other. Plain Seed-1.6 scores 23.62 (LLM) and 24.62 (VLM) on ContextSheetBench QA in Table 1 and Figure 3. Appendix A.2 reports 12.23 and 14.39 for the same "vanilla" models, and Table 5 has a vanilla VLM without folding at 40.91. These cannot all be the same setup.
- W13. Agent baselines look under-tuned. On RealHiTBench, SheetAgent, ReAcTable and TableZoomer are all far below plain Seed-1.6 (69.34), even though it is their own backbone.
- W14. I am not sure the retrieval baselines are fair. For Vanilla RAG and DeepSearch, the retriever, chunking and context budget are not described, while Sheet-Raptor uses a retriever trained on about 5M in-domain pairs (see W7). It is also unclear how the agent baselines are given the auxiliary files on ContextSheetBench.
- W15. The closest baselines are missing. ST-Raptor (the most similar prior system), SpreadsheetLLM and TableRAG are all discussed in related work, but none of them appears in the experiments.
Questions for authors
Check Weaknesses