- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper presents SpatialDataAgent, an LLM coding agent that turns public GEO spatial transcriptomics deposits into paired histology, coordinate and expression data. A screening module first classifies studies by the evidence for H&E pairing. A reconstruction agent then writes and runs study-specific code using 26 domain tools and an optional visual QC checker. On 40 GEO studies, the domain tools raise human-reviewed alignment from 54.7% to 77.6%, and to 81.0% with QC.
Critical strengths
- The problem is a real practical bottleneck for anyone working with public spatial data. The released HESRT collection is considerably larger than HEST-1k and STimage-1K4M, so it could be useful to the community.
- The controlled comparison is set up carefully. The model, inputs and budgets are the same across conditions, failed and missing outputs are kept in the denominator.
Critical weaknesses
-
My main concern is the contribution from an ML perspective. The system is an existing LLM coding agent (a Codex-based scaffold running Muse Spark 1.2) with 26 hand-written domain tools and a rule-based image checker. The main finding is that giving the agent domain-specific tools raises the pass rate, which is expected. The gain is also concentrated where the tools encode the file format: standard Visium goes from 60% to 99%, while containers and alternative arrays stay at around 50% (Fig. 2d). I could not find a new agent method, or a lesson that would transfer beyond this specific curation task. Also, the value of HESRT for ML is not shown. There is no downstream experiment, and Fig. 3b compares the resources only by the number of samples that pass four basic geometric checks (unique IDs, finite coordinates, unique centers, centers inside the image), which says little about alignment quality.
-
The headline results also overstate how well the data integrity claims are verified. The 81.0% in the abstract is a human visual PASS on overlays, which cannot catch small offsets, scale errors or altered expression values. However, the automated source audit, which checks exactly what the paper claims to preserve, passes only 133 of 331 samples for the full system. Depending on how unresolved cases are counted, the pass rate lies between 40% and 77% (App. C, D), and none of these numbers appear in the main text. Moreover, QC is presented as one of the three design principles, but its effect is not significant in either comparison (Table 2). This is despite Sec. 5.2 being titled "Domain Tools and Quality Checks Drive Successful Reconstruction". The screening reference was built with AI cross-review and sampled using the same model's earlier predictions (Sec. 4.1), so the 91.75% agreement is not fully independent either.
Besides, every released sample still went through human review (App. E), so "autonomous" seems too strong a description of the curation pipeline as a whole.
Questions for authors
- Given the paper's emphasis on preserving deposited measurements, why is the main evaluation based on visual PASS rather than on the source audit? How should readers interpret the gap between 81.0% (visual) and 133/331 (source audit) for the full system?
- How exactly were the 400 reference labels produced, and in how many cases did the AI cross-review change the initial human label?