acceptodds
Withdrawn from ICLR 2027

DeltaBind: Relation-Bound Retrieval for Visually Rendered Tables

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual table retrieval can fail when candidates contain the same words but associate values with different rows or columns. Page-wide late interaction can combine query-token matches from incompatible relations, a failure we call relation collision. We introduce DeltaBind, a training-free framework that makes a complete table relation the unit of query matching. Using predicted table structure, DeltaBind assembles source-image crops of a value, its row label, and its column header into a relation tile. A frozen retriever scores each tile with late interaction and ranks a table by its highest-scoring tile, keeping query-token matches within a shared relation. For aligned table versions, shared content can obscure the relations that distinguish candidates. A query-independent change index therefore focuses comparison on changed relations and enables sparse caching. DeltaBind improves ColQwen2's strict pair accuracy from 51.6% to 79.7% on TabFact-HumanBind. Relation binding alone also improves source-local retrieval on mined pools of unedited FinTabNet and PubTables-1M tables. OCR-BM25 controls and component ablations support the value of jointly preserving row, column, and value context.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
visual table retrieval, multimodal retrieval, relation binding, visual document retrieval, late interaction, table understanding, structured evidence

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_H9wqOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses relation collision in visual table retrieval. The problem arises when page-wide late interaction matches query tokens to values and labels from different rows or columns. The proposed method, DeltaBind, uses predicted table structure to crop each value together with its row label and column header into a relation tile, scores each tile with a frozen retriever, and ranks each table by its best tile. DeltaBind improves ColQwen2’s pair accuracy.

Critical strengths
  • The problem is well motivated.
  • The authors construct datasets that isolate the problem.
Critical weaknesses
  • The evaluated task is narrower than the claimed retrieval problem, which leaves the target community unclear. The title and abstract present the work as visual table retrieval, but what is actually evaluated is reranking within a candidate group given in advance. The candidates are 2–8 tables from the same source, and which tables form a group, and whether they are aligned, is decided by the authors rather than the system. The paper cites a large-scale benchmark that treats tables as images [1], on which the method could have been evaluated as a reranker on top of a first-stage retriever, but this was not done. As a result, it is unclear which community the paper aims to contribute to. It is framed as visual document retrieval, but its evaluation draws on claim verification datasets (TabFact, SciClaimEval), and the aligned setting resembles document version comparison. Each of these communities would expect different baselines and benchmarks, and the paper does not position itself against any of them.

  • The motivating setting is not evaluated. The introduction motivates the problem with revisions in which only a few cells change, so that shared content obscures the difference between versions. This problem should become more severe as tables grow larger and the fraction of changed cells shrinks, yet no evaluation covers this condition. HumanBind excludes tables with more than 12 rows or 8 columns, and on SciClaimEval, the only aligned setting not built from synthetic swaps, DeltaBind falls below the flat baseline. The core motivation therefore remains unsupported.

  • The failure to distinguish cases with the same words but different bindings has already been reported for VLMs [2, 3]. For tables, there are existing approaches that classify rows and columns separately and locate relevant cells at their intersection [4], and approaches that use a row combined with its headers as the retrieval unit [5]. The method itself is largely a combination of existing techniques. Splitting a table into relation units and taking the maximum score among them has the same structure as scoring a document by its best passage score [6]. Within each unit, standard late interaction [7] is used as is, and change filtering is classical image differencing. Since the components are not new, the contribution would need to rest on showing why their combination works, but the experiments do not isolate where the gains come from.


[1] D. Li et al., “Beyond Text-Only: Towards Multimodal Table Retrieval in Open-World,” ICLR 2026.

[2] T. Thrush et al., “Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality,” CVPR 2022.

[3] M. Yuksekgonul et al., “When and Why Vision-Language Models Behave like Bags-of-Words, and What to Do About It?,” ICLR 2023.

[4] M. Glass et al., “Capturing Row and Column Semantics in Transformer Based Question Answering over Tables,” NAACL 2021.

[5] W. Chen et al., “Open Question Answering over Tables and Text,” ICLR 2021.

[6] Z. Dai and J. Callan, “Deeper Text Understanding for IR with Contextual Neural Language Modeling,” SIGIR 2019.

[7] O. Khattab and M. Zaharia, “ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT,” SIGIR 2020.

Questions for authors
  • In a deployed system, who determines the candidate groups and whether a group is aligned? If this requires an oracle, the practical scope of the method changes considerably.
  • How much of the gain comes from restricting matches to a single relation, as opposed to cropping and enlarging cells and adding role labels? For example, what happens if the original page-level patch embeddings are pooled within each predicted cell box and the shared-witness rule is applied directly, without cropping or enlarging?
Reviewer_tswvOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies relation collision in visual table retrieval, where query tokens may match different rows or columns even though no single relation supports the query. DeltaBind constructs relation tiles containing a value, its row label, and its column header, and scores them using a frozen late-interaction retriever. For aligned table versions, it additionally uses pixel differences to identify changed relations and reduce the number of stored tiles. The method is evaluated on a controlled value-swap benchmark and two mined retrieval datasets based on FinTabNet and PubTables-1M.

Critical strengths

The relation collision problem is reasonable, and matching query tokens within complete table relations is an intuitive solution. The controlled value-swap benchmark is useful because it preserves page-level word content while changing the associations between values and labels. The improvement from 51.6% to 79.7% is encouraging.

The ablation in Table 4 provides some evidence for the relation-tile design. Including both row and column context performs substantially better than using either component alone. The OCR-BM25 experiments also suggest that the benefit of relation binding is not specific to a particular visual retriever.

Critical weaknesses

My main concern is whether the experimental results and manuscript were adequately checked against the actual implementation. Appendix C.3 reports precise encoding and scoring times, but then states that “the original hardware model and actual batch size are not recoverable from the available records.” I find this hard to accept in a paper that reports runtime measurements in support of its sparse-caching claim. Hardware specifications and batch size are basic information needed to reproduce timing experiments. The same appendix also acknowledges that the full and sparse CPU loops perform different workloads: the full loop includes page-baseline scoring, online change detection, and additional aggregation operations, while the sparse loop excludes them. The reported 11.60s vs. 0.144s comparison therefore does not establish a comparable runtime speedup. The 5.8% storage ratio is a query-independent count of how many tiles survive offline filtering; it is a property of the constructed index rather than an experimental result, and it does not establish an end-to-end efficiency improvement.

Other implementation details add to this concern. Appendix B states that the natural retrieval script divides the tile score by query length an additional time and documents tie-breaking behavior that the implementation does not explicitly guarantee. The extra normalization should not affect within-query rankings, as the authors explain, but these discrepancies between the mathematical description and the implementation should have been resolved rather than documented. Together with the missing experimental environment information, they raise questions about how thoroughly the final results were verified. This is difficult to reconcile with the AI Use Statement’s claim that AI-assisted content was reviewed for accuracy and consistency.

The evaluation does not adequately isolate the contribution of change filtering. On TabFact-HumanBind, the main result compares full-page retrieval against relation binding combined with change filtering. There is no corresponding visual relation-binding baseline without filtering, although the OCR experiments include this comparison. The two natural benchmarks use relation binding alone, so they provide no evidence for the change-filtering component. On SciClaimEval, the complete method actually performs worse than the flat baseline (54.0% versus 56.3%). It is therefore unclear whether change filtering offers a useful advantage outside the synthetic swap setting.

The natural evaluations are also limited in scale. FinTabNet contains only 41 queries from 14 reports, and PubTables-1M contains 71 queries from 38 papers. The candidate pools consist of mined same-source tables rather than a larger retrieval collection. Some reported improvements are not statistically significant, including ColSmol and OCR-BM25 on FinTabNet (p = .383 and p = .115, respectively). In addition, the structure predictor was pretrained on PubTables-1M and FinTabNet.c, which are the sources of the natural evaluation tables. The authors acknowledge this overlap but do not quantify its effect on the predicted-structure results, so it is unclear how much of the reported gain depends on near-oracle structure prediction.

I also found the paper harder to follow than the underlying idea warrants. The sentences are generally readable, but the actual retrieval pipeline and evaluation protocols are scattered across Sections 3–4 and Appendices A–B. Section 3.1 develops a theoretical argument over fixed evidence vectors, whereas the implementation uses separately encoded relation tiles. The distinction is acknowledged, but its relevance to the experimental results is not explained very clearly. Figure 2 is also quite dense. A more direct description of the complete pipeline and a clear breakdown of which components are evaluated in each setting would have made the paper easier to assess.

Questions for authors
  1. Why are the hardware model and batch size unavailable for the timing experiments in Appendix C.3? Can the authors provide the experimental environment and a timing comparison in which the full and sparse loops perform the same workload?

  2. What is the performance of visual relation binding without change filtering on TabFact-HumanBind? Can the authors show that change filtering improves retrieval on any naturally occurring aligned table versions, rather than only on synthetic value swaps?

  3. What were the train and calibration splits used for in a training-free method? Were the thresholds (ε, λ, the pixel threshold), tile dimensions, and the top-K budget selected on data from the same sources as the sealed sets? Since the TATR checkpoint was trained on PubTables-1M and FinTabNet.c, can the authors either report results with a structure predictor not trained on these sources, or quantify how often structure errors change the selected tile on the natural sets?

  4. Table 7 reports exactly 45.7% accuracy for three different aggregation strategies. Do these rules produce identical predictions, or only the same number of correct ones? Please report the per-rule tie counts, since zero differences are scored as errors.

  5. Will the authors release the code, the TabFact-HumanBind construction scripts, and the mined candidate manifests?

Reviewer_4WqnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates relation collision in visual table retrieval, where page-level late interaction can combine query-token matches from incompatible rows or columns. To address this issue, DeltaBind constructs relation tiles by combining source-image crops of a value, its row label, and its column header. A frozen retriever scores each tile, and the highest tile score determines the score of the candidate table. For aligned table versions, a query-independent pixel-change index restricts comparison to modified relations and enables sparse caching. On TabFact-HumanBind, relation binding combined with change filtering improves ColQwen2’s accuracy from 51.6% to 79.7%. Relation binding alone increases source-local R@1 from 46.3% to 80.5% on FinTabNet and from 49.3% to 85.9% on PubTables-1M. Experiments with ColSmol and OCR-BM25, together with component ablations, provide further supporting evidence. However, accuracy on SciClaimEval decreases from 56.3% to 54.0%, suggesting limited effectiveness for claims that require combining evidence across multiple relations.

Critical strengths
  1. The Experiments on unedited FinTabNet and PubTables-1M tables complement the controlled benchmark. These results show that, within the evaluated candidate pools, relation binding also helps retrieval handle naturally occurring differences in table content and layout.
  2. The binding-gap analysis clarifies the difference between independent token matching and matching within a shared witness. The top-K analysis also establishes when truncation preserves the winning candidate, making the behavior and assumptions of the aggregation rule easier to understand.
  3. Detailed experimental setting. The paper specifies dataset construction, checkpoints, thresholds, tie handling, and statistical tests. These details make the reported results easier to verify and provide a useful foundation for reproducing the experiments.
Critical weaknesses
  1. Some baseline comparisons are unfair. Whole-row crops and structured text representations using the same predicted structure would help establish the value of the proposed tile design. In addition, Qwen reranks only the top two visual results, whereas DeltaBind scores the full candidate pool. This difference limits the interpretation of their relative performance, particularly on FinTabNet.
  2. The method has limited support for queries requiring multiple relations. Selecting the highest-scoring individual tile does not ensure that all conditions in a complex query are jointly supported. The decrease on SciClaimEval is consistent with this limitation. A clearer characterization of suitable query types would help define when the method can be applied reliably.
  3. The contribution of relation binding is not fully isolated. The theoretical analysis compares aggregation over fixed evidence vectors, whereas the implementation also crops, enlarges, rearranges, and re-encodes the content. The observed gains may therefore partly reflect improved visibility or reduced distraction. Controls with matched layouts and shuffled bindings would help establish the importance of correct relational associations.
Questions for authors
  1. With crop sizes, layout, and resolution held constant, how much does performance fall when row or column labels are reassigned to incorrect values? Can whole-row crops or a simple row–column–value text serialization using the same predicted structure achieve comparable results?
  2. How does DeltaBind compare with ColQwen2 in computational efficiency when evaluated on the same hardware and candidate pool?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.