acceptodds
Withdrawn from ICLR 2027

Bind Only Under Pressure: Diagnosing Evidence Identity in Visual PDF Retrieval

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Conjunctive PDF retrieval imposes constraints on evidence identity: clauses may require different pages, whereas table predicates may require the same row. However, aggregate retrieval gains do not isolate the contribution of these constraints. We define binding pressure as a violation of a typed relation by independently optimal evidence assignments. We prove that per-role maxima alone cannot identify relational cost, that maximum-product binding preserves feasible independent assignments, and that the fraction of queries with score changes bounds the aggregate metric change. These results motivate PRESSUREAUDIT, a component-matched evaluation protocol, and PRESSUREBIND, an exact, training-free assignment layer. On n-Clue v1, conditionwise scoring with a frozen 4B encoder improves gold nDCG@10 from .4004 to .4802, but distinct-page binding contributes only +0.000001; 3.43% of candidates exhibit pressure. A fresh, checksum-verified n-Clue v2 evaluation yields first-stage, unary, and bound scores of .3183, .3587, and .3591, with 2.44% candidate pressure. On TCR selection queries, same-row scoring preserves Top-1 and MRR on 113 of 127 queries without positive pressure and improves Top-1 from .429 to .643 on the 14 pressured queries. These findings distinguish improvements in unary evidence processing from relational effects and motivate reporting pressure alongside matched ablations and aggregate performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
visual document retrieval, multimodal retrieval, evidence binding, binding pressure, structured retrieval, PDF retrieval, table retrieval, late interaction, compositional retrieval, retrieval evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jfJnOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether improvements in visual PDF retrieval come from unary evidence processing or from enforcing relations between evidence units. It introduces binding pressure to measure conflicts in independently selected evidence, PressureBind for constrained assignment, and PressureAudit for matched comparisons. Experiments on distinct-page and same-row retrieval show that most aggregate gains come from unary processing.

Critical strengths
  • The paper clearly distinguishes individual evidence retrieval from relational binding in document retrieval. This distinction provides a useful perspective for analyzing retrieval performance and identifying the sources of improvement.

  • The paper evaluates its approach across different retrieval settings and provides controlled comparisons to examine the effects of relational constraints.

Critical weaknesses
  • The proposed methods mainly rely on standard constrained assignment and controlled ablation studies. The theoretical results are relatively straightforward, and the methodological contribution appears limited.
  • The empirical findings are based on a single frozen visual encoder and two benchmarks. Since binding pressure depends on the encoder and retrieved candidates, it remains unclear whether the observed findings generalize to other retrieval settings.
Questions for authors
  • Among the 14 pressured TCR selection queries, how many Top-1 errors are corrected by binding, and how many new errors are introduced?

  • The main comparisons use α=0\alpha = 0 in Eq. (8). Does the near-zero effect of relational binding on n-Clue persist when α>0\alpha>0?

Reviewer_PXsrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether relational constraints in conjunctive visual PDF retrieval—distinct pages for separate conditions, or a shared row for table predicates—actually contribute to retrieval quality, or whether aggregate gains attributed to “structure” are really unary evidence improvements. It formalizes binding pressure as a violation of a typed relation by the independently optimal evidence assignment, defines a cost ρ = 1 − B/U, and proves three results: unary maxima cannot identify relational cost, maximum-product binding is non-intervening when the independent witness is feasible, and the fraction of queries with changed scores bounds the aggregate metric change. It then proposes PressureBind, an exact training-free DP, and PressureAudit, a matched evaluation protocol, and applies them to n-Clue v1/v2 and TCR-Bench.

Critical strengths

S1. The matched unary-vs-bound intervention is a genuinely useful diagnostic: it cleanly shows that most of the reported gain comes from conditionwise unary scoring rather than relational binding, which is a consequential correction to how structure-aware retrieval gains are usually attributed.

S2. The negative result is backed by unusually careful artifact work—checksummed v2 reconstruction, frozen configurations, and exact inference—which makes the null finding credible and gives the community a reusable protocol for pressure-stratified evaluation.

Critical weaknesses

W1. The central distinct-page constraint is a benchmark-construction artifact rather than a validated part of the information need, so the paper may be enforcing a relation the task never asked for, which undermines the main object being measured.

W2. The empirical case for binding is too thin to support the method framing: relational effects are roughly 1e-6 to 3.8e-4 on n-Clue, and the TCR gain comes from 14 queries while the confirmation split shows no change, so the paper reads as a negative result dressed up as a method contribution.

Questions for authors

Q1. Can you show that the distinct-page constraint is part of the query’s information need rather than an artifact of n-Clue’s construction—specifically, how should a document that satisfies both conditions on the same page be judged, and did annotators validate that as non-relevant? If same-page evidence is acceptable, PressureBind may be enforcing a constraint the task never imposed, which undermines the central object of the paper.

Q2. Given that the v2 relational effect is only +0.000377 (p = .125), the 23-query confirmation split shows no change, and gold-only nDCG evaluates the least-pressured candidates, is the method framing justified? Confidence intervals, an equivalence test, and a graded metric including partial matches would show whether this is a genuine null or an underpowered study.

Q3. How do pressure rates change with deeper K, a weaker first stage, and document length, and why is there no n-Clue depth sweep analogous to the TCR one? If pressure rises substantially outside the top-10 setting, the claim that pressure is sparse—and the conclusion drawn from it—would need to be qualified.

Q4. What fraction of hard-pressure cases are exact ties with ρ = 0, and what is the distribution of ρ rather than just its mean? With conditional mean B/U of .9879 on v1 and .9556 on v2, the 3.43% headline may mostly reflect near-ties rather than meaningful identity conflicts.

Q5. Can you provide a tighter result—say, a bound in terms of ρ and rank gaps, or a characterization of when binding can flip a top-k decision? Proposition 3 is vacuous at π_Q = .23 versus an observed 3.8e-4, so it does not currently support the empirical claims.

Q6. Eq. 10 is a soft row-average, not the max-product feasible assignment; can you report a hard same-row variant (∃r ∀k P_k(r)) and show how much of the TCR gain survives? And since TCR uses released CSV cells with no parser-accuracy evaluation, how much of the first-stage→independent gain is actually due to parsing and column grounding rather than visual retrieval?

Q7. Can you add the missing controls—page-level retrieval aggregated to documents, a Hungarian/ILP assignment, arithmetic vs geometric aggregation, and full vs 128-dim embeddings—plus a same-encoder whole-query baseline on v1? Without them, the null relational effect could be an artifact of the aggregation or encoder choice, which directly affects the conclusion.

Q8. Can you report bootstrap confidence intervals over queries/clusters and a perturbation analysis for batch size, rendering, and tie-breaking? Appendix B’s .11–.24 embedding differences are orders of magnitude larger than the reported 1e-6 effect, so the stability of the null is unclear.

Reviewer_jBZWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper

Critical strengths
  1. Isolating relational contributions is an interesting question and worth studying.
  2. The authors design experiments for isolating relational contributions.
  3. Comprehensive experiments and analysis are conducted.
Critical weaknesses
  1. The paper only considers two predefined structural constraints: distinct-page and same-row requirements, which makes it not general enough for more practical problems.
  2. The proposed method is too intuitive without theoretical analysis, making the paper lack theoretical or technique contributions.
  3. Overcomplicated presentation and excessive methodological framing making simple things harder for me to understand.
Questions for authors
  1. What does PressureBind offer beyond standard rule-based filtering or constrained assignment? Is the main contribution actually PressureAudit?
  2. Could you add counterfactual examples that specifically test binding, rather than relying on existing benchmarks with very few binding conflicts?
  3. Would the conclusions still hold with different visual encoders or first-stage retrievers?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.