acceptodds
Under review as a conference paper at ICLR 2027

How Should Geometry Guide Latent Reasoning? MeRa for Spatial Prediction

Abstract

Post-encoder refinement revisits an encoded history before prediction, but additional depth alone does not specify which relations guide that retrieval. For spatial tasks, this raises a specific question: how should iterative refinement use the geometry of the input? We study this question through MeRa (Metric-space Reasoning), a modular post-encoder interface that combines distance-conditioned cross-attention, fixed encoder memory, and gated residual updates. The original experimental record reports higher NDCG@10 with MeRa in six backbone–dataset settings, with relative gains of 2.5–13.9% for GETNext and 5.8–27.7% for LSTM. On NYC GETNext, the reported gap between refinement without and with distance bias is 4.5%. We characterize finite-depth sensitivity and fixed-memory information access, clarifying what the architecture guarantees and what must be assessed empirically. Depth comparisons and a CLEVR-derived nearest-object task provide complementary observations. The results motivate geometry-conditioned refinement as a design choice; unresolved historical run alignment and evaluation provenance limit causal and cross-protocol performance claims. Code is available at https://anonymous.4open.science/r/MeRaa.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Latent Reasoning, Spatial Prediction, Metric-Space Bias, Sequential Recommendation, Representation Refinement

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eK66OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes MeRa, which transforms real-world spatial distances into attention biases through an MLP and incorporates them into multi-step latent refinement. Experiments on spatial recommendation tasks show that introducing distance biases improves prediction performance. The authors also provide ablation studies and theoretical analyses.

Critical strengths

The method is simple and easy to integrate with different backbones, with reported performance improvements across multiple datasets.

Critical weaknesses
  1. Limited novelty.

Incorporating spatial distances into attention has been extensively studied. This paper mainly applies this established mechanism to multi-step latent refinement, with the overall architecture remaining a straightforward combination of distance-biased attention, FFNs, and gated residual updates. The paper lacks sufficient comparisons with alternative ways of incorporating spatial information and fails to demonstrate any distinctive synergy between distance conditioning and multi-step reasoning. Overall, I find the methodological novelty insufficient for ICLR.

  1. Verbose writing and unclear focus.

The paper contains excessive repetitive qualifications and defensive discussions that obscure its core method and contributions. The presentation could be substantially more concise, and Figure 1 lacks clarity and intuitiveness.

  1. Limited value of the theoretical analysis.

The finite-depth displacement and sensitivity bounds in Theorem 1 follow directly from standard norm inequalities and Lipschitz assumptions. The fixed-memory access property in Theorem 2 is also an immediate consequence of the architecture. Neither theorem reveals any distinctive property of geometry-conditioned refinement or explains the observed performance gains. Consequently, the theoretical analysis provides little substantive support for the paper’s central claims.

  1. Serious concerns about experimental reliability.

Appendices E and F acknowledge severe misalignment between POI indices and spatial coordinates in the available code, while the original results cannot be reliably linked to specific code versions, data splits, or checkpoints. More importantly, subsequent experiments using a different implementation and evaluation protocol failed to consistently reproduce the originally reported improvements. Although these issues do not establish that the original experiments were incorrect, the current evidence is insufficient to reliably support the paper’s main empirical claims.

Questions for authors

What unique advantages does multi-step refinement offer over a single distance-aware attention layer? Can the authors provide experiments demonstrating a synergistic effect between distance conditioning and iterative refinement?

Reviewer_X9f7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MeRa, a post-encoder refinement module for spatial prediction that repeatedly reads fixed encoder memory through distance-conditioned cross-attention and gated residual updates. It provides finite-depth displacement and sensitivity bounds and a fixed-memory access characterization. The empirical section preserves original results across two backbones and three next-POI datasets, distance-bias and depth comparisons, and a CLEVR-derived nearest-object task. The original record reports single-seed validation NDCG@10 gains in all six POI settings, but unresolved run configurations, input alignment, and evaluation provenance preclude treating these gains as verified evidence of the proposed mechanism’s effectiveness. A later implementation evaluated under a different test protocol reports mean NDCG@10 below its encoder baseline on all three datasets, providing no corroborating evidence for the gains, although it is not a matched-protocol replication.

Critical strengths
  1. A clear, modular formulation of geometry-conditioned refinement. MeRa explicitly separates fixed encoder memory, distance-conditioned retrieval, and gated residual updates in a post-encoder interface (§3.2–3.4, Algorithm 1), while retaining the existing prediction heads and training objective (Appendix E). This separation is valuable because it makes the placement and role of spatial conditioning explicit and provides distinct design axes for further controlled study. The interface accommodates different compatible encoders, although its empirical effectiveness across backbones remains subject to the reported provenance limitations.

  2. Explicit characterization of architectural properties and their limits. The analysis provides conditional finite-depth displacement and sensitivity bounds and characterizes access to fixed encoder memory (§3.5; Appendix B). It explicitly distinguishes these properties from convergence and dynamic message passing. The depth analysis also recognizes that an untied stack changes parameter count and does not establish test-time scaling of a fixed checkpoint (§4.4). These distinctions strengthen the assessment by clarifying what follows from the architecture and what still requires empirical evidence, particularly regarding multi-hop reasoning.

Critical weaknesses
  1. The empirical attribution to meaningful geometry is not established. Appendix E reports that the available loader can misalign POI identifiers and coordinates, with current-file checks finding mismatches for nearly all entries. These checks do not establish that the historical experiments were affected, but the original mappings and input hashes are unavailable. Consequently, the reported gains cannot currently be linked to verified geographic distances. This leaves a substantial gap in the evidence that meaningful spatial conditioning, rather than another property of the added module, drives the improvements.

  2. The reported predictive gains lack a fully traceable and robust validation. The original results are single-seed validation estimates, and Appendix F states that the table rows have not been bound to exact code, split files, checkpoint choices, or POI mappings. Appendix E further notes that the located validation logs do not recover every reported row under a best-NDCG@10 selection rule. Later experiments use a different implementation and protocol and do not provide a general positive replication; they therefore neither directly refute nor corroborate the original comparisons. Although the paper explicitly acknowledges these limitations, they leave the practical benefit of MeRa insufficiently established. A traceable, matched comparison across multiple seeds on a held-out test split would materially strengthen this evidence.

Questions for authors
  1. Clarification: Can the reported gains be linked to runs with verified geographic alignment? Appendix E identifies potential mismatches between POI identifiers and coordinate rows, while noting that their effect on the historical runs is unknown. Can you provide the mappings, input hashes, and code revisions used for the principal results, or other evidence that those runs used correctly aligned distances? This is central to whether the improvements can be attributed to meaningful geometry conditioning.
  2. Clarification: How closely matched were the distance-bias ablations? Can the historical records establish that the baseline, refinement without bias, and MeRa conditions used the same data splits, valid-position masks, training budget, and checkpoint-selection rule? In particular, which configuration produced each reported row, given the differences between the original specifications and the available implementation? This would determine how strongly the ablations support the proposed mechanism rather than implementation or selection differences.
  3. Clarification: What explains the different outcome of the later experiments? Appendix E reports that a later implementation under a different protocol did not reproduce a general advantage for three-step geographic refinement. Which implementation and protocol changes distinguish those experiments from the original results, and is there evidence identifying which changes account for the discrepancy? This materially affects whether I view the favorable historical pattern as robust or configuration-dependent.
Reviewer_1uZxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

MeRa adds a refinement module between a sequence encoder and its prediction heads. Each block reads fixed encoder memory using the current states as queries. Pairwise distances provide attention biases, and a gate controls the residual update. The paper gives finite-depth sensitivity and memory-access analyses. It reports results on three next-POI datasets with GETNext and LSTM, together with a CLEVR-derived nearest-object task.

Critical strengths
  • The question is worthwhile: how should a predictor use spatial relations when it revisits an encoded trajectory? For next-POI prediction, this is more specific and useful than simply asking whether extra layers improve accuracy.

  • The useful design choice is to put distance-conditioned retrieval in a separate post-encoder stage while keeping the prediction heads and objective unchanged. Fixed memory, evolving queries, and gated updates are clearly separated. This makes the method easy to formulate and gives a concrete set of choices that can be tested.

Critical weaknesses
  • The paper does include useful comparisons: two backbones on three datasets, no-bias variants, a depth study, and a deeper Transformer control. However, the original results are single-seed validation scores under an 80/20 split. Appendix E states that the original mappings and input hashes are unavailable and that the located logs do not reproduce every reported row under the stated selection rule. The later test results do not provide a general positive replication, but their different protocol also prevents a controlled comparison with the original results. Therefore, I cannot conclude that the reported gains are reliable effects of correctly aligned geometry. This affects the main results and the ablations together

  • Figure 1 is poorly designed, containing numerous sub-regions and excessive text descriptions with fonts too small to read. The main image appears to be entirely AI-generated, which does not meet the quality standards expected of an ICLR paper from a reviewer's perspective.

  • Table 7 reports 1.20M added parameters for three blocks, whereas Appendix C gives 4,317,126 for the available width-320 reference. The original width-128/dropout-0.1 specification also differs from the available width-320/dropout-0.3 configuration. Scores, parameter counts, and timing need to refer to identified configurations.

Questions for authors
  • What is the source of the GETNext baseline in Table 3, given that all 12 values match GTR-Mamba Table 2 despite the different stated splits? Were these reused reference values or independently obtained results?

  • What advantage is expected specifically from distance-conditioned reads of fixed memory after the encoder, rather than from distance-aware attention more generally? Do any existing results compare bias placement, fixed versus updated memory, or gated versus ungated updates under comparable budgets? These comparisons would help distinguish MeRa from the related mechanisms discussed above.

  • Which candidate coordinates and identities are visible at inference, and how are same-category objects distinguished? Can the reported targets be tied to a converter that keeps object IDs, coordinates, and labels aligned? If all candidate coordinates are visible, what does MeRa improve over a direct nearest-distance rule?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.