acceptodds
Under review as a conference paper at ICLR 2027

Beyond Endpoint Sufficiency: Inner-Time Trajectory as a Representation Channel in Iterative Systems

Abstract

Many iterative models treat the final converged state as the representation. We study endpoint insufficiency: convergence can erase task distinctions, and the trajectory can carry reliability information even when the endpoint remains the answer. We formalize representations as , give a contractive construction with coincident endpoints and a label-informative signed-area functional, and introduce audits with strong endpoint, residualization, permutation, and orientation controls. Exploratory KITTI reliability audits use frozen RAFT-Things and SEA-RAFT checkpoints not trained on KITTI. At retained coverage, passive history reduces retained flow error by beyond a learned endpoint-plus-convergence baseline on RAFT-Things; simple convergence reduces it by beyond endpoint plus native uncertainty on SEA-RAFT. Richer geometry adds no resolved utility beyond convergence profiles, and the KITTI-finetuned RAFT checkpoint supplies a negative boundary. Controlled dynamics, looped Transformers, diffusion features, and a residual-null visual case expose distinct mechanisms and limits. These results support endpoint-sufficiency auditing, not universal trajectory access: retain history when it improves the declared target beyond strong inexpensive controls.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
iterative models, endpoint sufficiency, computation trajectories, representation learning, reliability estimation, optical flow

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_UV1DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies if the final hidden state of an iterative model is enough to represent task-relevant information. It shows that computation trajectories can retain information lost at convergence, though their practical benefits are model- and task-dependent.

Critical strengths

The authors provide a rigorous theory analysis showing that distinct trajectories can converge to the same endpoint while preserving information. The experimental evaluation is carefully designed. The experiments also cover several model families and report both positive and negative results.

Critical weaknesses

The theory relies on specific contractive dynamics, and its generalizability to practical models remains unclear. Moreover, the experiments do not fully rule out the possibility that stronger nonlinear endpoint decoders could recover the same information. The practical evaluation is relatively weak,the main reliability improvements demonstrated on a few RAFT-family checkpoints.

Questions for authors

see weakness

Reviewer_ngnUOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies when an iterative model’s final state loses task-relevant information that remains in its computation trajectory. It gives a contractive construction where different inputs reach the same endpoint while their paths stay label-informative. It also proposes audits to check whether trajectory features add real signal: stronger endpoint baselines, residualization, and permutation controls. Experiments show selective benefits: trajectory features reduce retained optical-flow error by 8.1% on RAFT-Things at 80% coverage and improve small looped-Transformer language models, but gains disappear in several stronger-baseline or larger-model settings. The contribution is a diagnostic framework for deciding when retaining history is useful, rather than a universally better architecture.

Critical strengths

1. Careful controls make the framework informative. Strong endpoint baselines, residualization, and permutation controls separate real trajectory information from extra readout capacity or easier decoding. The reported negative results show when history is unnecessary. This makes the claims trustworthy at the level stated.

2. Broad coverage. The audits span optical flow (RAFT on KITTI), image reconstruction, diffusion features (SPair-71k), and looped-Transformer language modeling (WikiText-103) at different scales, plus several recurrent backbones. This suggests the question applies beyond a single architecture.

Critical weaknesses

The audit is post hoc and not exhaustive. The five audits rule out only the alternatives they test. A trajectory readout that beats all of them could still owe its gain to untested factors; residualization, for example, removes only affine endpoint information.

The audit also decides by measured performance, so whether to keep history is known only after training both variants and evaluating them on labeled held-out data. It gives no first-principles criterion for predicting when trajectory access will help. This matters because a diagnostic that requires running the full experiment adds little beyond standard ablation.

Questions for authors
  1. On RAFT-Things, the convergence baseline B already accounts for most of the gain over the endpoint E (AP 0.770 to 0.816), and full history adds only +0.019 AP (Table 20). B resembles existing convergence-based uncertainty [1]. Could feature attribution on the trained B+H classifier identify which history information goes beyond B? This would show whether history can improve on existing convergence-based uncertainty scores.

  2. With all the experiments, could the authors summarize generalizable insights about when simple aggregate statistics are enough and when richer, ordered history is needed?

[1] N. Durasov, D. Oner, J. Donier, H. Le, and P. Fua. Enabling Uncertainty Estimation in Iterative Neural Networks. ICML, 2024.

Reviewer_ss2LOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks when the final state of an iterative model is insufficient for predicting either an answer or the reliability of that answer. It gives a contractive rotation construction whose two classes can have the same finite-horizon endpoint but different path chirality, and proposes endpoint-sufficiency audits using stronger endpoint readouts, residualization, permutation, and shortcut controls. Experiments span controlled recurrent systems, optical-flow failure prediction, looped language models, and diffusion-feature correspondence. The authors report both positive and null outcomes and frame the contribution as a diagnostic protocol rather than a universal trajectory method.

I lean "weak acceptnce" because the exact endpoint-collision witness, controlled audits, and selective-reliability results together provide useful knowledge about when computation history can add value, while the explicit negative results keep the claim appropriately bounded. This remains a close decision: practical evidence is localized and exploratory, and the increment over convergence-based uncertainty needs sharper articulation. Independent confirmation and a clearer account of the reusable audit procedure would increase the whole confidence.

Critical strengths
  1. The question is relevant to iterative representations, and the paper carefully distinguishes answer information from reliability information. The exact-collision construction is a valid and clear witness under its stated conditions.

  2. The controlled studies include useful endpoint, shuffled-history, residual, and orientation controls. The learned-collision and path-functional cases provide evidence beyond the analytic example.

  3. The practical optical-flow audit uses pair-clustered uncertainty intervals and discloses failed comparisons. On RAFT-Things, adding passive history to a combined endpoint/convergence baseline improves AP from 0.8157 to 0.8345 and lowers retained EPE at 80% coverage from 1.0004 to 0.9196 pixels.

  4. The 10-seed small WikiText-103 comparison includes an endpoint-repeat control that executes the same fusion head. The paper is unusually explicit about transductive fitting, weak controls, cache provenance, and negative boundaries.

Critical weaknesses
  1. The practical positive results have a narrow scope. The main reliability test uses one previously inspected KITTI-2015 pair split and two non-KITTI-trained checkpoints from the RAFT family. SEA-RAFT full-history gains over its combined baseline are unresolved, and the KITTI-finetuned RAFT checkpoint is a negative boundary. The larger WikiText run also shows no resolved advantage over plain endpoint, although width and training budget change together. These findings support conditional utility. The concern is whether the positive evidence establishes sufficiently generalizable new knowledge, rather than the absence of a universal advantage, which the authors do not claim.

  2. The relationship to prior convergence-based uncertainty needs sharper positioning Durasov et al. (2024) already show that iterative convergence can predict error. This paper’s added value lies in the audit protocol and evidence beyond combined controls, rather than in the general observation that histories carry reliability signal. The breadth of settings makes that incremental contribution harder to assess. In Appendix A.13, convergence profiles alone already account for the useful RAFT history result, while richer geometry adds no resolved benefit. A more explicit, reusable audit procedure and a clear account of the decisions or new findings it enables would sharpen the contribution.

Questions for authors
  1. Can you repeat the main reliability audit on previously uninspected, scene-disjoint data or an additional architecture, with a prespecified split and feature set and repeated probe fits?

  2. Beyond existing convergence-based uncertainty, what reusable audit steps and new decisions or findings does the protocol enable?

Reviewer_MbQvOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors present a suite of results pertaining to one central question: in models that iteratively modify an internal representation, does the set of all intermediate representations provide useful information absent from the final representation. They suggested that information may be useful for either classifying points, or for measuring the uncertainty in decisions.

In their first experiment, they tested to see if trajectory information on RAFT-Things, SEA-RAFT, and RAFT-KITTI could improve accuracy. They found a good improvement on RAFT-Things, but minimal improvement on the other two.

They then showed experiments run on synthetic data to show that classifiers can successfully separate trajectories with similar endpoints, including those taken from intermediate model states for different architectures, and also extract information from the trajectory unrelated to the learning objective.

They also showed that they were unable to remove useful information from the trajectory by subtracting off information derived from the final converged point using affine transformations.

Finally, on the pair71k dataset, they found that one time step generally provides as much information as the entire trajectory, while on WikiText-103, they found that trajectory information is helpful in smaller models, but not larger ones.

Critical strengths

This topic is well motivated and potentially very useful:

  • It is very plausible that in real models, certain data points could have different trajectories with similar final states, thus lowering accuracy
  • Trajectory information is easy to access, and incorporating it into models could potentially improve model performance.
  • For these reasons, I think this topic is worthy of exploration.
Critical weaknesses

Experimental results could be stronger:

  • Synthetic data was heavily used, and the strongest results generally came from synthetic data.
  • Experiments that use real world data are significantly less promising. In the case of the RAFT experiments, very little benefit was shown, while the experiments using SPair and WikiText showed significant limitations.
  • Different datasets are used to show different results, impairing comparability.
  • (sec. 6) An affine transformation seems insufficient to accomplish the task at hand.
  • Thus, I am not sure that this result is strong enough for publication at this time.

Presentation is difficult to follow:

  • Sentences are very dense and difficult to comprehend.
  • Sometimes, long chunks of text could be used to convey an idea that could be conveyed much more concisely; proposition 1 could be replaced by a few sentence summary.
  • abstract does not adequately summarize the findings, related work section does not expand much on prior work.
  • Thus, I believe that the paper needs significant writing improvements in presentation before it is ready for publication.
Questions for authors
  • Despite the limitations shown on the KITTI, SPair, and WikiText experiments, how do you propose applying your findings to the training of new models, and how do you see this being useful?
  • In general, do you think that synthetic data reflects applicable real-world use-cases?
  • In section 6, why do you believe that affine transformations are sufficient?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.