acceptodds
Desk rejected at ICLR 2027

When Correct Predictions Require Incorrect States: A Compatibility Audit of Physical State Interfaces

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurate predictions can require a state estimate outside its physical tolerance. We audit this conflict when several prediction queries share one state. A task-passing input shows that the queries can be satisfied. Ruling out task success throughout the allowed state neighborhood then certifies necessary distortion for every successful readout using that interface. On RoboMNIST, learned readouts pass the tasks on windows certified to require distortion, including frozen tests at other speeds and on another robot. The conflict persists with nonlinear heads that reduce MSE at the reference velocity by 28–35%. We then evaluate acceleration augmentation on 1,440 further windows. Relative to ordinary constant acceleration, mean full joint passing rises by 1.11–2.78 percentage points in three settings and falls by .28 points at medium speed, under the declared .5 m/s² acceleration tolerance. For released robot models, we certify which distance relations cannot be satisfied anywhere in the allowed joint neighborhoods. We then check the calibrated models at the same recorded inputs. At .01-degree joint and 1 mm distance tolerances, MUKCa’s published calibrated models recover 98–100% of certified incompatible pairs on FR3 and two Panda robots, with weaker recovery on Kinova. The audit identifies conflicts that a readout change cannot resolve and tests whether changing the interface removes them.

Primary area:
applications to robotics, autonomy, planning
Keywords:
task-based learning, state representation, physical state interfaces, certification, neural network verification, robot learning, robot calibration, robotics

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_B8CpOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper starts from the observation that a robot can predict future motion accurately while describing its own motion incorrectly. For example, a robot may estimate its speed and direction, then predict future positions using a rule that ignores acceleration. In this situation, a less accurate estimate of physical motion can compensate for limitations in the prediction rule. Improving predictions therefore does not necessarily improve the accuracy of the underlying physical description.

The authors ask when this conflict is unavoidable under a given prediction rule and specified error limits. They require one shared physical estimate to support several predictions. They distinguish three possibilities: an estimate can satisfy all requirements; predictions can succeed only with a physically unacceptable estimate; or no estimate can satisfy all predictions simultaneously. To establish the second case, they find an estimate that satisfies every prediction requirement and use mathematical bounds to rule out every physically acceptable estimate. In such cases, improving motion estimation alone cannot solve the conflict.

Experiments with recorded robot movements (RoboMNIST) show that systems trained to improve future predictions sometimes succeed by producing estimates whose physical errors necessarily exceed the allowed limits (18 of 190 such windows for the affine prediction rule). The conflict persists even with more flexible prediction rules that reduce average prediction error.

The authors then test changes to the model itself. They add acceleration and adjust the prediction rule to account for how motion was measured over a recent time interval. This restores simultaneous accuracy of physical estimates and predictions in many previously impossible cases. They also audit robot calibrations published by other authors (MUKCa, and a CTU dataset with laser-tracker measurements), showing that the calibrated models can predict known distances accurately using unchanged recorded joint angles, where the original models could not succeed within the allowed angle adjustments.

The repairs are imperfect: some previously successful cases become unsuccessful, and the conclusions depend on the chosen error limits and assumptions about measurement uncertainty.

Critical strengths

The paper's main strength is its rigorous distinction between inaccurate physical estimates and conflicts built into the prediction rule itself. It can establish when every estimate that satisfies the prediction requirements must exceed the allowed physical error. This shows precisely when retraining the estimation component alone cannot resolve the problem while the prediction rule and requirements remain fixed. Another strength is evaluating changes on the same individual cases: the authors distinguish whether a satisfactory solution becomes possible from whether the tested system actually produces one, while also reporting previously successful cases that deteriorate. These checks reveal trade-offs that average prediction errors can conceal. Applying the approach to both learned motion prediction and robot calibration gives the diagnosis concrete meaning through explicit physical quantities and error limits. Finally, the certificates are verified carefully (independent high-precision recomputation, pre-registered evaluation protocol for the acceleration study), and the paper is transparent about which analyses were added retrospectively.

Critical weaknesses
  1. The paper's main weakness is the limited breadth of its learned prediction experiments. These focus on two test activities, two robots of the same type, short prediction horizons (0.1 and 0.2 s), and relatively small models. Although the calibration studies provide additional evidence, it remains unclear how frequently the diagnosed conflicts occur in more complex robotic tasks. The RoboMNIST experiments also use recorded motion feedback without evaluating estimators based on video, leaving their relevance to systems combining several sensors insufficiently tested.

  2. The tolerances are not independently justified, and the conclusions depend strongly on them. The benefits of the proposed acceleration extension depend on the chosen error limits: tightening the acceleration requirement from 0.5 to 0.25 m/s^2 reverses its advantage over the data/budget-matched comparison model in all four settings (Appendix D.1, Table 13), and this matched control was itself added after inspecting the confirmation results. The relative gain over ordinary constant acceleration is also small (1.11 to 2.78 points, and negative at medium speed). The same issue is more pronounced in the calibration audit: a joint tolerance of 0.01 degrees appears very small relative to typical joint-level errors on such arms (encoder offsets, elasticity, backlash), and the certified number of D cases depends heavily on it (for CTU, 99 old D poses at 0.01 degrees but only 21 at 0.05 degrees, with zero recovered; Table 19). With such tight tolerances, certifying "necessary distortion" for an uncalibrated model is close to restating that the model has kinematic errors larger than 0.01 degrees, which is the reason for calibrating it in the first place.

  3. The main learned-readout result is fragile under reference uncertainty. Of the 18 certified D windows where the task-trained readout succeeds, only 4 remain certified under an assumed 1 mm position error of the reference trajectory, and none under 1 mm plus a 2 ms timing error (Table 9). The paper notes that no per-point accuracy bound exists for the RoboMNIST references (Appendix A), so it is unclear whether the realized distortion reflects the robot's motion or errors in the reference itself. Similarly, in the repair study all paired certificates vanish at 2 mm (Table 14), and at 1 mm the residual head falls behind the matched control in every setting (Appendix D.2).

  4. The calibration audit evaluates models that the authors did not fit, on records whose role in the original model development is unknown (Section 5.1: "The available metadata leave the records' use during other stages of original model development unknown"). The 98 to 100% recovery rates on FR3 and Panda may therefore be partly in-sample. On Kinova, the calibrated model recovers 211 of 1,019 D pairs while losing 107 previously passing pairs, which deserves more discussion than it receives.

  5. The paper is very dense and introduces a lot of its own terminology (readout, interface, contract, semantic ball, D-task success, D to J), which makes it hard to follow without repeatedly consulting the appendices. A single worked example carried through the paper (e.g., the constant-velocity example from Appendix B.2) would help considerably.

Questions for authors

The title says "When Correct Predictions Require Incorrect States".

  1. What general lessons can a model designer draw from your experiments about when accurate predictions require inaccurate physical estimates, and which model changes can resolve this conflict? Which observed patterns support these lessons, and under what conditions would you expect them to apply to a new task?

  2. How were the tolerances (0.1 m/s, 20 mm, 0.01 degrees, 1 mm) chosen? Is there an application or hardware argument for them (e.g., encoder resolution for the 0.01 degree joint tolerance), and how would the main conclusions change under tolerances matched to realistic measurement errors?

  3. Given that only 4 of the 18 D successes survive a 1 mm position error assumption, what evidence do you have that the RoboMNIST reference trajectories are accurate at the sub-millimeter level?

  4. Were the MUKCa records used in the audit (the 14 non-KPI locations) used at any stage of fitting the published calibrated models? If so, how should the recovery rates be interpreted?

Reviewer_Rj61OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when accurate predictions require physical state estimates to exceed their allowed error. It proposes an audit to identify conflicts that cannot be resolved by improving the state estimator alone. Experiments on trajectory prediction and robot calibration examine these conflicts and whether changing the prediction model or adding state information can resolve them.

Critical strengths

S1. The paper clearly distinguishes errors made by a particular estimator from conflicts inherent in the fixed prediction interface.

S2. The experiments connect the identified conflicts to actual model outputs and track whether the same cases recover after an interface change.

S3. The validation and reporting are careful, including unresolved cases, sensitivity analyses, and failures introduced by repair.

Critical weaknesses

W1. The contribution lies mainly in the diagnostic framework and empirical analysis, with a modest advance in general verification theory. To be specific, Eq. (4) follows directly from joint infeasibility, and the affine and ReLU checks use established verification techniques (Appendices B.3 and C.4).

W2. The repair experiments lack key controls needed to separate the effects of acceleration information, the adjusted prediction formula, and the residual network.

W3. Repair gains depend strongly on the chosen error thresholds and assumptions about reference accuracy. Some gains reverse under alternative settings, while the accuracy of the trajectory references is not independently established.

Questions for authors

Q1. Could you please provide a concrete example where the audit leads to a better model-selection or repair decision than prediction error and joint passing alone?

Q2. How does the repair compare with ordinary constant acceleration plus the same residual network, and with a learned prediction head using both velocity and acceleration? Please compare recovery on the same cases.

Q3. What measurement evidence or application requirements support the main uncertainty assumptions and tolerances? Can the nonlinear results also be evaluated under a justified reference-error range?

Reviewer_xCd8OpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposed a test for setups where a model estimates one physical quantity (the "state," such as velocity), and several prediction formulas then use that one state. The goal is to tell two failures apart: whether the model simply picked a poor state, or whether the formulas force any state that predicts well to be physically wrong, beyond an error margin the authors choose. For each data sample, the authors mathematically prove which case applies. They then check whether redesigning the formulas fixes the second case, for example by adding acceleration to a constant-velocity predictor. They apply the test to robot hand-motion data and to published robot calibration models. They find that learned models sometimes report physically wrong states to predict well, and that redesigned formulas fix many such samples.

Critical strengths
  1. The paper tackles an important problem. Low prediction error alone can not tell apart the underlying issue. The paper separates "the model picked a poor state" (a better model can fix it) from "the formulas force a wrong state" (only redesigning the formulas can fix it).
  2. The experiments are well-designed and cover a wide range of settings in robotics.
Critical weaknesses
  1. Some insights are somewhat intuitive. For example, in Section 4, it is intuitive to use acceleration instead of velocity. This is the first common validation: the state must be designed to support every formula that uses it.
  2. The paper is very hard to read. The abstract and introduction use several terms and acronyms without defining them.
  3. The limitations of the work and practical applications can be discussed to strengthen the paper
Questions for authors
  1. In every experiment, standard checks reach the same conclusion as the audit. These checks are state error, prediction error, and trying a richer formula [Table 2, Table 4, Table 6]. The only disagreement (TACO) is not significant (CI [−5.00, 1.94]) [App. G]. Can you show a case where the D labels reveal a problem or guide a decision that these checks would miss?

  2. Every label depends on the tolerances in Table 7 (0.1 m/s, 20 mm, 0.01°, 1 mm), which are not tied to a downstream requirement or to sensor accuracy. App. How should a practitioner choose tolerances so that a D label is meaningful? How do the RoboMNIST D counts change across tolerances?'

Without a principled way to set tolerances, does the D/F distinction say something about the physical system, or mainly about the chosen numbers?

  1. In each experiment, the fix (add acceleration, recalibrate) is what one would do anyway [Sec. 4–5]. In what situations would the audit lead someone to act differently than standard evaluation would?

  2. Of the 18 D samples where learned models pass, only 4 survive 1 mm of reference position error, and none survive 1 mm plus 2 ms of timing error [Table 9]. TACO reports timestamp mismatch of up to 17 ms [App. A]. How robust are the main claims to realistic measurement error?

  3. All experiments use low-dimensional physical states with simple formulas [Sec. 3–5]. Does the idea, and the ability to certify it, extend to learned representations in larger models, where it would matter most?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.