- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper examines whether accurate prediction leads to learning the underlying process structure. Using a synthetic benchmark, the authors find that OCM performs poorly in recovering latent states and transformations. However, the experiments do not clearly explain the reasons behind this failure.
Critical strengths
-
The paper addresses an interesting question about the relationship between prediction and structural learning.
-
The benchmark provides ground truth for evaluating structural recovery beyond prediction accuracy.
Critical weaknesses
-
The novelty is limited. The main finding is closely related to existing identifiability problems, and the paper does not clearly establish what new insight it offers.
-
OCM itself has relatively weak predictive performance, making it difficult to tell whether the failure comes from predictive learning or from the model and its optimization.
-
Important baselines and ablations are missing. Structural consistency constraints, partial supervision, and structural probes on other predictive models would help clarify the findings.
-
The experimental analysis is insufficient. The large gap between state recoverability and OCM alignment is not explained, and the reliance on hard transition maps may overlook useful structural information. The limited synthetic setting and the unexplained difference from the positive control also raise concerns about generalizability.
Questions for authors
-
How can the authors distinguish a limitation of predictive learning from the weak predictive performance or optimization issues of OCM?
-
What causes the large gap between 98.31% state recoverability and 15.72% OCM alignment?
-
Would adding structural consistency constraints or partial state supervision improve structural recovery?
-
Could soft transition metrics or structural probes on other predictive models lead to different conclusions?