- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3(+1)
- Soundness
- 3(+1)
Read review
Paper summary
This paper investigates action-conditioned state transition modeling in relational databases. While standard relational deep learning focuses on static target prediction (), this work formulates relational database updates as dynamic transitions where an action on state produces a sparse set of structured edits (INSERT, UPDATE, DELETE across multiple tables). Adopting the Prior-Data Fitted Network (PFN) paradigm, the authors introduce WORLDREL, a procedural synthetic data generator for relational schemas, actions, and transition rules. They propose TabWM, which combines a Relational Transformer backbone with a DETR-style set prediction decoder and gated affine/additive numeric transfer heads to infer system dynamics from demonstrated transitions in context without parameter updates. Finally, TabWM is evaluated on synthetic held-out worlds and tested as a recursive rollout model for 3-step candidate action sequence ranking.
Critical strengths
- An Interesting Attempt to Extend Relational Learning to Action-Conditioned Dynamics: Moving relational learning beyond passive, static property prediction into modeling state transitions under external actions is a sensible and exploratory conceptual direction. It provides an interesting angle for exploring how tabular representations might interact with dynamic decision-making environments.
- Plausible Architectural Adaptation for Discrete Database Edits: Formulating multi-table database changes as an unordered set of sparse edits via DETR-style bipartite matching is a reasonable engineering choice for capturing discrete database operations. The inclusion of action/consequence shuffling experiments also provides an intuitive sanity check that the model is sensitive to behavioral correspondence in context.
Critical weaknesses
-
Unclear Positioning and Baseline Representativeness Relative to TabPFN: The underlying learning mechanism of TabWM shares the Prior-Data Fitted Network (PFN) paradigm (Müller et al., 2022; Hollmann et al., Nature 2025). However, the comparison with TabPFN would benefit from greater clarity and fairness. In Table 1, "Flat-TabPFN" is evaluated on fixed-width vectors without schema names or relational edge topology, which naturally limits its performance. Given that the authors cite the recent relational foundation model TabPFN-Rel (2026), it remains unclear why a multi-table PFN extension equipped with transition heads was not considered as a baseline, and what unique structural necessity TabWM offers beyond a natural adaptation of the PFN framework.
-
Practical Motivation of the Setting and Unjustified Planning Horizon (H=3):
- Conceptual Tension in Setting: We acknowledge that real-world operational database benchmarks with aligned before-and-after action traces currently do not exist in the public domain, which reasonably justifies the use of a procedural synthetic environment (WORLDREL). However, there remains an inherent conceptual tension in the problem motivation: in real-world enterprise databases, business logic and cascade updates are typically deterministic and hardcoded into backend software services. If the system rules are already specified in code, the practical necessity of training an approximate neural world model to predict state transitions remains somewhat ungrounded.
- Unmotivated Horizon Choice: In Section 8 and Appendix B.4, the model-based planning is evaluated exclusively at a fixed horizon of . The paper does not provide domain statistics, workflow analysis, or empirical justification for why 3 steps was selected as the representative decision depth.
Questions for authors
I will consider raising my score if the authors can address my concerns below:
- On TabPFN Positioning and Baseline Comparison: Could the authors clarify the conceptual and empirical boundary with TabPFN? Specifically:
- (a) How was Flat-TabPFN constructed, and does stripping relational topology and schema context account for its low score?
- (b) Could a relational variant of the PFN paradigm (such as adapting TabPFN-Rel with edit-prediction heads) serve as a more representative baseline, and what distinct capabilities does TabWM provide over such an extension?
- On Evaluation Settings (Planning Horizons and Real-World Traces):
- (a) Choice of and Alternative Settings: Could the authors clarify the rationale behind fixing the planning horizon specifically at , and why experiments under alternative horizon settings (e.g., or variable-length horizons) were not explored? Is this because representative decision workflows in these worlds naturally conclude within three steps, or was this guided by search complexity or compounding error considerations?
- (b) Feasibility of Real-World Traces: Supplementing the evaluation with real-world operational execution traces (even on a small scale from enterprise change logs or workflow benchmarks) would be ideal. We fully recognize that aligned before-and-after database benchmarks are difficult to obtain in the public domain and understand if this is not possible for the rebuttal, but any preliminary evidence or discussion on real-world feasibility would be welcomed.
5 comments
Thanks For Response
Thank you for the response and the additional experiments. The H = 1, 2, 3, 5 evaluation addresses my concern regarding the fixed horizon, and the clarifications on the external API-agent setting, Flat-TabPFN, and TabPFN-Rel improve the paper's positioning. Accordingly, I have raised Presentation and Soundness to 3.
However, I am maintaining Informativeness at 2 and my overall rating at 2 (Weak Rejection) due to the following unresolved issues:
- Circular Validation on Synthetic Dynamics: Even in Section 7, only the schemas and value distributions are imported while all transition rules remain generated by WORLDREL. Evaluating a PFN exclusively on dynamics sampled from its own procedural training prior does not establish transfer to independently authored business logic (e.g., external workflow benchmarks such as World of Workflows, or real SQL/application rules).
- Reliability, Consistency, and Calibration Gap: In the stated external API-agent setting—where unsafe trial actions cannot be executed—TabWM still achieves ~0.66–0.67 Edit F1 at K = 64 and incurs ~0.21 normalized regret for H ≥ 2 (ρ ≈ 0.41–0.49). The paper neither guarantees that recursively rolled-out states satisfy database integrity constraints (e.g., foreign-key consistency) nor provides uncertainty calibration/abstention when predictions are unreliable.
- Limited Insight for the Community: Beyond engineering an integration of a PFN-style prior, Relational Transformer, and DETR-style set decoder for sparse database edits, the takeaway insight offered to the broader world-modeling and relational-learning community remains below the ICLR bar.
Thank you for recognizing the paper's "sensible and exploratory conceptual direction" and for acknowledging that our additional experiments addressed the horizon concern and improved the paper's positioning (Presentation and Soundness both increased from 2 to 3). We clarify the remaining points below:
1. Evaluation scope.
WORLDREL enables controlled tests of how a model learns hidden relational dynamics from executions. As the original review noted, the lack of aligned real operational traces makes a procedural setting reasonable. Beyond this practical need, controlled worlds let us isolate the evidence that matters.
In §3, we hold the state and action fixed while changing the hidden rules. The resulting edits differ in 65% of paired-world probes. Table 2 tests reassigned action mechanisms, while Appendix F.4 introduces four mechanisms absent from pretraining. These experiments show both adaptation and its limits, particularly for exact value transfer.
Section 7 separately tests real-derived structures and value distributions, not real business rules. We already discuss this limitation in §10 and will clarify the evaluation scope. Beyond addressing the lack of real execution traces, WORLDREL provides a controlled setting for studying how relational models identify and generalize system dynamics. We hope it can also serve as a starting point for further research on this problem.
2. Methodological contribution and insights.
TabWM contributes an in-context learning framework for relational dynamics, not simply another prediction head. Unlike static relational prediction, this task requires learning which changes follow an action in a particular system. WORLDREL varies the dynamics across tasks, the encoder preserves the correspondence between actions and their observed consequences, and the decoder predicts structured edits that can be applied recursively. The numeric transfer ablation further confirms the value of this design (Appendix F.5).
Our experiments reveal several findings beyond the architectural design.
- Behavioral correspondence matters. Shuffling actions reduces Edit F1 from 0.664 to 0.041 (§6), showing that the model relies on which actions produced which consequences.
- The model learns more than past edit patterns. With four matching executions, TabWM reaches 0.700 Edit F1 versus 0.216 for direct template transfer (Appendix F.3).
- Better predictions do not always lead to better decisions. As K increases from 16 to 64, ranking correlation improves from 0.590 to 0.622, while normalized regret remains nearly unchanged (0.208 vs. 0.207, §8).
These findings extend in-context system identification [1] to sparse, multi-table consequences. World of Workflows independently demonstrates the importance of hidden cascading effects in enterprise systems. Our contribution is both a relational model that learns such behavior from experience and a controlled framework for understanding when and how this learning succeeds. We will make these findings and their supporting evidence more prominent.
3. Reliability, consistency, and calibration.
Our planning claim is that learned dynamics improve action choices, not that every simulated state is exact or safe to execute. Section 8 tests this directly. At H=3 and K=64, TabWM achieves 0.207 normalized regret, compared with 0.393 for no-change and 0.392 for shuffled context, a reduction of about 47%. Spearman correlation reaches 0.622 versus 0.345 and 0.338 for these controls.
Prior work also shows that transition accuracy and decision utility are different objectives [2, 3]. Known foreign-key constraints can be checked separately before accepting predicted edits, whereas the hidden transition rules are precisely what TabWM learns from executions. Calibration and abstention are important for autonomous deployment, but our experiments evaluate offline consequence prediction and candidate ranking, not guaranteed safe execution. We will make this distinction clearer.
We hope these clarifications make the scope and significance of our contributions clearer, and we would be truly grateful if you would reconsider your overall rating in light of our responses.
[1] Forgione et al. "From System Models to Class Models: An In-Context Learning Paradigm." IEEE Control Systems Letters, 2023.
[2] Grimm et al. "The Value Equivalence Principle for Model-Based Reinforcement Learning." NeurIPS, 2020.
[3] Lambert et al. "Objective Mismatch in Model-based Reinforcement Learning." L4DC, 2020.
Thank you for the thoughtful review and for recognizing the value of extending relational learning to action-conditioned dynamics and structured edit prediction! We would like to address your concerns below:
Q1(a) / W1. Flat-TabPFN and relational information.
Flat-TabPFN converts action-conditioned transition tasks into fixed-width feature-based prediction problems. As detailed in Appendix B.2, its features summarize action bindings, relational links, column statistics, and observed changes. It therefore retains relational and behavioral cues, although not the full relational topology or the structure of individual executions.
Does this information loss explain its lower score? Table 1 cannot isolate that effect, since the models also differ in training and prediction interfaces. We therefore use Flat-TabPFN as a feature-based reference, rather than a matched topology ablation. Our controlled experiments in §6 and Appendix F.5 separately examine behavioral correspondence and model components. We will clarify this distinction.
Q1(b) / W1. TabPFN-Rel and TabWM.
TabWM's contribution is a framework for identifying system-specific relational dynamics from executions in context, not simply a new PFN output head. The PFN paradigm learns from tasks sampled from a prior [1]. TabPFN-Rel [2] uses relational feature synthesis for entity-level prediction. In contrast, WORLDREL supplies a prior over executable dynamics (§4.1), TabWM represents aligned state–action–edit executions (§4.2), and its decoder predicts variable-size edits with relational pointers and numeric transfer (§4.3). These edits support recursive future-state prediction (Eq. 8).
The distinction matters empirically. In §3, identical states and actions yield different edits in 65% of paired-world probes, making observed behavior essential for identifying the active dynamics. In Figure 4, shuffling actions reduces Edit F1 from 0.664 to 0.041 at K=64; Table 2 shows adaptation to reassigned action-to-mechanism mappings; and the matched numeric-transfer ablation improves Edit F1 from 0.476 to 0.616 (Appendix F.5). A TabPFN-Rel-based transition model is possible, but adding an edit head alone would not supply the execution-conditioned training prior or representation. We will make this distinction clearer.
Q2(a) / W2. Planning horizon.
H=3 is a controlled test of composing predicted transitions, not a claim about typical enterprise workflow lengths. We additionally evaluated H=1, 2, 3, and 5 on 40 held-out worlds (K=16, N=8, shared candidate prefixes), with 95% world-bootstrap confidence intervals:
Ranking correlations remain positive with confidence intervals above zero at every horizon, showing that the ranking signal is not specific to H=3. This is a separate horizon-sensitivity test; §8 compares candidate selection methods at H=3. We will clarify the evaluation purpose.
Q2(b) / W2. Practical motivation and real-world traces.
A deterministic backend is not necessarily a simulator available to an external agent. An agent operating through an API may observe actions and database changes but lack access to the rule engine or a safe way to execute candidate actions in advance. Predicting consequences from past executions is useful in precisely this black-box setting. World of Workflows [3] independently studies forward audit prediction under hidden business rules, including cascading changes across tables.
We also test transfer to real database structures in §7, using topologies from THE JOIN and value distributions from RelBench. At K=64, TabWM obtains 0.657 Edit F1 on imported topologies versus 0.671 on matched synthetic ones. These are not real operational execution traces; such traces could be constructed by aligning application action logs with database audit records. We will clarify this distinction and the intended deployment setting.
Thanks again for the valuable feedback! We hope this can address your concern adequately. If you find the clarifications satisfactory, we would appreciate your consideration in re-evaluating the manuscript.