acceptodds
Under review as a conference paper at ICLR 2027

Learning Precise Spatiotemporal Relational Reasoning in Large Language Models

Abstract

Inventory tracing and historical audits require complete answers about which objects reached a destination, when they shared a location, and what incomplete observations establish. These queries combine changing spatial dependencies, temporal aggregation, and evidence limits; a plausible item or a correct final location alone may not complete the request. We study how Large Language Models can learn these joint reasoning requirements. Executable task construction supplies questions, answers, and derivations for both evaluation and supervision. To scale language supervision, we use a strong teacher's interactions on 9,000 seed instances to train two smaller question and reasoning polishers. Applied to newly generated histories, they yield 85,940 accepted training records under source and rendering constraints. We examine learning with smaller corpora and follow the expanded-corpus solver through supervised fine-tuning and reinforcement learning. Recorded Qwen3.5-0.8B validation accuracy starts at 0%, reaches 38% at the supervised-to-reinforcement-learning handoff, and reaches 45% for the selected RL model. Paired answers show corrected temporal and occupancy results alongside persistent evidence-state and value errors.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
spatiotemporal reasoning, large language models, executable supervision, synthetic data generation, supervised fine-tuning, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_k3rnOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This work focuses on improving the spatiotemporal relational reasoning capabilities of LLMs, particularly for tracking dynamic object relationships, temporal events, and incomplete or ambiguous observations. The authors construct a programmatically generated benchmark with verifiable answers, and propose a teacher-student pipeline to generate large-scale training data, followed by SFT and RL to improve a smaller model's reasoning capabilities. The results show that the qwen3.5 0.8b model achieves 45% accuracy after training, compared with 38% after SFT alone, demonstrating the potential of this approach for improving complex relational reasoning.

Critical strengths
  1. The proposed strategy is meaningful in practice. For tasks operating in real physical environments, the execution trajectories naturally involve various spatial memory and reasoning requirements, which are essential for developing AI systems that can operate reliably in the physical world. However, I think a more promising direction for this work may actually be laboratory environments rather than household scenarios. Many laboratory tasks involve potential safety risks and require small on-device models to make quick decisions, while the long reasoning time of large models could cause serious delays. In this regard, achieving such noticeable improvements with a relatively small 0.8B model is quite impressive (though still not enough).

  2. Another interesting aspect of this work is the framework for programmatically generating verifiable datasets, which could theoretically produce sufficiently diverse training data at scale.

Critical weaknesses

If I have not missed anything, I think there are three main limitations:

  1. The current ablation study is somewhat limited. For example, are both polishers really necessary, and how exactly do they contribute to the accuracy? It would be helpful to compare the performance using the original data directly, data generated by the teacher model without polishing, and data further refined by the polishers.

  2. The generalization ability has not been sufficiently validated. The current training and testing seem to be conducted within similar types of environments, which may not be enough to demonstrate strong generalization. Future experiments could consider training in household environments and testing in laboratory environments, or introducing unseen systems, objects, and object states during testing.

  3. The current accuracy is still not sufficient for real-world deployment, especially in scenarios where reliability is critical. Perhaps the authors could also explore slightly larger models in future work, since the inherent capability limitations of a 0.8B model may already be a major bottleneck.

Questions for authors
  1. How do the temporal length and spatial complexity of events affect the model's performance? The paper covers 90 types of query algorithms, but it seems that the authors do not report the length of event sequences or the complexity of spatial dependencies. I think these statistics would be very helpful for understanding the model's performance and limitations. For example, in real-world applications, models may need to handle much longer event sequences and more complex spatial relationships. If the performance drops significantly under such conditions, it could become a major limitation for future practical applications.

  2. If feasible, could the authors provide some additional experiments to evaluate the model's generalization ability? For example, as mentioned above, testing on environments that are fundamentally different in type from those in the training set, or using event sequences involving interactions with previously unseen objects. Such experiments would help better demonstrate how well the model generalizes beyond the training scenarios.

6 comments
AuthorsOpenReview ↗

To W1: How the two polishers affect solver accuracy and supervision quality

Polishing only the rationale helped the solver correctly answer 81 and 83 more of the 900 answerable questions than training on the original text in the two 1,200-example studies. The two polishers learn from teacher demonstrations to express executable queries as questions and explain program derivations as rationales. Within each study we kept the source examples and program-computed answers fixed. We trained each solver from the same Qwen3.5-0.8B weights for 500 SFT updates and evaluated it on the same 1,080 questions (Section 5.7 of the revised paper).

Training textSet 1: overall accuracySet 1: answerable correct /900Set 2: overall accuracySet 2: answerable correct /900
Program-generated (Raw)15.83%2415.93%1
Polish question only11.85%7916.30%2
Polish rationale only13.61%1058.80%84
Polish both (Dual)8.15%736.39%63

Adding question polishing to the polished rationales reduced false refusals on answerable questions from 212 to 92 in set 1 and from 64 to 19 in set 2. Rationale-only gave the most complete answers on answerable questions in both sets, and Raw scored higher than Dual overall.

We also trained each solver on the same 356 examples for which the teacher produced both texts. Dual yielded 161 correct answers out of 1,080, 21 more than question-only (140) and 61 more than rationale-only (100). The counts were 125 with Raw, 111 with teacher-direct text and 53 with text from a prompted original 4B model. Relative to Raw, Dual correctly answered 97 more insufficient-evidence questions (153 vs. 56 of 180) and 61 fewer answerable questions (8 vs. 69 of 900), for a net gain of 36.

Training textAll correct /1,080Answerable correct /900Insufficient-evidence correct /180
Raw1256956
Question-only14023117
Rationale-only1005545
Dual1618153
Teacher-direct1116348
Prompted original 4B532726

Does polishing preserve the question and its derivation? From the 356 sources we selected two from each of the 15 query-family and evidence-state combinations before reading the texts. We first reviewed Raw, Dual and teacher-direct, then reviewed the three additional conditions on the same 30 sources. Each comparison used two review rounds with method labels hidden. A pair passed when the question preserved the executable query's meaning, the rationale followed from the public evidence, and the two agreed.

Supervision (30 pairs each)PassedClear issueUnresolved
Raw2136
Dual2136
Teacher-direct2046
Question-only2082
Rationale-only1965
Prompted original 4B6168

Both single-polisher conditions passed on all six sources where prompted 4B passed. In one example, the question polisher kept the restriction to chairs originally in Kitchen 1, which selects only Chair 4. Prompted 4B asked about both Chair 4 and Chair 3, then gave a rationale for Chair 4 alone.

On the 19 sources where Raw, Dual and teacher-direct all passed, Dual raised the number of questions rated natural from 10 to 16 and clear from 16 to 19 relative to Raw. Teacher-direct scored 16 and 18, respectively (Table 12 of the revised paper).

AuthorsOpenReview ↗

To W2: Learning on unseen scenes and event histories

Training raises complete-answer accuracy from 1.39% for the original model to 29.91% after SFT and 31.76% after RL on 1,080 questions about 100 residential scenes absent from training and development. These gains demonstrate learned reasoning on new scenes and event histories, using the same public observations and queries for all three models.

AuthorsOpenReview ↗

To W3: Larger-model comparisons and batch inference cost

The trained 0.8B solver achieves 31.76% accuracy, exceeding original 4B/9B direct answering at 20.37%/27.78% on the same independent questions. Both polishers construct supervision offline; only the solver runs at inference.

Compared with single-pass 4B executable querying, the trained solver uses 97.5% fewer generated tokens and 98.2% less batch processing time. The trained 0.8B solver and the single-pass 4B query model both enable Thinking with the same 16,384-token budget per generation. The query model returns one program for execution; the trained solver returns the structured answer directly.

SystemComplete-answer accuracyBatch time (seconds)Generated tokens, including Thinking (millions)
Original 4B, single-pass executable query30.46%9,28514.02
Trained 0.8B, RL checkpoint answering directly31.76%1710.35

Batch times cover generation, query execution where applicable, scoring, and output recording for the same 1,080 questions on an A800, after model loading, input preparation, and request admission.

Reviewer_heB5OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates precise spatiotemporal reasoning over dynamic relational histories, focusing on three key challenges: tracking evolving spatial relations, reasoning over temporal histories, and handling ambiguous or insufficient evidence. The authors develop an automated framework for constructing executable QA tasks and train two 4B polishers using demonstrations from a stronger LLM to generate training examples. A 0.8B solver trained with SFT and GRPO achieves 45% accuracy.

Critical strengths
  • The paper addresses a meaningful combination of spatial, temporal, and evidence-sensitive reasoning.

  • Executable task generation provides verifiable answers and scalable supervision.

Critical weaknesses
  • The method mainly combines synthetic data generation, SFT, and GRPO, without introducing a specialized spatiotemporal reasoning mechanism.
  • The paper lacks direct comparisons with existing spatiotemporal reasoning methods.
  • The necessity of training two 4B polishers is unclear, as comparisons with unmodified 4B models and unpolished synthetic data are missing.
  • The paper does not establish whether the model generalizes to unseen scenario,only ProcTHOR
  • The paper reports results only on a 100-question validation set, which is also used for checkpoint selection. No independent test set
Questions for authors
  • How does the learned solver compare with an LLM that translates natural-language questions into executable queries?
  • Does the solver learn generalizable spatiotemporal reasoning, or primarily adapt to the 90 predefined query algorithms?
9 comments
AuthorsOpenReview ↗

To W1: Task-specific supervision for joint spatiotemporal reasoning

Our framework constructs executable supervision for joint reasoning over changing nested relations, historical queries, and incomplete or ambiguous evidence. It couples a precise answer standard with task-specific data construction and solver learning:

  • Task and answer standard. A query can require tracking changing containment, selecting a historical interval, and resolving ambiguous references together. A complete answer must include every referent binding and its value supported by the public observations, or identify the evidence needed to answer.
  • Executable construction. Object, relation, and event transitions reconstruct historical states. Programs execute the query over the histories consistent with the public observations, producing answers and derivations for supervision.
  • Scalable language supervision and solver learning. Strong-teacher demonstrations train question and rationale polishers to express executable queries and derivations in natural language, while programs supply answer labels. The resulting supervision trains a compact solver through SFT, followed by complete-answer GRPO.

Both polishers operate offline during data construction; the solver answers directly at inference.

AuthorsOpenReview ↗

To W2: Comparison with related reasoning approaches

On the same 1,080 independent questions, our trained 0.8B solver answers directly at 31.76% accuracy, compared with 20.37% for original 4B direct answering and 30.46% for 4B executable querying.

  • Task requirements. Entity Tracking studies evolving object states and complete object lists. Our queries combine changing nested containment with historical time conditions and require all referent-binding/value answers supported by incomplete or ambiguous public observations.
  • Supervision construction. SpaRP verbalizes symbolic spatial paths, while SpaRTUN combines scene graphs, logical reasoning, and language generation. Our executor reconstructs relations through event histories and computes historical answers, complete referent coverage, and missing-evidence requirements from public observations.

SynLogic and Enigmata establish the broader use of verifiable synthetic tasks and SFT/RL. Our framework makes historical answers, complete referent coverage, and missing-evidence requirements executable, providing supervision for the joint reasoning task.

AuthorsOpenReview ↗

To W3: Polisher roles and language-quality comparisons

Dual improves question naturalness and clarity over Raw. In a fixed 30-source comparison of program-generated question–rationale pairs (Raw), outputs from the two learned polishers (Dual), and strong-teacher outputs (teacher-direct), natural-question ratings rise from 10/19 to 16/19 and clear-question ratings from 16/19 to 19/19. The comparison uses the same 19 sources where all three question–rationale pairs passed semantic review.

The two polishers learn complementary mappings: the question polisher expresses entity scope, spatial relations, quantifiers, and time boundaries from public query specifications; the rationale polisher explains execution-derived calculations and evidence branches. Programs supply the answer labels, and both polishers operate offline.

The learned pipeline also reduces clear question-semantic errors relative to prompted original 4B: 1 versus 8 in a separate fixed 30-source review. For example, the learned question preserves “with 5 cross-room moves before 14:09”, while prompted original 4B changes this to “at least 5 cross-room moves before 14:09”.

Reviewer_xmzmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes an automatic pipeline to generate a benchmark for spatiotemporal relational reasoning and supervision for LLMs. The pipeline uses program execution to construct event histories, queries, answers, and derivations. The authors then use outputs from a strong teacher model to train two smaller student models, called polishers, to generate reasoning explanations and question wording. The resulting resources include approximately 86K question-answer pairs for training and 100 questions for validation. The task distinguishes determinate answers, ambiguous referents, and insufficient evidence. The authors evaluate five LLM families spanning open-weight and closed-source models and study how smaller LLMs learn spatiotemporal reasoning through supervised fine-tuning and reinforcement learning using the generated data.

Critical strengths
  • The paper studies a challenging problem that requires language models to combine spatial and temporal reasoning.
  • The paper proposes an automatic pipeline for constructing a dataset to supervise LLMs on spatiotemporal reasoning tasks. The task explicitly distinguishes determinate answers, ambiguous referents, and insufficient evidence, which could be useful for investigating model capabilities and learning.
  • The paper evaluates five LLM families spanning open-weight and closed-source models.
  • The paper examines how smaller LLMs learn spatiotemporal reasoning through supervised fine-tuning (SFT) and reinforcement learning (RL) using the generated data.
Critical weaknesses
  • The paper needs substantial improvement in writing and organization, especially in the methodology section. Although separate subsections describe dataset generation and solver training, their relationships are difficult to follow, making it harder to understand the motivation, purpose, and novelty of the approach. A clearer separation between the automatic data-generation pipeline and the study of how LLMs learn spatiotemporal reasoning would improve readability.
  • The novelty and focus of the paper are unclear. If the main contribution is the automatic generation pipeline, a more explicit comparison with existing datasets and generation approaches would help establish its value. If the main contribution is understanding how LLMs learn spatiotemporal reasoning, the paper would benefit from a clearer emphasis on the experimental questions and the setup used to investigate them.
  • The evaluation is limited to 100 questions. Section 4 states that this same validation set is used for model evaluation and solver checkpoint selection, without an independent test-set result. This limits confidence in the reported performance and its generalizability.
  • The learning experiments are limited to smaller models within the Qwen3.5 family, and the SFT-to-RL study uses only Qwen3.5-0.8B. It is unclear whether the observed learning trends extend to larger models or other model families.
  • The paper does not evaluate whether training on the generated dataset improves performance on other related spatiotemporal reasoning benchmarks. Such evaluation would help establish whether the learned capabilities transfer beyond the constructed tasks (generalization test).
  • Although the paper provides learning curves, reference-model breakdowns by answer status, and a small paired SFT analysis, it does not systematically examine category-specific performance across the original, SFT, and RL solver checkpoints. This makes it difficult to determine which reasoning requirements improve with training. An additional study varying the proportions of determinate, ambiguous, and insufficient-evidence questions could further clarify how the training-data composition affects learning.
Questions for authors

Please address the weaknesses, especially clarification of novelty, evaluation subset, and generalization.

7 comments
AuthorsOpenReview ↗

To W1: Executable supervision for learning spatiotemporal reasoning

Our framework turns executable spatiotemporal queries into question–rationale–answer supervision and trains a compact solver to answer them directly. Three connected stages support reasoning over changing spatial relations, historical queries, and public evidence:

  • Executable supervision: programs reconstruct relational states from event histories and compute query answers and derivations, including whether the public evidence supports a complete answer.
  • Language supervision: strong-teacher demonstrations train two mappings. The question polisher expresses the query's object scope, spatial relations, and time conditions; the rationale polisher uses execution records to explain intermediate computations and evidence branches. Programs supply the answer labels. The learned mappings produce language supervision at scale.
  • Solver learning: SFT teaches the solver from this supervision, and complete-answer GRPO rewards correct query completion. Both polishers operate offline; the solver answers directly at inference.

For example, a first-return query requires tracking when an object count returns to an earlier value: execution determines the relevant changes and answer, while the polished question and rationale express the same object scope and temporal condition.

AuthorsOpenReview ↗

To W2: Contribution and relationship to prior work

Our framework constructs executable supervision for joint reasoning over changing nested relations, historical queries, and incomplete or ambiguous evidence. It also establishes how SFT and complete-answer RL improve query accuracy, referent coverage, and binding-specific values on unseen scenes and event histories:

  • Task and answer standard: jointly require reasoning over changing nested spatial relations, historical time conditions, and incomplete or ambiguous evidence. A correct answer must include every supported referent alternative and its value, or identify the evidence needed to answer.
  • Executable supervision construction: reconstruct states from objects, relations, and events, then execute queries to generate verifiable answers and derivations. Answerability is determined from public observations.
  • Scalable language supervision and solver learning: train question and rationale polishers from strong-teacher demonstrations to express executable queries and derivations in natural language, while programs supply answer labels. The resulting supervision trains a compact solver through SFT, followed by complete-answer RL. We evaluate the solver's learning gains on unseen scenes and event histories.

Entity Tracking studies evolving object states and complete object lists; SpaRP constructs spatial paths and verbalizes them for training. Our task combines changing containment with historical queries and requires all answers supported by the available evidence. SynLogic establishes verifiable synthetic tasks, and Enigmata combines reasoning traces with SFT and RL. Our framework makes historical answers, complete referent coverage, and missing-evidence requirements executable, providing a common basis for supervision and learning evaluation.

AuthorsOpenReview ↗

To W3: Independent evaluation beyond the 100 development questions

On 1,080 independent questions from 100 scenes absent from training and development, complete-answer accuracy rises from 1.39% for the original model to 29.91% after SFT and 31.76% after RL.

SFT adds 28.52 percentage points (scene-paired 95% interval [26.22, 30.92]); RL adds 1.85 points [0.28, 3.48].

The checkpoints were fixed before their independent evaluation, with the RL checkpoint selected using development performance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

To all reviewers: Executable supervision, learned reasoning, and compact inference

Our framework turns dynamic spatial relations, historical queries, and public evidence into executable supervision that a compact model learns to use. The resulting 0.8B solver reaches 31.76% complete-answer accuracy on unseen scenes and uses 97.5% fewer generated tokens than single-pass 4B executable querying. Additions and substantive changes in the revised manuscript are highlighted in blue.

1. Executable supervision scales through reusable language generators (Section 3; Appendix D, Table 12). Strong-teacher demonstrations on 9,000 seed instances train question and rationale generators that produce 85,940 question–rationale–answer records for solver learning. Programs compute the complete answers, including supported referent alternatives and missing-evidence requirements; the learned mappings express query conditions and execution derivations in language. Both generators operate offline; only the solver runs at inference. The learned Dual pipeline also improves question expression over program-generated Raw: natural-question ratings rise from 10/19 to 16/19, and clear-question ratings from 16/19 to 19/19. These comparisons use the same 19 sources where Raw, Dual, and strong-teacher pairs all passed semantic review, drawn from a fixed 30-source comparison.

2. The solver learns to apply the 90 query computations to unseen scenes and event histories (Sections 5.4–5.5; Table 3; Appendix C). In the additional learning study, complete-answer accuracy on 1,080 independent questions from 100 scenes absent from training and development rises from 1.39% for the original 0.8B model to 29.91% after SFT and 31.76% after RL. SFT adds 28.52 percentage points (scene-paired 95% interval [26.22, 30.92]); RL adds 1.85 points [0.28, 3.48]. The checkpoints were fixed before their independent evaluation, with the RL checkpoint selected using development performance. Training improves all five query families, and RL further improves four. On ambiguous questions, RL increases complete binding sets from 126 to 146/360 and correctly valued bindings from 135 to 160/720.

3. Learned answering combines small-model accuracy with substantially lower inference cost (Section 5.6). On the same questions, the trained 0.8B solver reaches 31.76% accuracy, compared with 20.37% and 27.78% for original 4B and 9B direct answering. Against single-pass 4B executable querying, which reaches 30.46%, the solver uses 97.5% fewer generated tokens and 98.2% less batch processing time: 0.35M versus 14.02M tokens, and 171 versus 9,285 seconds. Both enable Thinking with a 16,384-token budget per generation. Timing covers generation, query execution where applicable, scoring, and output recording for the same 1,080 questions on an A800, after model loading, input preparation, and request admission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.