acceptodds
Under review as a conference paper at ICLR 2027

EviTrans: Composable State Transitions for Video Reasoning

Abstract

Video state tracking requires associating events with persistent entities and preserving their history-dependent effects. A fixed interval-level state change can become invalid when earlier records are revised. We present EviTrans, a framework that makes evidence-grounded, composable state transitions the reusable unit of video reasoning. Each interval is represented as an executable function of its entry state and boundary memory, rather than the change realized under one history, preserving state-dependent effects during temporal composition and revision. A typed observation-to-transition interface connects query requirements, visual entity binding, and event decoding to executable updates, while retaining evidence references and unresolved information. Integrating this representation with a persistent segment tree and deterministic query programs enables compatible questions to share an executable history and compiled records to be revised while preserving earlier versions. On VSTAT's 834 videos and 1,500 questions, the frozen, offline configuration achieves an overall score of 51.11, exceeding the matched direct video question answering baseline by 13.58 points, with unavailable inputs retained as failures. In backend diagnostics on 211 videos retaining 33 versions, persistent maintenance reduces edit-to-answer latency by 22.2% and retained memory by 52.1% relative to blocked sequential execution, while historical queries are slower under the measured caching policies.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Reasoning, Visual State Tracking, Composable State Transitions

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_K5uLOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents EviTrans, a composable state transition framework for video state tracking and reasoning. To address the limitations of conventional video question answering methods, which struggle to maintain long-term historical states, handle event dependencies, and rectify early-stage errors, EviTrans converts visual observations into executable state transition functions with evidence associations, rather than fixed state delta values. Through typed query contracts, entity binding, and event decoding, the framework enables structured state updates. It leverages a Persistent Segment Tree to compose state transitions across temporal intervals, supporting historical version preservation, efficient revision, and query reuse.

Experiments demonstrate that EviTrans achieves an overall score of 51.11 on the VSTAT dataset, outperforming multiple baseline methods. Compared with block-wise sequential execution, it reduces the latency from modification to answer generation by 22.2% and cuts memory usage for version storage by 52.1%. This study shows that explicit and composable state transitions can improve the accuracy and maintainability of long-horizon video reasoning.

Critical strengths
  1. Improves the accuracy of video reasoning. EviTrans converts events occurring in videos into state transitions and continuously updates the world state, enabling the model to better understand the chronological order and mutual influences among events, and avoiding double-counting or omitted state changes. Meanwhile, query programs are adopted to directly compute answers, mitigating potential errors during LLM inference.

  2. Enables efficient historical revision and information reuse. EviTrans employs a Persistent Segment Tree to manage state transitions within videos. If misrecognition occurs for prior events, only relevant records need to be updated, instead of reconstructing the entire historical structure. Furthermore, different queries can reuse the existing world state to reduce redundant computation, boosting inference efficiency and saving memory.

Critical weaknesses
  1. There is a lack of stronger video reasoning baselines. Although ProViQ and VideoTree are relevant methods, they focus on programmatic VQA and hierarchical video representation respectively, and may not represent the most competitive solutions for stateful video reasoning. The related work section of the paper also discusses structured video memory approaches such as StreamForest and Event-Causal RAG, yet the experiments do not include direct comparisons against these methods.

  2. The paper primarily conducts end-to-end evaluation only on VSTAT, which fails to demonstrate generalization and raises concerns of benchmark-specific design.

The core contribution of EviTrans is a general stateful video reasoning framework, yet VSTAT mainly targets tasks including counting, localization, attribute prediction, and state tracking. There is insufficient evidence to show that this framework generalizes to more diverse video understanding scenarios, such as complex human interactions, multi-event reasoning in long videos, and open-ended question answering.

Additionally, the paper adopts predefined query contracts, state schemas, and domain adapters. These components require manually defined state fields and operational rules tailored to the task. Consequently, the strong performance on VSTAT may partially stem from task-specific handcrafted rules rather than a highly general state transition framework.

Without cross-benchmark evaluation, it remains difficult to quantify the amount of extra manual engineering required when migrating the framework to new tasks.

Questions for authors
  1. ProViQ and VideoTree are adopted as the main compared methods, while other structured video memory approaches including StreamForest and Event-Causal RAG are only discussed in related work without experimental comparison. Could the authors add direct experimental results against these stateful video reasoning baselines and analyze the performance gaps?

  2. The evaluation is limited to the VSTAT benchmark, which raises concerns about benchmark-specific design and limited generalization. Since EviTrans is claimed as a general stateful video reasoning framework, could the authors provide cross-benchmark evaluation on additional video reasoning benchmarks covering complex human interactions, long-video multi-event reasoning, or open-ended VQA, to verify its generalization and reduce reliance on task-specific handcrafted rules (query contracts, state schemas, domain adapters)?

2 comments

Thank you for identifying closer baselines and the evidence needed to assess transfer.

W1 & Q1. Structured video-memory baselines.

StreamForest organizes visual tokens into event-level memory, while Event-Causal RAG retrieves observed State-Event-State records. EviTrans represents intervals as executable transitions conditioned on incoming state and boundary memory. These differences motivate a direct system-level comparison.

Table R4: VSTAT results available for comparison.

MethodOverall
Direct Qwen3-VL-32B37.53
ProViQ18.91
VideoTree26.44
StreamForest42.63
Event-Causal RAG47.38
EviTrans51.11

W2 & Q2. Cross-benchmark scope and task adaptation.

CATER Task 3 complements VSTAT with an independent evaluation of object identity and final-position inference under occlusion and recursive containment. Table R5 presents the cross-benchmark results.

Table R5: CATER Task 3 results.

MethodTop-1 (%)
Direct Qwen3-VL-8B19.75
Chronological facts + 8B LM41.19
Explicit state + 8B LM63.66
EviTrans: sequential execution71.65
EviTrans: persistent execution71.65

Explicit state raises accuracy from 41.19% to 63.66%, and deterministic execution further improves it to 71.65% with either backend. These results demonstrate the benefit of explicit state representation and execution beyond VSTAT.

Task adaptation specifies position and containment fields, event and binding rules, geometry mapping, and a final-grid query through the transition interface. We will provide the adapter code and document the task-specific changes.

Reviewer_K5uLOpenReview ↗

Response to Rebuttal

I thank the authors for their rebuttal. While it was helpful, I still find the paper’s overall contribution and novelty to be limited, and the explanations provided in the rebuttal remain insufficient. I believe further revisions are needed and therefore maintain my original score.

Reviewer_Dg18OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EviTrans, a framework for video state tracking and reasoning based on composable state transitions. It converts visual observations into executable state updates and maintains them using a persistent segment tree, supporting historical queries and revisions of earlier events. The method is evaluated on VSTAT and achieves improvements over direct VQA and program-based baselines. Additional experiments examine the contributions of explicit state maintenance and the efficiency of versioned execution.

Critical strengths
  1. The paper addresses an interesting problem in video reasoning. Representing events as composable state transitions provides a structured way to maintain historical information and handle state-dependent updates.

  2. The results on VSTAT are encouraging, with clear improvements over several baselines. The component analysis and backend efficiency experiments also help clarify the contributions of explicit state execution and persistent memory maintenance.

Critical weaknesses
  1. Limited generalization. The framework relies on predefined state schemas, operation types, and task-specific rules. While effective for structured state-tracking questions, it is unclear how well the approach generalizes to unseen event types or more open-ended video reasoning tasks. The evaluation is limited to VSTAT.

  2. Unclear practical benefit of the persistent backend. Persistent segment trees are well-established data structures, and the proposed backend does not improve QA accuracy over sequential execution. The measured efficiency gains are relatively small in absolute terms compared with the cost of visual processing. It is unclear whether this additional complexity provides a meaningful advantage in practical video reasoning applications.

Questions for authors
  1. How does EviTrans handle questions requiring state variables or operations that are not covered by the predefined contracts? Can the authors provide evidence of generalization beyond VSTAT?

  2. Can the authors evaluate the method on another video reasoning benchmark or provide examples involving more complex, unseen state transitions?

  3. How significant are the benefits of persistent segment tree maintenance in an end-to-end setting that includes visual observation and entity binding?

1 comment

Thank you for examining the contract scope and the practical role of persistence.

W1, Q1 & Q2. Contract coverage and generalization.

Typed contracts define supported state fields and operations. A new state field or primitive requires an adapter specifying operands, state access, update semantics, and unresolved-evidence handling. Within supported contracts, UNKNOWN preserves unresolved information. The transition interface and composition rule operate across these explicitly defined adapters.

Appendix E evaluates 128 new combinations of supported operations under complete observations: exact query accuracy is 100.00% for EviTrans and 62.50% for the comparator. Longer-history diagnostics contain 32 questions at each of three lengths, with EviTrans at 100.00% throughout. These results establish compositional execution under complete observations.

CATER Task 3 supplies an independent 36-class final-location task. A container movement affects the snitch according to the containment chain at interval entry. A controlled comparison requires RGB inputs, an adapter fixed using development data, and shared predicted observations for the state-based variants.

Table R3: CATER Task 3 results.

MethodTop-1 (%)
Direct Qwen3-VL-8B19.75
Chronological facts + 8B LM41.19
Explicit state + 8B LM63.66
EviTrans: sequential execution71.65
EviTrans: persistent execution71.65

The CATER results extend the end-to-end evaluation to an independent video benchmark, complementing VSTAT and the controlled execution diagnostics.

W2 & Q3. Practical benefit of the persistent backend.

Persistent segment trees are standard data structures; their role here is maintaining versioned compositions of executable transitions. The expanded fixed-binding audit records 100.00% state and answer agreement over 45,924 query checks against independent replay. The identical audited-subset scores of C and D reflect their shared execution semantics.

Reviewer_ohmiOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

EviTrans treats a video interval as a program that maps an entry world state and boundary memory to an updated state. The proposed pipeline connects a typed query contract, visual entity binding, event decoding, a World State Executor, deterministic query programs, and a persistent segment tree. On VSTAT (834 videos and 1,500 questions, including 31 unavailable inputs scored as failures), the frozen system obtains an overall score of 51.11 and outperforms the direct VQA baselines reported in the paper. The authors also separate two kinds of evidence: a component comparison on 356 audited high-quality questions from 211 videos, and backend measurements of edit latency, retained memory, and historical queries under controlled replacement workloads.

Critical strengths
  1. The state-transition formulation is well specified. The paper retains entry state, boundary memory, and unresolved information, allowing an interval transition to be executed under different histories. This is more suitable for repeated observations, entity deduplication, and historical revision than storing a fixed realized state change for each interval.

  2. The evaluation covers both answer quality and backend maintenance cost. In addition to VSTAT accuracy, the paper measures version retention, historical queries, edit latency, and memory. The state-agreement and historical-query checks in the appendices strengthen the implementation evidence.

Critical weaknesses
  1. The component ablation does not identify component effects on the full benchmark. Table 2 changes the system only for the 356 audited high-quality questions from 211 videos. Predictions for the other 1,113 available-input questions are copied from the original EviTrans run, and the 31 unavailable-input questions remain failures. The A-E numbers therefore support a conditional comparison on the audited subset, rather than a full-benchmark attribution of the gains to state maintenance, deterministic queries, or the observer. The high-quality subset scores 87.36, while the reported complementary population scores 39.83; that population also contains the 31 unavailable questions, so the gap should not be read as a clean estimate of the effect of observation quality alone.

  2. The persistent-tree efficiency result is narrower than the headline speedup suggests. The main measurement uses 211 videos, 32 record replacements per video, and 33 retained versions; the expanded check uses 64 replacements and 65 versions. These runs measure backend maintenance, state execution, and the original queries, but exclude visual re-observation and recompilation of dependent records after an evidence or entity-binding change. The reported 22.2% latency and 52.1% memory reductions are therefore valid for the stated leaf counts, replacement pattern, and caching policy. The paper should make clear how much of the end-to-end revision cost remains outside this measurement.

Questions for authors
  1. Can you report component ablations on all 1,500 questions, or at least report A-E separately for the high-quality and lower-quality groups? If not, why is the component comparison restricted to 356 questions?

  2. Variants C and D have identical answer scores, while the backend measurements exclude visual re-observation and dependent-record recompilation. Should the persistent segment tree contribution be stated as version maintenance and backend efficiency, and could recompilation costs exceed the measured benefit in typical use?

  3. Can you provide the input-availability manifests, prompts, model-call budgets, per-video baseline results, code, evaluation logs, and raw measurements underlying Tables 3 and 15, together with uncertainty estimates for paired differences?

1 comment

Thank you for the careful assessment of component attribution and measurement scope.

W1 & Q1. Scope of the component analysis.

The audit defines 356 questions from 211 videos for component analysis conditional on usable front-end evidence. A-D share cached observations, and A/B use the same 32B answer model. Table R1 distinguishes these component scores from the independently evaluated full-system score.

Table R1: Component scores and resolved-answer coverage.

VariantHQ scoreHQ coverage (%)Full score
A: Facts + LM72.5394.1042.96
B: Explicit state + LM85.0697.4748.98
C: Sequential execution87.3696.9151.11
D: Persistent execution87.3696.9151.11
E: VLM-only observer0.841.400.67

Within the audited subset, explicit state raises the score from 72.53 to 85.06, and deterministic queries raise it to 87.36. The original Table 2 aggregates these subset interventions with unchanged D predictions for the other 1,113 available-input questions. Thus, these results demonstrate the benefits of the proposed approach within the audited subset.

The complementary score of 39.83 covers 1,144 questions, including the 31 missing inputs; removing their zero contribution gives approximately 40.94 for the 1,113 available-input questions. The groups also differ in source and task composition, so their score gap is a descriptive comparison. E's 5/356 resolved answers identify a severe adapter-validity problem; quantifying the contribution of auxiliary experts requires a functioning adapter.

W2 & Q2. Persistence and revision cost.

Persistence preserves sequential execution semantics while retaining historical versions. The base workload contains 211 videos, 13-128 leaves per video, 32 edits per video, and 33 retained versions.

Table R2: Backend costs under the base replacement workload.

MetricBlockedPersistent
Edit-to-answer (ms)0.23500.1828
Historical read (ms)0.03220.1427
Retained objects (MiB/video)1.7300.828

The edit-to-answer saving is 0.0522 ms (22.2%), and retained memory decreases by 52.1%. Historical reads favor blocked execution under the measured caching policy. The timings include new-leaf compilation, maintenance, execution, and queries.

Visual re-observation and recompilation of other dependent records add costs outside this measurement. A binding correction can invalidate multiple records, so dependent recompilation can exceed the backend saving. Historical correctness under such corrections requires version-specific evidence and binding contexts.

Q3. Reproducibility and paired uncertainty.

We will release the complete code, input-availability manifests, prompts, model-call budgets and usage, per-video baseline results, evaluation logs, and raw timing and memory measurements underlying Tables 3 and 15. Both tables report the same base experiment. We will also provide 95% confidence intervals for paired differences using a video-level paired bootstrap, resampling videos jointly across methods and keeping each video's questions together.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.