acceptodds
Withdrawn from ICLR 2027

Criterion-guided Evidence Chain Composition and Symbolic Reasoning for Parent-Child Interaction Video Assessment

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Parent-child interaction (PCI) video assessment requires determining whether caregiver and child behaviors jointly satisfy standardized criteria governing participant roles, target identity, and temporal relations. Relevant cues are distributed across time and modalities, so retrieving relevant segments alone is insufficient to establish whether these constraints are jointly satisfied. Moreover, existing video understanding models may miss relevant behavioral cues in long videos, risking the misinterpretation of missing observations as evidence that a behavior did not occur. To address these challenges, we introduce EviChain, a framework for criterion-guided evidence chain composition and symbolic reasoning over PCI videos. EviChain organizes multimodal observations into a reusable pool of interaction chains, supporting multiple assessment criteria without repeating the initial perception stage. Each criterion is compiled into a Question Logic Program (QLP) that specifies the required evidence roles and logical constraints, guiding retrieval and incremental composition of complementary chains to fill gaps in the evidence required for execution. Sufficiency-aware three-valued symbolic execution distinguishes support, refutation, and insufficient evidence, mitigating the risk of missing observations being treated as negative evidence. Only unresolved cases are passed to an evidence-conditioned video-language verifier. We also introduce PCI-Bench, a benchmark for standardized PCI assessment comprising 243 real-world videos averaging 153 seconds in duration and 17,739 criterion-level assessments. On this benchmark, EviChain achieves 69.90% accuracy and 63.75% Macro-F1, exceeding the strongest evaluated baseline for each metric by 2.40 and 4.52 percentage points, respectively.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
long-form video understanding, multimodal evidence composition, neuro-symbolic reasoning, temporal logic, parent-child interaction assessment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GfUwOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes EviChain, a criterion-guided evidence chain reasoning framework for parent-child interaction video evaluation. The approach first extracts atomic behavioral events via multimodal perception to construct reusable interactive evidence chains. Evaluation criteria are then compiled into Question Logic Programs (QLP) to guide the retrieval of relevant evidence, role binding and complementary combination. Three-valued symbolic reasoning is adopted to distinguish among support, refutation and insufficient evidence; for undecided cases, a video-language model is invoked for further verification.

The authors also build PCI-Bench, a dataset containing 243 videos and 73 evaluation criteria. Experiments show that EviChain achieves an accuracy of 69.90% and a Macro-F1 of 63.75%, outperforming existing baselines and verifying the effectiveness of structured evidence composition and symbolic reasoning.

Critical strengths
  1. This work introduces the Video Agent paradigm into domain-specific video evaluation tasks. A major advantage of this work is that it formulates the multi-stage decision pipeline of "perception – retrieval – evidence composition – reasoning – verification" for long videos in a Video-Agent-like manner, further integrating professional standards and structured criteria for parent-child interaction evaluation.

Compared with directly prompting general video large models for answers, EviChain can actively seek and supplement required evidence according to specific evaluation criteria. It is better suited for domain video tasks that demand professional rules, interpretable judgments and cross-temporal behavioral analysis, and demonstrates the potential of transferring Video Agent approaches to specialized scenarios such as clinical and behavioral assessment.

  1. The evidence organization and reasoning mechanism are rigorous.

Instead of merely retrieving relevant video clips, EviChain leverages QLP to perform role binding and complementary composition for evidence scattered across different timestamps and modalities. It explicitly examines participants, targets, and temporal relations. For this reason, it is more suitable for complex behavioral evaluation than vanilla Top-K retrieval or direct MLLM judgment.

Critical weaknesses
  1. The overall method leans toward engineering-oriented design with relatively limited core technical innovations.

The primary contributions of EviChain lie more in system integration: it chains multi-modal perception, event graphs, evidence chain retrieval, QLP, three-valued logic, and the VLM verifier into an end-to-end pipeline. While this combination proves effective for the target task, many of its components have been investigated in prior research. Therefore, this work is more of a system design and engineering integration tailored to a specific problem, rather than a novel foundational model, learning mechanism, or theoretical framework. As can also be observed from the paper’s Related Work section, it builds upon existing ideas including long-video retrieval, neuro-symbolic reasoning, MMR, and three-valued logic.

  1. The task and validation scope are relatively narrow, and the generalizability remains unproven.

The proposed method is only validated on the specific domain of parent-child interaction video evaluation and the authors’ self-constructed PCI-Bench dataset. QLP, role definitions, event types, and the sufficiency checker are tightly coupled with standardized PCI criteria. As a result, migrating this framework to other video domains will likely require redesigning the event taxonomy, logic programs, and decision rules. Although the paper claims that the framework “may extend to other clinical and behavioral video assessment tasks”, this currently represents a potential future direction rather than an experimentally validated conclusion.

Questions for authors
  1. Could the authors clarify and explicitly state which modules represent their original innovations, and which modules are adapted from prior work and applied to this domain-specific task?

  2. The method is only validated in the parent-child interaction evaluation scenario. Could the authors further fully demonstrate its cross-domain generalizability and transferability?

2 comments
AuthorsOpenReview ↗

Response to Reviewer GfUw

We thank the reviewer for the careful assessment of our work. We clarify which components are adopted from prior work and which constitute our methodological contributions, supported by controlled ablations. We also add an external IT-HOME evaluation.

1. Which components are original, and which are adapted from prior work?

We use existing perception/VLM backbones, MMR, and Strong Kleene truth tables. Our methodological contributions are criterion-guided relational evidence composition and sufficiency-aware reasoning with failure-trace-guided verification.

Criterion-guided relational evidence composition. We formulate evidence acquisition as constrained combinatorial assembly: QLPs guide the selection of complementary chains by balancing required-role coverage against redundancy and participant, target, and temporal conflicts, followed by joint cross-chain binding and execution (Eqs. 6–7). The objective is to assemble observations that jointly satisfy a criterion, beyond their individual relevance. With the observations and candidate pool fixed, full composition improves accuracy over Top-3 union by 8.58 points. Crucially, local role expansion achieves comparable symbolic coverage but falls 9.74 points behind in Macro-F1, showing that filling required roles alone does not ensure reliable assessment. Removing conflict control further reduces Macro-F1 by 2.86 points (Table 3). These controls support the importance of both evidence complementarity and relational compatibility.

Sufficiency-aware reasoning and targeted verification. We couple three-valued execution with criterion-specific evidence sufficiency to distinguish incomplete observations from behavioral refutation. Unresolved cases retain their composed evidence, partial role bindings, and predicate-level failure traces for VLM verification (Eq. 8). Symbolic execution thus determines both which conclusions can be accepted and what reasoning context accompanies unresolved judgments. Compared with an AST-conditioned VLM answering every pair, this policy improves accuracy and Macro-F1 by 5.41 and 1.66 points, respectively, while requiring final VLM decisions for only 13.79% of assessments (Table 4). With accepted symbolic predictions fixed, verification also improves over assigning Unknown to No by 3.85 and 1.73 points. Separately, adding AST traces to otherwise identical verifier inputs improves both metrics on the fixed unknown subset (Table A10, M4→M5).

2. Can the approach transfer beyond PCI-Bench and its 73 criteria?

To test transfer beyond PCI-Bench and its 73 criteria, we obtained additional recordings and corresponding labels from the authors of Xu et al. (2024) and conducted an external evaluation based on IT-HOME. IT-HOME is an established standardized inventory of the caregiving environment. We selected 30 videos and 14 video-assessable observational items from its 45-item inventory, yielding 420 binary assessments. We generated QLPs for these new criteria and evaluated EviChain and all ten baselines on the same assessment pairs, testing the approach under changes in both the recordings and the assessment requirements. Table 1 reports all ten baselines; selected comparisons are summarized below:

MethodIT-HOME Acc. (%)IT-HOME Macro-F1 (%)
TSPM45.0042.03
Qwen2.5-VL-7B55.2451.54
VideoLLaMA2.1-7B68.3346.86
Qwen2.5-Omni-7B66.4358.35
EviChain79.7667.34

As detailed in Appendix A.4, the graph-construction weights, retrieval parameters, and family-conditioned sufficiency thresholds are transferred from PCI-Bench without retuning. Only the criterion-specific QLPs are generated anew using the same compilation procedure, without access to reference labels. EviChain exceeds the strongest baseline for each metric by 11.43 accuracy points and 8.99 Macro-F1 points. These results demonstrate applicability to a different standardized set of behavioral criteria without dataset-specific parameter tuning.

Our focus is on criterion-based behavioral video assessment, where standardized criteria specify not only which behaviors matter, but also the evidence and relations needed to justify a judgment. This structure is shared by parent–child interaction assessment, caregiving-environment evaluation, and potentially video-observable items in clinical assessment scales. EviChain separates these criterion-specific requirements from the procedures for composing and evaluating evidence, allowing different standards to guide the same reasoning framework. The IT-HOME experiment provides evidence for transfer across assessment standards within the caregiving and behavioral domain.

Reviewer_GfUwOpenReview ↗

Exclusively human-written assessment of the paper

将下面的审稿意见翻译为英语:

任务和验证范围较窄,通用性尚未得到证明。目前方法只在亲子交互视频评估这一特定领域、以及作者构建的 PCI-Bench 上进行验证。QLP、角色定义、事件类型和 sufficiency checker 都与标准化 PCI 准则结合得很深,因此迁移到其他视频领域时,很可能需要重新设计事件体系、逻辑程序和判定规则。论文虽然声称框架“may extend to other clinical and behavioral video assessment tasks”,但这目前更多是潜在方向,而不是实验已经证明的结论。

Reviewer_gfJuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces EviChain, a framework for parent–child interaction video assessment, that integrates criterion-guided evidence chain composition and sufficiency-aware symbolic reasoning to assess if standardized criteria of PCI are satisfied. The paper also introduces PCI-Bench, a benchmark for standardized PCI assessment. On this benchmark, EviChain achieves 69.90% accuracy and 63.75% Macro-F1, outperforming the strongest baselines by 2.40% and 4.52% respectively.

Critical strengths
  • Retrieving relevant events individually does not guarantee that they collectively satisfy a criterion; criterion-guided evidence chain composition addresses this by combining related observations. Sufficiency-aware three-valued symbolic reasoning allows the system to distinguish between evidence supporting a criterion, evidence refuting it, and insufficient evidence to reach a conclusion.
  • PCI-Bench contains 243 real-world videos, 73 standardized behavioral criteria, and 17739 (=243x73) binary video–criterion assessment pairs. It is a fine-grained social interaction understanding video benchmark instead of a conventional video QA benchmark.
Critical weaknesses
  • EviChain appears to be more of an engineered system integrating existing techniques than a novel research method. While this does not diminish its value as a framework, the limited 2.4% accuracy improvement over existing video understanding and reasoning method TSPM raises questions about whether claimed contribution is justified.
  • The authors do not evaluate the production VLM independently on the full benchmark, but only on the 2446 “Unknown” cases after symbolic reasoning, making it difficult to see whether the symbolic reasoning component provides meaningful improvements over a plain VLM.
Questions for authors
  • What would be the performance difference if we give the same production VLM all 17739 video–criterion pairs and ask it to answer Yes/No, without EviChain's symbolic pipeline, instead of passing through symbolic True/False decision first and only the 2446 Unknown cases passed to the verifier?
1 comment
AuthorsOpenReview ↗

Response to Reviewer gfJu

We appreciate the reviewer's comments and suggestions. We clarify the production VLM's performance on the full benchmark and add pre-AST and AST-informed evidence controls using the same backbone.

1. How does the same production VLM perform on all 17,739 pairs?

The backbone is Qwen2.5-VL-7B-Instruct. On all 17,739 PCI-Bench pairs, direct video–question inference achieves 58.72% accuracy and 58.13% Macro-F1, compared with EviChain's 69.90% and 63.75%. The gains are therefore 11.18 and 5.62 percentage points. This direct baseline was included in the original main table. Revised Table 4 places it alongside the new pre-AST and AST-informed controls, explicitly identifying the shared backbone:

Configuration, all 17,739 pairsPairs receiving a final VLM decisionAcc. (%)Macro-F1 (%)
Direct video reasoning100%58.7258.13
Chain evidence without AST execution100%63.2643.28
Chain evidence with AST execution (EviChain with all-pair verification)100%64.4962.09
Full EviChain13.79%69.9063.75

The pre-AST control directly tests whether supplying our evidence to the same VLM is sufficient without symbolic execution. It improves accuracy but reduces Macro-F1. Incorporating AST execution improves both metrics, yet letting the VLM make every final decision still trails EviChain by 5.41 points in accuracy and 1.66 points in Macro-F1. Thus, the benefit extends beyond supplying a richer prompt: accepted symbolic conclusions contribute directly to the final answers.

2. What is the technical contribution beyond integrating existing techniques?

Our first contribution formulates criterion-guided evidence acquisition as constrained relational composition. QLP-guided assembly optimizes required-role coverage against redundancy and relational conflicts (Eq. 6), then binds and executes observations jointly across chains (Eq. 7). A retrieved caregiver prompt and child response constitute valid evidence only when their participant, target, and temporal relations satisfy the criterion under a joint binding. Table 3 tests this formulation with the same observations and QLPs: full composition improves accuracy over Top-3 union by 8.58 points; local role expansion achieves slightly higher coverage but falls 9.74 points behind in Macro-F1; and removing conflict control reduces Macro-F1 by 2.86 points (Table 3(b)).

Our second contribution couples sufficiency-aware execution with evidence- and failure-trace-conditioned verification. Execution distinguishes insufficient evidence from behavioral refutation and passes unresolved cases to the VLM together with the composed evidence bundle, partial role bindings, and predicate-level failure trace. This preserves the reasoning context and identifies what remains unresolved for subsequent verification. With accepted symbolic predictions fixed, conditional verification improves over assigning Unknown to No by 3.85 accuracy points and 1.73 Macro-F1 points (Table 4(b)). Separately, adding AST traces to otherwise identical chain-based inputs improves accuracy and Macro-F1 on the fixed unknown subset by 6.74 and 4.86 points, respectively (Table A10, M4→M5). These controls support both preserving uncertainty during execution and using explicit execution feedback to inform its resolution.

3. Is the improvement over TSPM substantial enough to support the contribution?

The 2.40-point accuracy gain captures only part of the comparison. Against the same TSPM baseline, EviChain improves Macro-F1 by 9.36 points (63.75% versus 54.39%). Table A9, reproduced below, shows gains across all three evidence-requirement groups, with the largest gains on global coverage and exclusion criteria (C3). Here, isolated behaviors cannot establish criterion satisfaction, nor can missing observations establish absence—the two difficulties addressed by relational evidence composition and sufficiency-aware reasoning.

Evidence requirementTSPM Acc.EviChain Acc.Δ Acc.TSPM Macro-F1EviChain Macro-F1Δ Macro-F1
C1: Atomic evidence69.1469.53+0.3955.3463.01+7.67
C2: Compositional and temporal reasoning60.3262.10+1.7850.6658.62+7.96
C3: Global coverage and exclusion80.4087.01+6.6160.6177.12+16.51

The new external IT-HOME evaluation further yields 79.76% accuracy and 67.34% Macro-F1, exceeding the strongest baseline for each metric by 11.43 and 8.99 points. Together with the same-backbone and fixed-evidence controls, these results support a contribution that is broader than a small aggregate accuracy increase on one benchmark.

Reviewer_y8Z5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper automates PCI (parent-child interaction) scoring against a standardized checklist of 73 items (yes/no questions). The proposed EviChain system has four stages: (1) five perception models are run on each video, (2) an LLM translates each checklist item into a small program ("question logic program"), (3) for each video + item, chains are combined and used to fill roles, (4) the program is executed to get a yes/no/unknown output (for unknown, a final VLM is applied to resolve to yes/no). The authors also introduce PCI-Bench to measure this, with 243 sessions and 17.7k yes/no labels (w/63% yes) - data is available on request and not public (which is understandable but makes it hard to assess this contribution's quality).

Critical strengths

The setting applies AI to a real and clinically grounded setting, with 73 items on 243 real sessions. In general, this seems like it could be potentially an interesting domain for further study.

Critical weaknesses

It's unclear if the improvement reported for the method actually maps to its components (I found it hard to trace which model is actually used in each of the stages). PCI-Bench is also not public as far as I can tell, so it is hard to assess the quality of this benchmark and assessment.

Questions for authors

Please clarify the points raised in weaknesses above, thanks!

1 comment
AuthorsOpenReview ↗

Response to Reviewer y8Z5

We thank the reviewer for the helpful feedback. We have added a module-level configuration table and candidate-construction ablations to clarify the implementation and the contribution of individual components. Below, we also explain the benchmark's annotation and access procedures.

1. Which models are used at each stage?

We have added Table A6 to explicitly identify the models and configurations used throughout EviChain. For clarity, the stage-to-model mapping is summarized below:

StageModel / implementation
Speech and acoustic perceptionQwen3-ASR-1.7B-hf, FSMN-VAD, CAM++, and a wav2vec2 age/gender model for speaker-role cues
Gaze perceptionGazelle with a DINOv2 ViT-L/14 backbone
Body/hand pose and hand–object interactionsRTMW; SAM3 with hand tracking and deterministic event fusion
Offline QLP compilationdeepseek-v4-flash-0731 via the Bailian API
Candidate construction, evidence assembly, and symbolic executionExplicit graph construction, QLP-guided composition, and three-valued execution with structural sufficiency checks
Conditional verificationQwen2.5-VL-7B-Instruct

Perception and verification weights are frozen. On PCI-Bench, method parameters are selected on a separate 10-video development set excluded from evaluation. QLPs are compiled offline for each criterion. Table A6 additionally specifies parameter selection and what is held fixed or varied in each ablation, connecting the implementation details to the component-level comparisons in Tables 2–4.

2. Do the gains map to the proposed components?

The revised experiments separately examine perceptual inputs (Table 2), candidate construction and evidence composition (Table 3), and symbolic decision-making and verification (Table 4).

The results distinguish evidence structure from simply obtaining more symbolic answers:

  • Candidate construction: simplifying chain construction reduces final accuracy by 2.72–3.21 points and Macro-F1 by 2.50–3.30 points, although aggregate symbolic coverage remains close to the full method. This links final performance to candidate organization, rather than coverage alone.
  • Composition: Top-3 union reaches only 30.63% symbolic coverage and 61.32% final accuracy, versus 86.21% and 69.90% for full composition. Local role expansion reaches comparable coverage (86.77%) but only 54.01% Macro-F1, versus 63.75%. Covering roles is therefore insufficient without selecting jointly compatible evidence.
  • Decision policy: with the same Qwen2.5-VL backbone, giving AST-informed evidence to the VLM for every pair achieves 64.49% accuracy and 62.09% Macro-F1. Retaining accepted symbolic decisions and verifying only unresolved cases improves these to 69.90% and 63.75% (Table 4).

These controls identify contributions beyond the choice of pretrained perception models or verifier.

3. Benchmark quality and access

PCI-Bench contains expert-verified criterion-level labels, with development and evaluation separated at the video and family levels. Section 4.1 describes expert voting and subsequent verification. Appendix A.1 reports the construction from 255 source sessions: two with incomplete labels are excluded, 10 are reserved for development, and 243 enter evaluation. Table A1 reports label balance, durations, and activity distribution; Table A2 provides representative criterion structures; Appendix A.3 defines the evaluation metrics.

The recordings are not publicly downloadable because they contain identifiable parent–child interactions. The Reproducibility Statement describes the access-by-request arrangement, and the manuscript provides an anonymous code repository and implementation details.

To assess whether our findings extend beyond PCI-Bench, we obtained additional recordings and corresponding labels from the authors of Xu et al. (2024) and conducted an external evaluation using IT-HOME, an established standardized inventory of the caregiving environment. Appendix A.4 documents the data source, item selection, and evaluation protocol: 30 videos are assessed against 14 video-assessable observational items, yielding 420 binary assessments. We evaluated EviChain and all ten baselines on the same pairs, retaining the parameters selected on PCI-Bench without retuning. Table 1 reports the complete comparison; selected results are reproduced below:

MethodIT-HOME Acc. (%)IT-HOME Macro-F1 (%)
TSPM45.0042.03
Qwen2.5-VL-7B55.2451.54
VideoLLaMA2.1-7B68.3346.86
Qwen2.5-Omni-7B66.4358.35
EviChain79.7667.34

4. Privacy and the face visible in Figure 1

Thank you for flagging this. We have applied stronger face masking to Figure 1. The Ethics Statement records institutional ethics approval for the multicenter study; the committee name and approval identifier are withheld during double-blind review.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank the reviewers for their constructive feedback. We have revised the manuscript to clarify the methodological contributions and added experiments addressing evidence composition, decision-making, and applicability beyond PCI-Bench.

  • Methodological clarification. We have clarified the distinction between established building blocks and our contributions: criterion-guided relational evidence composition, and sufficiency-aware execution coupled with evidence- and failure-trace-conditioned verification. The revised presentation explains how criteria govern both evidence assembly and decision acceptance.

  • Controlled experiments (Tables 3–4). We have added candidate-construction ablations and same-backbone controls using pre-AST and AST-informed evidence. The reorganized experiments distinguish compatible evidence composition from retrieval or role filling, and conditional verification from delegating every final decision to a VLM.

  • External evaluation (Table 1; Appendix A.4). We have added an IT-HOME evaluation on 30 videos and 14 video-assessable observational items. EviChain achieves 79.76% accuracy and 67.34% Macro-F1, outperforming all evaluated baselines on both metrics. This provides evidence of applicability across assessment standards within the caregiving and behavioral domain.

  • Implementation transparency and privacy. Table A6 identifies the models, parameter-selection procedures, and ablation settings. We have expanded the module descriptions in the appendix and applied stronger face masking to Figure 1.

Detailed responses and supporting comparisons are provided in the individual reviewer threads. All section and table references refer to the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.