acceptodds
Withdrawn from ICLR 2027

DrGait: Biomechanically Grounded Visual Reasoning for Interpretable Clinical Gait Analysis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Current automated gait analysis for clinical applications often relies on black-box classifiers with limited interpretability. Although Vision-Language Models (VLMs) offer strong reasoning capabilities, applying them directly to gait videos can produce unsupported explanations, because they struggle to measure subtle geometric deviations from raw visual contexts. To address this, we introduce DrGait, a training-free agentic framework that shifts the VLM's role from a direct visual reasoner to a clinical planner. DrGait decouples semantic reasoning from geometric perception through a structured Triage-Verification-Synthesis (TVS) workflow. Given an input video and a set of basic spatiotemporal metrics, the DrGait agent first performs a heuristic triage to propose hypotheses on gait patterns, which are then verified by autonomously calling deterministic biomechanical tools that operate on reconstructed 3D mesh trajectories, segmented 2D pose tracks, and event-centered video evidence. Finally, a closed-loop mechanism recursively updates the agent's reasoning context based on the feedback. By anchoring VLM reasoning in verifiable geometric and temporal measurements, DrGait reduces hallucinations and improves gait classification, while generating transparent and interpretable clinical reports that greatly assist in clinician's diagnosis in practice.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Gait Analysis, Vision-Language Models, Interpretability, Multimodal Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_HSX9OpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies interpretable clinical gait analysis with vision-language models (VLMs). This paper proposes DrGait, a training-free harness framework for verifying VLM hypotheses with biomechanical tools. This paper reports higher classification accuracy and better report quality than direct VLM baselines on two clinical gait datasets.

Critical strengths
  1. Good Motivation: VLMs often hallucinate geometric measurements in gait videos. Using the VLM as a planner and deterministic tools for measurement is a reasonable design.

  2. Interpretable and Low-Cost: Each conclusion comes with tool-based evidence and a full reasoning trace. Human evaluators prefer DrGait reports in 73.4% of GAVD cases. DrGait with Gemini-3-Flash costs about the same as direct Gemini-3.1-Pro and performs better.

Critical weaknesses
  1. Unreliable Results: (1) Sec. 4.4 reports 92.4%/51.4%, but Tables 1–2 report 93.2%/52.8%. (2) Sec. 4.5 says removing the dictionary drops 7.9%/19.5%, but Table 7 shows 11.5%/15.9%. (3) In Table 1, the baseline numbers match the class sizes (e.g., Gemini-3.1-Pro: 89/107=0.832 , 25/27=0.926 , 114/134=0.851). However, DrGait’s 99.3% specificity seems hard to obtain with 27 normal videos, and its 93.2% overall accuracy does not seem to match either. Please clarify these results again.

  2. Limited Evidence for the Agentic Loop: (1) Triage-only already reaches 44.9% subtype accuracy, so most of the gain seems to come from the metrics and prompting. A baseline that gives the VLM the same metric text would help. (2) DrGait runs only 1.06 rounds on average, yet removing recursion drops accuracy by 8.9%. Please explain this gap.

Questions for authors

My main concern is unreliable numbers in this paper.

2 comments

Dear reviewer,

As the rebuttal period is approach to the end, we would like to kindly check if our rebuttal has sufficiently addressed all your concerns and questions. Please feel free to let us know if any further clarifications, explanations or experiments are needed, and we will be more than happy to discuss further in any aspect.

Again, thank you so much for all your attention and efforts!

Thanks for the response, but my concerns are not resolved.

First, attributing the inconsistencies to typos is not a convincing explanation. It instead reinforces my concern that the reported numbers are unreliable. I understand that the inconsistencies may come from mixing results across multiple versions of experiments. However, the final paper should be clear and consistent. Otherwise, it heavily confuses readers, such as me. I suggest revising the whole paper.

Second, the numbers in the response (96.3%, 92.5% and 51.4%) cannot be found in Tables 1 and 2. I have no idea where these numbers come from. And the response also does not explain how DrGait’s numbers in these tables were obtained. In Table 1, the baseline numbers match the class sizes (e.g., Gemini-3.1-Pro: 89/107=0.832, 25/27=0.926, 114/134=0.851). However, DrGait’s 99.3% specificity and 52.8% subtype accuracy cannot be obtained with 27 normal and 107 abnormal videos (26/27=96.3% or 27/27=100%; 56/107=52.3% or 57/107=53.3%). Therefore, DrGait’s numbers in Table 1 are not reliable.

So I will lower my rating to 1.

Reviewer_YrBAOpenReview ↗
Rating
1(−2)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

DrGait is a training-free framework for video-based gait analysis using a Triage Verification Synthesis workflow. A vision-language model proposes dictionary-constrained gait hypotheses from video and initial gait measurements, selects tools that compute additional measurements from estimated poses or reconstructed meshes and retrieve event-centered visual evidence, and synthesizes the observations into an assessment and report. The process can iterate to refine unresolved hypotheses. The evaluation covers general pathological gait analysis on GAVD and prosthetic gait deviations on ProGait, using classification metrics, LLM-based semantic evaluation, human report evaluation, component ablations, an additional VLM planner, and pose-perturbation experiments. The reported results suggest improvements over direct VLM inference, although several aspects of the evaluation and the claimed reliability of measurement-based reasoning require clarification.

Critical strengths

Strengths 1. A well-motivated separation between high-level reasoning and explicit measurement.

DrGait delegates numerical measurement to external tools rather than relying on a VLM to infer precise quantities directly from video. The Triage–Verification–Synthesis workflow connects candidate hypotheses to explicit verification objectives, records the rationale for each tool call, and organizes supporting and conflicting evidence in structured outputs. This makes the intermediate assessment process more inspectable than a final prediction accompanied only by a free-form explanation. The paper also explicitly distinguishes normalized 2D image-plane proxies from calibrated 3D measurements. These design choices provide a useful basis for examining which observations support an assessment and where uncertainty remains, even though inspectability alone does not establish measurement accuracy.

Strengths 2. Evaluation beyond classification, with component studies and substantial supporting documentation.

The evaluation covers general pathological gait analysis on GAVD and prosthetic gait analysis on ProGait, and considers report evaluation in addition to classification. The authors also investigate individual components, an additional VLM planner, and sensitivity to pose perturbations. In Table 7, the full system reports higher GAVD abnormal-subtype accuracy than the triage-only variant, 52.8% versus 44.9%, and a higher ProGait LLM-judge semantic-agreement score, 1.98 versus 1.30. These comparisons provide initial empirical support for the usefulness of the verification stage beyond the initial triage context. Appendix B.1 also reports improvements over direct inference when Qwen3.6-Plus replaces the main planner, suggesting that the observed gains are not confined to the primary backbone configuration.

Critical weaknesses

Weaknesses 1. Several central evaluation results are insufficiently specified, and the report-quality comparison may conflate reasoning quality with output requirements. Table 4 reports “Overall” and “Best Win %” without sufficiently explicit definitions of their construction and aggregation. Section 4.3 describes three rating dimensions and selection of the best report, and mentions excluding missing responses, but this does not fully specify how the reported statistics are calculated across cases and evaluators. Similarly, Tables 9 and 10 report “label retention,” “inconclusive,” and “abstention” without defining the compared outputs, case-level counting rules, denominators, or treatment of these outcomes in accuracy. Appendix A refers to “agreement coefficients” without identifying the statistic or presenting the corresponding results. These omissions concern the interpretation and reproducibility of the evaluation, rather than demonstrating that a particular reported number is mathematically incorrect.

The human report comparison also needs clarification. The GAVD DrGait prompt explicitly requests a final report containing a conclusion, rationale, and suggested action or next step. The direct baseline prompt instead requests a hypothesis and brief reasoning/uncertainty notes, without an equivalent final-report requirement. Since human evaluation includes clinical usefulness and report preference, different output requirements could contribute to the observed preference. The paper should explain whether the displayed reports were generated under matched instructions or underwent a common report-generation step. This is a potential confound, not evidence that the human evaluation is necessarily invalid.

There are also reporting inconsistencies that affect interpretation. In Section 4.3, values from the Diagnostic column are described as overall scores. In Section 4.5, the stated performance decreases for removing the dictionary do not correspond to the “No dictionary” row, and the prose describes the one-pass ProGait judge score as higher even though the table reports 1.84 versus 1.98 for the full system. These should be reconciled with the underlying evaluation outputs. Collectively, the missing definitions and inconsistent descriptions make it difficult to assess the strength of the preference, robustness, and component-level conclusions.

Weaknesses 2. The reliability of the measurement-to-interpretation chain is not sufficiently established to support the strongest grounding claims.

Deterministic computation on an estimated pose does not by itself establish the accuracy of the resulting measurement or the validity of its interpretation. Appendix B.2 usefully investigates sensitivity to pose perturbations, but it fixes the reasoning plans. Consequently, it does not directly establish absolute measurement accuracy against a reference or characterize error propagation through the complete adaptive workflow. The concern is not that the method must use SKEL instead of SMPL or 2D poses; it is whether the selected measurements are sufficiently defined and validated for the conclusions drawn from them.

A concrete inconsistency appears in the GAVD case trace. Appendix K defines get_foot_clearance as normalized by body height. Appendix D reports a tool result of “0.76,” but the subsequent internal synthesis interprets it as “0.76m” and uses it to support a gait interpretation. This inconsistency occurs in the internal synthesis evidence, not in the final user-facing report. It nevertheless requires reconciliation: the tool declaration, recorded output, and interpretation should agree in a system whose central contribution is auditable numerical evidence. The example does not establish the overall frequency of such errors, but it shows that tool use alone does not guarantee correct interpretation.

The knowledge base introduces another reliability dependency. The “Toe-Out Asymmetry / Externally Rotated Foot” entry includes both outward-rotation recommendations and an inward-rotation recommendation, without stating conditions that distinguish their applicability. Since the method uses this knowledge to guide its outputs, the expert-verification process and handling of conflicting recommendations require a clearer account. This is a knowledge-consistency concern, not evidence of actual clinical harm.

I acknowledge that the paper explicitly limits its clinical interpretation and states that the system should not independently determine diagnosis, treatment, or prosthetic adjustment. Nevertheless, claims such as ensuring that reasoning is always biomechanically grounded are stronger than the current validation supports. The paper should distinguish inspectable tool-supported reasoning from independently validated measurement and interpretation.

Questions for authors

Q1. Evaluation definitions and aggregation. Please provide operational definitions of Overall and Best Win % in Table 4, including the statistical unit, denominators, aggregation across cases and evaluators, and missing-response handling. Please also define label retention and the case-level inconclusive/abstention statistics in Tables 9 and 10. Which agreement statistic supports the statement in Appendix A, and what are its results for the reported evaluator group? For Table 1, how are the prediction scores used to calculate AUROC obtained for each model and for the human reference?

Q2. Label normalization and scoring. How is a final prediction selected from the hypothesis list for scoring, and are confirmed, rejected, and inconclusive hypotheses treated differently? Appendix H includes Hemiparetic Gait under both Stroke and Cerebral Palsy: is evaluation based on a unique predicted dataset class or on membership in an acceptable-label set? For ProGait, how are repeated general categories and trials without a scoreable broad label handled in each metric? These details are necessary to determine precisely what the reported classification results measure.

Q3. Measurement and knowledge-base verification. What is the exact definition and unit of the foot-clearance value in Appendix D, and why is it interpreted differently from the tool declaration? Please clarify the measurement formulas and conventions used for central quantities such as the symmetry index and the body-oriented reference frame. What reference-based or expert-annotated checks were performed on these measurements and their downstream interpretations? Please also explain the expert verification of the knowledge base, including the apparently opposing recommendations in the toe-out entry.

Q4. Human report comparison. Were all systems asked to produce the same report components, including follow-up suggestions, before the human evaluation? How were the baseline notes and DrGait final reports converted into the text shown to evaluators? A comparison with matched report-generation requirements would help determine whether the usefulness and preference gains reflect stronger evidence-based reasoning rather than differences in the requested output.

Q5. Incremental contribution of adaptive verification. The existing ablations are informative, but how much of the improvement specifically requires adaptive tool selection and recursive synthesis, rather than supplying geometric measurements to a VLM? A comparison with a single-pass system given the same dictionary, a fixed set of relevant measurements, and matched report requirements would help isolate this contribution. Please also clarify whether the one-pass and recursive variants share their first-pass outputs or are evaluated through independent generations. The conceptual distinction would benefit from discussion of closely related approaches, including BioGait-VLM: A Tri-Modal Vision–Language–Biomechanics Framework for Interpretable Clinical Gait Assessment, accepted at MICCAI 2026, which also introduces explicit 3D skeletal information into vision-language gait assessment. The relevant distinction is the contribution of DrGait’s training-free, on-demand verification workflow, rather than the presence of biomechanical information alone.

2 comments

Dear reviewer,

As the rebuttal period is approach to the end, we would like to kindly check if our rebuttal has sufficiently addressed all your concerns and questions. Please feel free to let us know if any further clarifications, explanations or experiments are needed, and we will be more than happy to discuss further in any aspect.

Again, thank you so much for all your attention and efforts!

Thanks Authors for the response. The clarifications are helpful, but my concerns remains.

First, the corrected results are still inconsistent. In your reply to Reviewer HSX9, the counts give 92.5% binary accuracy and 51.4% subtype accuracy. However, the proposed correction to Section 4.4 and the ablation table in the same reply still report 93.2% and 52.8%. I understand that mistakes can happen during revision, but the corrections should agree with each other. Describing these as typos does not resolve the remaining inconsistency. I need a consistent set of results to assess the reported improvements.

Second, the explanation of the recursion experiment makes its contribution less convincing. In the full run, correct abnormal-subtype predictions increase from 54 to 55 out of 107, a net gain of one correct prediction. The separate one-pass run has 47 correct predictions, but you also explain that its first-pass outputs differ from those of the full run. Therefore, the larger difference between these runs cannot be attributed to recursion alone. This does not mean the overall workflow has no value, but the current comparison does not isolate the benefit of recursion.

Third, allowing triage and tool selection to change is a useful addition to the sensitivity test. However, the new table still reports performance under pose noise, rather than joint-angle or gait-event timing errors against an independent reference. I still do not see the requested evidence showing how accurate the measurements themselves are.

I still think the idea is reasonable, but these concerns affect the experimental evidence, not just the writing. After considering the rebuttal, I will lower my rating from 3 to 1.

Reviewer_pweEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DrGait is a training-free framework for gait analysis. A VLM proposes hypotheses, calls biomechanical tools, and revises its assessment through a Triage-Verification-Synthesis loop. Tools use reconstructed 3D trajectories, 2D poses, and event frames. Evaluation covers GAVD and ProGait, with classification, report ratings, ablations, and pose-perturbation tests.

Critical strengths
  1. A sensible division of labor. External tools supply numerical evidence, while the VLM plans and synthesizes. Supporting evidence, contradictions, and unresolved findings are exposed in the report. The explicit distinction between 2D proxies and calibrated measurements is valuable.

  2. Promising evidence for tool use. Removing verification tools reduces reported GAVD subtype accuracy from 52.8% to 44.9%. ProGait top-3 accuracy falls from 32.6% to 13.1% (Table 7). The additional Qwen planner also improves over direct inference. Detailed prompts and example traces help inspect the workflow.

Critical weaknesses
  1. Measurement validity remains unestablished. Deterministic tools operate on estimated poses. No independent reference errors are reported for key joint angles or gait-event times. Appendix B provides useful perturbation tests, but keeps reasoning plans fixed. It therefore tests synthesis stability, rather than robustness of the full adaptive pipeline. Blinded report preferences do not validate numerical measurements or establish clinical benefit. The clinical-benefit language exceeds the evidence.

  2. Central evaluation results need reconciliation. Table 4's best-report rates total 104% on GAVD and 114% on ProGait. Yet Appendix F requests a single best report. Section 4.3 also calls the Diagnostic scores 3.71/3.43 Overall scores; Table 4 gives 3.64/2.76. Moreover, 27 normal videos cannot yield 99.3% specificity under a single unweighted evaluation. Any additional averaging is unexplained. These issues affect the headline results, rather than just presentation.

Questions for authors
  1. Validation: What are the errors of representative angles and event times against independent references? Does robustness persist when perturbations also affect triage and tool selection?

  2. Clarification: Please provide correct/eligible counts, run aggregation, missing-rating handling, and corrected report-preference totals. How are the four unscoreable ProGait trials treated? Please also report participant-level results, given only four prosthesis users.

1 comment

Dear reviewer,

As the rebuttal period is approach to the end, we would like to kindly check if our rebuttal has sufficiently addressed all your concerns and questions. Please feel free to let us know if any further clarifications, explanations or experiments are needed, and we will be more than happy to discuss further in any aspect.

Again, thank you so much for all your attention and efforts!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.