- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In the paper, the authors first formulate action applicability (vision-based and human-specific) and introduce the BodyReady benchmark for the visual action applicability of a specific human, which evaluates whether a body state enables an action. The benchmark is built on 280 exocentric images and contains 1,190 questions. There are 6 tasks evaluating different aspects of action readiness, and they evaluate 18 VLMs (both closed- and open-source). They find a gap between the execution possibility of an action and its plausibility.
Critical strengths
- Clear formulations and tasks.
- Comprehensive evaluation (multiple different VLMs) and rigorous diagnosis of the unmet body conditions.
Critical weaknesses
- Generalization to other embodiments: Robotic hands, humanoids, and similar entities have different body structures and different sets of possible actions due to differences in angles, joints, heights, and so on. Formulations are usually general first and then narrowed down to a specific setup. Can the formulation of action applicability and readiness be extended to these other embodiments?
- Motivation and Application: If the goal is for a robot to execute a sequence of applicable actions toward a goal, why would the vision be from an exocentric point of view, and why is it human-centric? These cases seem applicable only when there is a need to train a robot or agent from human actions, or for human-assistance applications. Therefore, since the benchmark acts as a test set, can this evaluation framework be utilized for test-time self-critique to enable replanning loop before action execution?
- Narrow data collection setups: The BodyReady data collection process introduces several narrow setups:
- Static action matching: Because the analysis is performed on individual, static frames, it limits the ability to expand the task to include the temporal axis (e.g., pace, duration).
- Object detection: Objects are detected using bounding boxes rather than semantic masks. This limits tasks that might relate to the feasibility of holding challenging shapes (e.g., narrow, sharp, or overly large objects) or interacting with object materials and additional attributes (e.g., cold, hot, or slippery surfaces).
- Limited camera angles.
Questions for authors
- How complex are the visual scenes in terms of object density, distractors, and so on? Are there statistics regarding the complexity of the data (both the scenes and the task instructions)?
- How is reachability assessed? The distance should be calibrated by the relative locations of the person and the target object with respect to the camera location.
7 comments
Response to Reviewer Aht7 (2/4)
W2: Motivation and application
Thank you for raising this. Assistance and human–robot collaboration, the second case you list, are the target setting of BodyReady. For example, a robot judges whether this person can begin an action now, and uses that judgment to decide whether and how to help. By human-centered planning in the title, we mean planning by a robot or assistive system of how to work with a person, including which steps the person performs, which the robot performs, and when to intervene. The exocentric view follows from this setting. An assistant observes the person from the outside, whereas an egocentric view shows little of the wearer's own body.
Test-time self-critique. Yes. Before a plan is executed, BodyReady's tasks can serve as the checks of a self-critique and replanning loop. The loop checks whether each step can begin, finds where the plan first fails, and diagnoses what blocks that step. It then revises the plan by inserting a preparation or a help action that lets the step begin, and repeats the checks on the revised plan. Each check can be run and scored on its own, so the benchmark measures how reliable each check would be.
| Step of self-critique before execution | BodyReady task | Claude Opus 5.5 |
|---|---|---|
| Can this step begin? | AR / AS | 82.4% / 63.8% |
| Where does the plan first fail? | SV | 79.9% |
| What blocks it? | BD | 65.7% |
| Which preparation would let it begin? | PS | 83.8% |
Even the strongest model stays below 85% at every step, so such self-critique cannot yet be trusted without oversight.
A case study on one round of the loop. We ran one round of this loop for a single action of the person, in an assistance setting. Before the person acts, the assistant checks whether the person can begin the action and what blocks it. If it finds a blocker, it revises the plan by inserting a help action before the person's step, following the just-right assistance policy of Liu et al. (2026), extended from reach to all three channels.
The help action takes the object occupying a hand, or brings the target within reach. We compare these decisions with decisions the same model makes directly, without self-critique, given the same policy.
Decisions with self-critique come from the BD answers in Table 2, and direct decisions come from a new run on the same questions. Both are scored against the policy applied to the reference blockers. A full loop would repeat this round after each revision and over every step of a plan, using the SV and PS checks above.
| 282 BD questions (no movement or posture blockers) | Fully correct ↑ | Over-assistance ↓ | Under-assistance ↓ | Wrong type ↓ |
|---|---|---|---|---|
| Claude Opus 5.5, direct | 55.7% | 18.6% | 16.1% | 43.5% |
| Claude Opus 5.5, self-critique | 62.4% (+6.7) | 20.0% (+1.4) | 11.8% (−4.3) | 35.8% (−7.7) |
| GPT-5.6 Terra, direct | 45.4% | 18.6% | 29.2% | 52.0% |
| GPT-5.6 Terra, self-critique | 56.0% (+10.6) | 14.3% (−4.3) | 25.0% (−4.2) | 38.4% (−13.7) |
| Qwen3.8-27B, direct | 44.7% | 11.4% | 28.9% | 57.3% |
| Qwen3.8-27B, self-critique | 47.5% (+2.8) | 42.9% (+31.4) | 13.2% (−15.7) | 48.9% (−8.4) |
| Qwen3-VL-32B-Instruct, direct | 28.4% | 7.1% | 73.1% | 73.7% |
| Qwen3-VL-32B-Instruct, self-critique | 31.6% (+3.2) | 71.4% (+64.3) | 6.1% (−67.0) | 65.3% (−8.4) |
| Qwen3-VL-8B-Instruct, direct | 26.2% | 5.7% | 86.3% | 72.4% |
| Qwen3-VL-8B-Instruct, self-critique | 36.9% (+10.6) | 21.4% (+15.7) | 42.0% (−44.3) | 60.2% (−12.3) |
| Always help | 27.3% | 100% | 0% | 63.7% |
| Never help | 24.8% | 0% | 100% | – |
One round of self-critique improves the help decisions, and how much depends on how reliably the model diagnoses blockers. It raises fully correct decisions for all five models, significantly for Claude Opus 5.5 (p = 0.02), GPT-5.6 Terra (p < 0.001), and Qwen3-VL-8B-Instruct (p < 0.001) in paired exact McNemar tests. For the two strongest models, it improves assistance overall, as under-assistance and wrong-type help fall while over-assistance changes little. For the open models, over-assistance also rises, because their diagnoses report blockers that are not there. Qwen3-VL-32B-Instruct, for example, rarely helps when deciding directly (78.0% "no help"), but its diagnoses report a blocker in 50 of the 70 questions that need no help. BodyReady's BD task measures this reliability, with 65.7% and 50.6% for the two strongest models against 26.2% for Qwen3-VL-32B-Instruct. For the strongest models, the remaining limit is reading the person, as Claude Opus 5.5 decides correctly in 87.9% of cases with the verified state.
Our results suggest that BodyReady's tasks could serve as the checks of such a loop, and could indicate for which models the loop is likely to help. We therefore see test-time self-critique and replanning as a promising use of BodyReady, and thank you for suggesting it. The revision adds a paragraph relating the tasks to this loop.
Response to Reviewer Aht7 (3/4)
W3: Data collection setups
Thank you for these points. We chose Ego-Exo4D and Nymeria as sources because both extend directly to video and to other camera views.
Static frames and temporal aspects. BodyReady evaluates readiness at a given moment, that is, whether this person can begin an action now. Temporal aspects such as pace and duration are beyond its current scope, and the framework can be extended to them with the source videos. Every annotated frame has a timestamp in its source video, and each action's completion effect specifies which channel changes when the action is done, for example the hand becoming free after placing an object. Annotating when each such change happens in the video would make readiness a function of time, and would support questions such as how long a blocker persists, when the person becomes ready for an action, or how quickly the person completes a sequence of steps. The current questions and reference answers already apply to video clips that end at the annotated moment, without re-annotation; if the reviewers find it useful, we would be glad to release these clips together with the images. The revised Limitations section states this scope.
Camera angles. Both datasets provide time-synchronized, calibrated cameras. Ego-Exo4D records each scene with 4–5 exocentric cameras and an egocentric camera, and Nymeria adds the wearer's egocentric view to the observer's view. A person's bodily state at an instant does not depend on the viewpoint, so the annotations and reference answers carry over to the other views; only the visibility of the relevant body parts and objects needs to be re-checked, so the extension is straightforward.
Bounding boxes rather than masks. We agree that BodyReady does not annotate segmentation masks or object attributes such as shape, material, and temperature. At the same time, the labels do not rely on the bounding box alone. Whenever an action is bound to an object, human reviewers check whether that action applies to this object in its current state and scene (object-level eligibility, Section 3.1 and Appendix C; Figure 2). An action enters the benchmark only if it passes this check. For example, “close the cabinet” is not posed for a cabinet that is not open, and “pick up” is not posed for an object that offers no part a person could grasp. In this way, object properties that decide whether an action applies can enter the labels in the current setting. The revised Limitations section states this scope explicitly, noting that the benchmark has no segmentation masks or attribute labels.
Q1: Scene and task complexity
Yes. The revised appendix adds the following statistics, for the scenes and for the task instructions.
| Scenes | Mean | Median | Range |
|---|---|---|---|
| Object density: annotated objects per image and person | 11.2 | 10 | 1–39 |
| Distractors: annotated objects not mentioned in the question | 10.1 | 8 | 0–38 |
| Same-category distractors (e.g., a second cup) | 1.2 | 0 | 0–16 |
| Image area occupied by the designated person | 13.8% | 11.6% | 2.0–37.9% |
| Task instructions | Mean | Median | Range |
|---|---|---|---|
| Prompt length (words) | 280 | 287 | 219–379 |
| Objects mentioned per question | 1.8 | 1 | 0–8 |
| Candidate actions per question: AS | 4.1 | 4 | 2–5 |
| Candidate actions per question: PS | 2.8 | 3 | 2–5 |
| Steps per sequence: SV | 2.6 | 3 | 2–3 |
Object counts cover annotated objects only, so they are lower bounds on scene content. The appendix reports the same statistics separately for Ego-Exo4D and Nymeria.
Response to Reviewer Aht7 (1/4)
Thank you for the positive assessment and the thoughtful questions. We appreciate your recognition of the clear formulation and tasks, and of the comprehensive evaluation.
W1: Extension to other embodiments
Thank you for this question. Yes, the formulation extends to other embodiments: its logic does not depend on the human body, and only its content must be redefined for each body.
For example, for a two-armed mobile robot, the hand channel would record what each gripper holds, the posture channel the configuration of its base and torso, and the reach channel what its arms can reach from the current base position. "Pick up the box with both hands" would then require two free grippers and the box within reach. The angles, joints, and heights you mention enter through the posture and reach channels and through R(a); the computation of readiness, blockers, and effects stays the same.
We start with humans because assistance requires judging the person, and because the human body has constraints, such as posture transitions and the coordination of two hands, that need dedicated definitions. Our data cover only humans, so we present the extension at the level of the formulation. The revision adds a short paragraph, "Beyond human bodies", with this table.