acceptodds
Under review as a conference paper at ICLR 2027

CtrlRecon: Steering Expressive Human Reconstruction toward Prescribed States

Abstract

Expressive human pose and shape estimation (EHPS) increasingly underpins digital-human systems by mapping visual inputs to the poses, hand articulations, body shape and facial expressions enacted by an avatar. If an adversary can control these outputs, an innocuous input could induce attacker-specified behavior absent from the source, elevating the threat from reconstruction degradation to deliberate behavioral manipulation. Yet existing attacks against EHPS remain predominantly untargeted that reconstructions can be corrupted or driven toward implausible configurations, but not whether an adversary can steer them toward a prescribed state. To address this gap, we introduce CtrlRecon, a framework for targeted attacks on EHPS. Specifically, CtrlRecon constructs an image-specific target by integrating attacker-specified articulations with the corresponding clean predictions, holding it constant throughout the optimization process. Guided by this static target, a representation-aware objective jointly enforces the prescribed modifications and preserves the unmodified components. Subsequently, a two-stage procedure optimizes over the bounded continuous space and refines the result into the discrete 8-bit domain, ensuring adversarial effectiveness is fully maintained after standard image export. Experiments across three EHPS estimators validate the effectiveness of CtrlRecon for targeted control. In VLM-based evaluation, CtrlRecon receives 60.10% of the votes, exceeding the strongest baseline by 36.68 percentage points.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Expressive Human Pose and Shape Estimation, Targeted Adversarial Attacks, Adversarial Robustness

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_t8EwOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work studies targeted white-box attacks on expressive human pose and shape estimation (EHPS). Given a clean image and a fixed estimator, CtrlRecon builds an image-specific target by substituting requested pose, hand, or expression blocks with prescribed values while keeping the remaining blocks at their clean predictions. It then optimizes a representation-aware objective that combines rotation matching, a hand-specific loss designed around non-unique 6D rotation outputs, expression matching, and camera preservation. Optimization proceeds in two stages: bounded continuous search followed by discrete 8-bit refinement, with final attacks evaluated after PNG export and reload.

Critical strengths

<1>Steering a reconstructed avatar toward a chosen pose, gesture, or facial expression is more security-relevant than merely inflating reconstruction error. Separating requested changes from preservation targets gives the attack a concrete definition, and the controlled white-box experiments show that such selective manipulation is feasible.

<2>Multiple raw 6D outputs can decode to the same rotation, and near-degenerate outputs can be fragile under perturbation. The appendix supplies propositions on when the hand loss reaches zero and on local conditioning of the Gram–Schmidt denominators. Evaluating exported and reloaded 8-bit images is also a sensible practical choice. Geometric metrics, joint-success measurements, and multi-view visual comparisons each capture a different aspect of the attack.

Critical weaknesses

<1>The classical baselines use a plain unweighted L2 distance over raw parameters, include shape in that distance, and lack matrix-level rotation, decoded-geometry, and positive-preimage terms; CtrlRecon excludes shape. Their update counts are matched only to accepted continuous steps, whereas CtrlRecon also uses line-search evaluations, curvature VJPs, and discrete refinement. The ablations remove entire articulation terms but do not isolate the hand loss from a decoded-rotation loss, or integer refinement from continuous optimization followed by the same quantizer. It therefore remains unclear how much of the improvement comes from the distinctive design rather than from objective scaling and extra optimization.

<2>Table 10 allows per-joint rotation errors up to 90° for Body and 50° for Hand, applies these tolerances to active and inactive rotations alike, and permits camera deviations of 0.30 m and 0.20 m. Shape is excluded from the success criterion, and clean inputs that already satisfy the target are counted in the denominator without a separately reported clean success rate. The VLM evaluation ignores inactive components and forces a winner even when no candidate matches the target. The high ASR and 60.10% vote share therefore do not by themselves establish accurate control with strong preservation.

Questions for authors

<1>Does CtrlRecon keep its advantage when PGD and C&W are given the same representation-aware objective, identical targets and preservation terms, and matched forward/backward computation? Please report actual computation and wall-clock time, including curvature construction and integer refinement.

<2>How do the results change under stricter rotation and camera tolerances? Please separate active-target errors from changes to inactive components, and report the clean-input success rate, so that target attainment and preservation can be assessed independently.

<3>Can the authors compare the hand loss against a decoded-rotation loss and a variant without the positive-pivot margin, while holding the optimizer fixed? Likewise, what is the incremental benefit of integer refinement over continuous optimization followed by the same feasible quantization?

5 comments
AuthorsOpenReview ↗

Response to reviewer t8Ew

We thank the reviewer for the detailed feedback. Additional experiments and clarifications are highlighted in blue (Section 5 and Appendices C–D).

W1/Q1: Baseline fairness and computational cost.

  • Baseline comparison. Our task controls pose/expression while preserving inactive components and camera translation, without constraining shape. Since the original baselines penalized shape, we reran FGSM, PGD, and C&W with CtrlRecon's four-term loss, retaining their update rules, budgets, and candidate selection; C&W retains its distortion penalty. Joint ASR (%) on H32/UBody (2,351 held-out inputs per target; ϵ=0.06\epsilon=0.06):
    MethodBodyHandFace
    Original FGSM0.1700
    FGSM + CtrlRecon loss0.1700
    Original PGD11.9565.0445.98
    PGD + CtrlRecon loss32.5871.3383.45
    Original C&W85.9689.4575.54
    C&W + CtrlRecon loss91.6277.0792.3
    CtrlRecon91.9691.4595.32
    The loss improves PGD on all targets and C&W on Body/Face, but lowers C&W's Hand ASR. CtrlRecon leads all three comparisons (Appendix D.4).
  • Computational cost. Baseline budgets use CtrlRecon's mean continuous updates on each dataset, not equal total computation. Appendix D.7 (Tables 22–23) reports forward/backward counts, curvature VJPs, and available timings. Matched-runtime experiments are underway, but their scale have delayed complete results. We will report them when available.

W2/Q2: Control accuracy, preservation, and VLM evaluation.

  • Stricter success criteria. We evaluate two stricter (rotation, expression RMSE, camera) criteria, (15°, 0.3, 10 cm) and (5°, 0.1, 5 cm), for both target attainment and preservation. CtrlRecon leads seven H32/UBody configurations under both, although Body/Hand ASR drops substantially (Table 17).
  • Target attainment and preservation. We separately evaluate target attainment and preservation under stricter criteria (Appendix D.7, Table 20). At 15°/0.3/10 cm, Hand is primarily limited by target attainment (147/2,351), while 2,082 inputs satisfy preservation. Body is constrained by both requirements, whereas Face achieves high success on both.
  • Clean-input success. We test whether already-successful clean inputs inflate ASR (%):
    SettingBodyHandFace
    Clean1.870.090
    CtrlRecon (all inputs)91.9691.4595.32
    CtrlRecon (clean successes excluded)91.8191.4495.32
    Exclusion changes ASR by at most 0.15 percentage points; no clean input meets either strict criterion (Table 20).
  • VLM evaluation. Forced-choice voting among Clean and four attacks measures relative visual target matching, not absolute success or preservation. Appendices C.6 and D.8 clarify normalization and valid-response coverage.

W1/Q3: Loss design and integer refinement.

  • Loss design. We evaluate three loss variants with fixed optimizer settings and model-evaluation limits: (1) Body uses hand loss, applying the hand formulation to non-finger rotations; (2) Hand uses rotation loss, replacing the hand-specific formulation with a standard rotation-matrix loss; and (3) Hand loss with the positive-pivot margin μ reduced from 0.1 to 10−610^{-6}, while keeping all other loss settings the same as in full CtrlRecon. Table 3(a) reports joint ASR (%):
    SettingBodyHandFace
    Body uses hand loss74.8690.7795.87
    Hand uses rotation loss91.283.9295.79
    Reduced positive-pivot margin89.8874.4495.36
    CtrlRecon91.9691.4595.32
    Against CtrlRecon, standard rotation loss and reduced margin lower Hand ASR to 83.92% and 74.44%; applying hand loss to body rotations lowers Body ASR to 74.86%. This supports the region-specific formulations and margin.
  • Integer refinement. We compare three configurations using the same continuous outputs (Table 3(c)): Continuous evaluates the outputs before quantization; Quantized w/o refinement directly quantizes them to 8-bit images; and CtrlRecon further applies integer refinement to the same quantized images.
    SettingBodyHandFace
    Continuous94.0091.8895.87
    Quantized w/o refinement65.3890.9462.14
    CtrlRecon91.9691.4595.32
    Quantized w/o refinement substantially reduces Body/Face ASR, while integer refinement recovers most of this loss, with a smaller gain on Hand, at additional computational cost.
AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer t8Ew,

We have completed the matched-runtime experiments mentioned in our previous response. As reported in the newly added Appendix D.5 (Figure 6 and Table 18), CtrlRecon achieves joint ASRs of 98.2%, 93.4%, and 99.4% on Body, Right-hand, and Face, respectively, outperforming the strongest baseline by 7.4, 0.8, and 2.8 percentage points under the original success criteria.

As the discussion period draws to a close, we thank you once again for your detailed review and constructive suggestions. Your comments on computational fairness, target attainment, preservation, and visual evaluation have helped us substantially strengthen the experimental analysis in the revised manuscript (highlighted in blue). For your convenience, here is a summary of the key revisions:

  • W1/Q1 Baseline fairness and computational cost: Evaluated baselines using CtrlRecon's objective and completed matched-runtime comparisons (Appendices D.4–D.5).

  • W2/Q2 Target accuracy, preservation, and VLM evaluation: Added stricter success criteria, clean-input analysis, and separate evaluations for target attainment and preservation (Appendices D.2 and D.7), while also clarifying the VLM voting protocol and reporting valid-response coverage (Appendices C.6, D.3, and D.8).

  • W1/Q3 Loss design and integer refinement: Added controlled comparisons between our proposed hand loss and standard rotation loss, analyzed the positive-pivot margin, and isolated the contribution of integer refinement (Section 5.4).

We sincerely appreciate your time and thoughtful feedback.

AuthorsOpenReview ↗

Follow-up before the discussion closes

Dear Reviewer t8Ew,

As the discussion period approaches its conclusion, we would like to thank you once again for your thoughtful review. We hope our response and the summary above have addressed your questions. In light of these results, we would greatly appreciate it if you could take another look and consider raising your score. Please let us know if any further clarification would be helpful.

Best regards,

The Authors

Reviewer_9HaKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CtrlRecon, a white-box targeted adversarial attack framework for expressive human pose and shape estimation. The framework aims to drive selected body poses, hand joints, or facial expressions toward preset states while preserving other pose and expression components as well as camera translation. The method combines a fixed image-specific target, a perceptual-target objective function (including a dedicated loss for the 6D hand representation), and a refinement step of continuous optimization followed by 8-bit quantization. The evaluation is conducted on the UBody and PW3D datasets against three estimators, comparing with targeted FGSM, PGD, and C&W attacks, using metrics such as geometric error, joint attack success rate, and vision-language model (VLM)-based visual matching, and includes ablations on the loss and perturbation budget. All attacks are evaluated after PNG export and reloading.

Critical strengths

This paper investigates targeted manipulation of selected body, hand, or facial components while attempting to preserve the rest of the reconstruction. This goes beyond merely demonstrating degradation in reconstruction performance and reveals a practically relevant vulnerability in systems that directly use estimated parameters to drive avatar behavior.

CtrlRecon achieves lower full-body MPVPE for body targets and lower facial MPVPE for expression targets across all three estimators and both datasets. For example, on the UBody/H32 dataset, the MPVPE for facial targets decreases from 27.62 mm under the C&W attack to 3.93 mm. The evaluation conducted after PNG export and reloading further strengthens the evidence that these improvements persist through standard image serialization.

Critical weaknesses

The contribution of the proposed design is insufficiently isolated. The baseline uses an unweighted raw parameter loss, whereas CtrlRecon changes both the objective function and the optimization pipeline; even when the number of accepted updates is matched, the total computation is not matched. The ablation removes entire loss terms rather than comparing alternative formulation designs, and it does not isolate the integer refinement step. Consequently, the current results cannot demonstrate that these dedicated designs are superior to simpler but appropriately configured alternatives.

Questions for authors

What evidence can distinguish the gains brought by the dedicated hand loss and integer refinement from those brought by the optimizer and additional computation? Under a matched computational budget, comparisons with a standard rotation loss and with a quantization method without refinement would help determine whether these components are truly necessary.

5 comments
AuthorsOpenReview ↗

Response to reviewer 9HaK

We appreciate the reviewer's insightful comments on isolating the contributions of our method. In response, we have conducted additional controlled experiments on loss design and integer refinement, with the corresponding revisions highlighted in blue (Section 5 and Appendices C–D).

W/Q: Contributions of Loss Design and Optimization.

  • Baseline Comparison with CtrlRecon Loss. To distinguish the contributions of the loss and optimization procedure, we reran FGSM, PGD, and Adam-based C&W using CtrlRecon's four-term loss, while retaining their original optimization settings and candidate selection procedures. C&W also retains its distortion penalty. All methods use the same targets and 2,351 held-out H32/UBody inputs per target (ϵ=0.06\epsilon=0.06). Joint ASR (%) is reported below:

    MethodBodyHandFace
    Original FGSM0.1700
    FGSM + CtrlRecon loss0.1700
    Original PGD11.9565.0445.98
    PGD + CtrlRecon loss32.5871.3383.45
    Original C&W85.9689.4575.54
    C&W + CtrlRecon loss91.6277.0792.3
    CtrlRecon91.9691.4595.32

    Using CtrlRecon's loss improves PGD across all targets and C&W on Body/Face, but reduces C&W's Hand ASR. Full CtrlRecon achieves the highest joint ASR across all three targets, indicating that loss replacement alone does not reproduce its performance (Appendix D.4).

  • Hand loss. We compare alternative loss formulations while keeping the optimizer settings and query limits fixed (Table 3(a)). Joint ASR (%) is reported below:

    ConfigurationBodyHandFace
    Body uses hand loss74.8690.7795.87
    Hand uses rotation loss91.283.9295.79
    Hand loss, numerical floor onlyReduced positive-pivot margin89.8874.4495.36
    CtrlRecon91.9691.4595.32

    Replacing our hand loss with a standard rotation loss reduces Hand ASR from 91.45% to 83.92%. Reducing the positive-pivot margin from 0.1 to 10−610^{-6} further lowers it to 74.44%. These results support the dedicated hand loss and its positive-pivot margin.

  • Integer refinement. We compare continuous optimization, direct 8-bit quantization, and subsequent refinement using the same continuous outputs and quantizer (Table 3(c)). Joint ASR (%) is reported below:

    ConfigurationBodyHandFace
    Continuous94.0091.8895.87
    Quantized w/o refinement65.3890.9462.14
    CtrlRecon91.9691.4595.32

    Quantized w/o refinement substantially reduces Body/Face ASR, while integer refinement recovers most of this loss, with a smaller improvement on Hand, at additional computational cost.

  • Computational budget. We set the baseline iteration budgets based on CtrlRecon's mean continuous updates on each dataset. However, this does not fully account for the additional line-search evaluations, curvature computation, and integer refinement in CtrlRecon. We have therefore reported actual forward/backward counts and available timing measurements in Appendix D.7 (Tables 22–23). Matched-runtime experiments are currently underway, but the large experimental workload have prevented us from obtaining complete results so far. We will report them as soon as they become available.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer 9HaK,

We have completed the matched-runtime experiments mentioned in our previous response. As shown in the newly added Appendix D.5 (Figure 6 and Table 18), CtrlRecon achieves joint ASRs of 98.2%, 93.4%, and 99.4% on Body, Right-hand, and Face, respectively, outperforming the strongest baseline by 7.4, 0.8, and 2.8 percentage points under matched runtime budgets.

As the discussion period draws to a close, we would like to thank you again for your insightful comments and constructive suggestions. Your feedback regarding the loss formulation, integer refinement, and computational budget has led to valuable controlled experiments in our revised manuscript (highlighted in blue). To facilitate your final review, here is a brief summary of the main additions:

  • W/Q Baseline fairness: Compared baselines using CtrlRecon's four-term matching objective and completed matched-runtime experiments (Appendices D.4–D.5).

  • W/Q Hand loss design: Compared our proposed hand loss with a standard rotation loss and analyzed the impact of the positive-pivot margin (Section 5.4).

  • W/Q Integer refinement: Isolated its contribution by comparing continuous optimization, direct quantization, and subsequent refinement starting from identical continuous outputs (Section 5.4).

We sincerely appreciate your careful assessment and hope these additional experiments help address your concerns.

AuthorsOpenReview ↗

Follow-up before the discussion closes

Dear Reviewer 9HaK,

As the discussion period approaches its conclusion, we would like to thank you once again for your thoughtful review. We hope our response and the summary above have addressed your questions. In light of these results, we would greatly appreciate it if you could take another look and consider raising your score. Please let us know if any further clarification would be helpful.

Best regards,

The Authors

Reviewer_13GwOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies targeted attacks on expressive human reconstruction. CtrlRecon combines image-specific targets, a representation-aware loss, and continuous-to-discrete optimization to control selected outputs while preserving other components.

Critical strengths

1.Targeted manipulation is a meaningful extension beyond simply degrading reconstruction. 2. Evaluating exported and reloaded images is a useful practical detail.

Critical weaknesses
  1. The baselines are not sufficiently strong or fairly configured. PGD and C&W use an unweighted raw-parameter loss, while CtrlRecon benefits from a representation-aware objective and additional optimization steps. The baselines also penalize shape changes, whereas CtrlRecon does not. Please compare against well-tuned PGD and Adam using the same objective, constraints, and computational budget. Without these controls, it is difficult to tell where the gains come from.
  2. The computational budgets are not matched. Baseline iterations are calibrated to accepted continuous updates, while CtrlRecon additionally uses line searches, integer refinement, and 400 residual VJPs for H32. Results under matched runtime or derivative budgets are needed.
  3. The success thresholds seem loose for precise control. Table 10 allows rotation errors up to 90° for Body and 50° for Hand, including preserved joints. High ASR under these thresholds does not necessarily imply close target matching. Please report threshold sensitivity and how often clean inputs already satisfy the criteria.
  4. The ablations do not test the distinctive design choices. Removing a target loss predictably hurts that target. More informative comparisons would replace the proposed hand loss with a standard rotation loss and compare integer refinement against simple rounding. 5.The generality of the results is limited. Only one template per target type is tested. Moreover, C&W outperforms CtrlRecon on all three UBody targets for Hand4Whole in Table 13. These results deserve discussion in the main text.
Questions for authors

How much ASR is lost through quantization, and how much does integer refinement recover? Does the advantage remain with more diverse targets and stricter preservation thresholds?

5 comments
AuthorsOpenReview ↗

Response to reviewer 13Gw

We thank the reviewer for the constructive feedback. We have conducted additional experiments and revised the manuscript accordingly, with changes highlighted in blue (Section 5 and Appendices C–D).

W1–W2: Baseline fairness and computational cost.

  • Baseline Fairness. Our task controls pose/expression while preserving inactive components and camera translation, without constraining body shape. Since the original baselines penalized shape, we reran FGSM, PGD, and Adam-based C&W using CtrlRecon's loss, retaining their update rules and C&W's distortion penalty. Joint ASR (%) on H32/UBody (2,351 held-out inputs per target; ϵ=0.06\epsilon=0.06).

    MethodBodyHandFace
    Original FGSM0.1700
    FGSM + CtrlRecon's loss0.1700
    Original PGD11.9565.0445.98
    PGD + CtrlRecon's loss32.5871.3383.45
    Original C&W85.9689.4575.54
    C&W + CtrlRecon's loss91.6277.0792.3
    CtrlRecon91.9691.4595.32

    Using CtrlRecon's loss improves PGD across all targets and C&W on Body/Face, but reduces C&W's Hand ASR. CtrlRecon retains the highest joint ASR across all three targets in these comparisons (Appendix D.4).

  • Computational Cost. Baseline iteration budgets were set using CtrlRecon's mean continuous updates on each dataset, which does not ensure equal computational cost. Appendix D.7 (Tables 22–23) reports actual forward/backward counts, curvature costs, and available timings. Matched-runtime experiments are underway, but their scale has delayed complete results. We will report them as soon as available.

W3: Attack Success under Stricter Evaluation Criteria.

  • To assess control precision, we evaluate stricter criteria of (15°, 0.3, 10 cm) and (5°, 0.1, 5 cm) for rotation, expression RMSE, and camera error. CtrlRecon achieves the highest joint ASR among seven configurations on H32/UBody under both criteria, although Body/Hand ASR drops substantially (Table 17). We also examine whether clean inputs inflate ASR:

    Evaluation settingBodyHandFace
    Clean1.870.090
    CtrlRecon (clean successes excluded)91.8191.4495.32

    Excluding clean successes changes ASR by at most 0.15 percentage points, indicating that the reported success rates are not driven by these inputs. Appendix D.7 (Table 20) further separates target attainment and preservation.

W4/Q: Missing hand-loss and refinement comparisons.

  • Hand loss. Replacing our hand loss with a standard rotation loss reduces Hand ASR from 91.45% to 83.92%. Reducing the positive-pivot margin from 0.1 to 10−610^{-6} further lowers ASR to 74.44%, supporting the benefit of the proposed formulation (Table 3(a)).

  • Integer refinement. We compare continuous optimization, direct 8-bit quantization, and subsequent refinement using the same continuous outputs (Table 3(c)). Joint ASR (%) is reported below:

    ConfigurationBodyHandFace
    Continuous94.0091.8895.87
    Quantized w/o refinement65.3890.9462.14
    CtrlRecon91.9691.4595.32

    Quantized w/o refinement substantially reduces Body/Face ASR, while integer refinement recovers most of the loss (Table 3(c)), at additional computational cost.

W5/Q: Target Diversity and Hand4Whole Performance.

  • Target diversity. We further evaluated 15 additional targets (five each for Body, Hand, and Face) on 20 fixed UBody images. CtrlRecon achieves the highest overall joint ASR (87.00% vs. 80.33% for C&W), tying for the highest Body ASR and leading on Face, while C&W performs better on Hand (Appendix D.6, Table 19). These results demonstrate that CtrlRecon can successfully attack diverse targets beyond the original three templates.
  • Hand4Whole limitation. C&W outperforms CtrlRecon on all three Hand4Whole/UBody targets. For Hand, CtrlRecon has a lower median target-rotation error (43.30° vs. 47.06°), but higher 90th-percentile rotation error (59.93° vs. 51.19°) and camera drift (25.41 vs. 4.30 cm). These larger errors help explain its lower joint ASR. We discuss this limitation in Section 5.2, with details in Appendix D.7 (Table 21).
AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer 13Gw,

We have completed the matched-runtime experiments, with the results presented in the newly added Appendix D.5 (Figure 6 and Table 18). Using CtrlRecon's four-term matching objective under matched runtime budgets on H32/UBody, CtrlRecon achieves joint ASRs of 98.2%, 93.4%, and 99.4% on Body, Right-hand, and Face, respectively—outperforming the strongest baseline by 7.4, 0.8, and 2.8 percentage points.

As the discussion period draws to a close, we thank you once again for your thorough review and constructive feedback. Your insights on baseline fairness, computational budgets, evaluation rigor, and target diversity have motivated several key additions to our revised manuscript (highlighted in blue). For your convenience, we summarize the main updates below:

  • W1–W2 Baseline fairness and computational cost: Evaluated baselines using CtrlRecon's objective and conducted matched-runtime comparisons (Appendices D.4–D.5).

  • W3–W4/Q Evaluation rigor and design choices: Added stricter success criteria, clean-input analysis, comparisons with a standard rotation loss, and integer refinement ablations (Appendices D.2 and D.7; Section 5.4).

  • W5/Q Target diversity and generality: Evaluated 15 additional targets and analyzed the weaker results observed on Hand4Whole (Appendices D.6–D.7).

We sincerely appreciate your time and consideration.

AuthorsOpenReview ↗

Follow-up before the discussion closes

Dear Reviewer 13Gw,

As the discussion period approaches its conclusion, we would like to thank you once again for your thoughtful review. We hope our response and the summary above have addressed your questions. In light of these results, we would greatly appreciate it if you could take another look and consider raising your score. Please let us know if any further clarification would be helpful.

Best regards,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their constructive feedback. We have uploaded a revised PDF with changes highlighted in blue (main text within 10 pages). The main additions, organized by question, are summarized below:

  • Are the baseline comparisons fair, and what does the proposed loss contribute? (13Gw, 9HaK, t8Ew) We re-evaluated FGSM, PGD, and C&W using CtrlRecon's representation-aware loss while retaining their optimization settings (Appendix D.4, Tables 16–17). CtrlRecon achieves the highest joint ASR across all three targets on H32/UBody, although the shared loss improves several baselines.
  • Does targeted control remain effective under stricter success criteria? (13Gw, t8Ew) We added two common stricter criteria (15°/0.3/10 cm and 5°/0.1/5 cm), clean-input success analysis, and separate target-attainment and preservation evaluations (Appendices D.2, D.7; Tables 14, 20). CtrlRecon retains the highest joint ASR among seven configurations on H32/UBody under both strict criteria, although precise Body/Hand control remains limited (Table 17). Excluding already-successful clean inputs changes ASR by at most 0.15 percentage points.
  • What are the contributions of the hand loss and integer refinement? (13Gw, 9HaK, t8Ew) New ablations compare our hand loss with a standard rotation loss and a reduced positive-pivot margin (Table 3(a)). Replacing the hand loss lowers Hand ASR from 91.45% to 83.92%. We also compare continuous optimization, direct 8-bit quantization, and integer refinement using the same continuous outputs (Table 3(c)). Refinement restores Body/Face ASR from 65.38%/62.14% to 91.96%/95.32%.
  • Does the method work across diverse targets and estimators? (13Gw) We evaluated 15 additional targets across Body, Hand, and Face (Appendix D.6, Table 19), achieving 87.00% overall joint ASR versus 80.33% for C&W on the tested subset. We also explicitly discuss the weaker Hand4Whole results in Section 5.2, supported by error-distribution analysis (Appendix D.7, Table 21).
  • How are computational cost and visual evaluation accounted for? (13Gw, 9HaK, t8Ew) We added actual forward/backward counts, curvature costs, and available timing measurements (Appendix D.7, Tables 22–23). Matched-runtime experiments are underway, with complete results still pending. We also clarified the five-candidate VLM voting protocol, normalization, and valid-response coverage (Appendices C.6 and D.8).

Detailed responses to each reviewer's concerns are provided in the individual replies.

AuthorsOpenReview ↗

Summary of the discussion

Dear Reviewers, ACs, SACs, and PCs,

Thank you for your hard work on the review process. We also thank all reviewers for their helpful feedback and for finding our method sound. Here is a short summary of the discussion. Reviewers 13Gw and t8Ew raised their ratings from 2 to 3, and Reviewer 9HaK maintained their rating of 3. All revisions are highlighted in blue in the updated manuscript.

Reviewer feedback after the rebuttal

  • 13Gw noted that the matched-runtime comparison “addresses my main concern about baseline fairness” and that the new ablations answered their questions regarding the hand loss and quantization.

  • t8Ew highlighted the matched-runtime comparison, stricter evaluation, and new ablations, stating that these “address my main concerns.” Both 13Gw and t8Ew encouraged us to note the precision gap between face and body/hand control in the main text.

  • 9HaK thanked us for the additional experiments and confirmed that they would maintain their rating.

Main concerns and our responses

  • Does the advantage hold under a fair computational budget? On H32/UBody, we compared PGD and C&W using CtrlRecon's four-term matching objective against full CtrlRecon under matched runtime budgets. Under the original success criteria, CtrlRecon outperformed the strongest baseline by 7.4 percentage points on body, 0.8 on right hand, and 2.8 on face (Appendix D.5, Figure 6 and Table 18).

  • What do the hand loss and integer refinement contribute? Ablations on H32/UBody evaluate the contributions of both components. Right-hand joint ASR dropped from 91.45% with our proposed hand loss to 83.92% with a standard rotation loss, and further to 74.44% when reducing the positive-pivot margin (Section 5.4, Table 3(a)). Starting from identical continuous outputs, integer refinement recovered most of the joint ASR lost to 8-bit quantization, raising body ASR from 65.38% to 91.96% and face ASR from 62.14% to 95.32% (Section 5.4, Table 3(c)).

  • How precise is the control? We decoupled target attainment from preservation of inactive components and evaluated two stricter criteria (15°/0.3/10 cm and 5°/0.1/5 cm for rotation, expression RMSE, and camera drift). CtrlRecon achieved the highest joint ASR among all seven compared configurations on H32/UBody under both strict criteria (Appendix D.4, Table 17). Excluding already-successful clean inputs changed ASR by at most 0.15 percentage points (Appendix D.7, Table 20). However, strict evaluation reveals a clear precision gap: under 15°/0.3/10 cm, joint ASR remained 88.60% for Face but dropped to 14.76% for Body and 6.25% for Hand. For Hand specifically, 2,082 of 2,351 cases satisfied preservation, whereas only 147 attained the target (Appendix D.7, Table 20).

  • Do the results extend to other targets and estimators? Across 15 additional targets on 20 UBody images, CtrlRecon achieved the highest overall joint ASR at 87.00% vs. 80.33% for original C&W, while C&W performed better on the additional hand targets and led on all three target types for Hand4Whole on UBody (Section 5.2 and Appendix D.6, Table 19).

In the final version, we will make the difference in precision between face, body, and hand control explicit in the main text alongside the Hand4Whole results, as 13Gw and t8Ew suggested.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.