- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how cloth-support information can guide dual-arm unfolding. On a transparent table, morphological operations on a top-view silhouette and a bottom-view near-table mask produce support-residual, contact, frontier, and edge channels for an attention encoder and a residual action prior. SAC learns continuous corrections, while an online linear scorer adjusts grasp-target rankings using reward or unfolding progress. The design is evaluated through fixed-initial-state ablations, randomized initial states, and physical experiments. The contribution is primarily a system-level combination of support-aware representation and action selection, rather than a new reinforcement-learning algorithm or a bandit algorithm with regret guarantees.
Critical strengths
- The connection between representation and control is explicit and interpretable. The same parameter-free geometric channels support both state encoding and grasp proposals, rather than merely adding camera input. In the randomized campaign, replacing raw dual-view inputs with dual-view affordances improves the reported F/C from 0.0466/0.766 to 0.0066/0.968. The top-depth control also helps distinguish the role of structured representation from that of additional bottom-view information. This provides useful empirical evidence for task-relevant inductive bias.
- The paper meaningfully examines the gap between proxy metrics and actual unfolding. The authors do not equate low F with successful unfolding. The flat-folded-start experiment uses independent resets, coverage measurements, and human fold checks: the full method succeeds in 51/60 trials, versus 36/60 for top-depth and 39/60 for edge-only. Bottom-view noise, physical calibration, and illumination experiments also reveal operating conditions. In particular, the paper reports registration failures and improvements after calibration, rather than showing only ideal perception results.
Critical weaknesses
- The core method and controlled experiments remain incompletely specified. This paper does provide the grid, network widths, and scorer hyperparameters. However, reward-component weights, the exact progress-feedback formula, the mixing rule, and adaptive prior gating are not defined. The section titled “Scorer Update and Gating Pseudo-Code,” provides only a high-level update procedure. These are precisely the dimensions varied in Table 3. The paper states that networks and budgets are matched, but does not enumerate the randomized campaign's actual training budget, the prior/scorer modules enabled in each row, or the construction of the top-depth channels. The attribution to representation alone therefore cannot be independently checked from the reported configuration details.
- The learned scorer's gains do not yet establish a robust, generalizable advantage. The main ablation uses a shared fixed initial state, only three seeds per variant, and 25k training steps. In Table 16, the heuristic prior obtains a return of −68 for one seed but 31.1 for another, showing that differences in means are sensitive to training variation (Section 4). The additional 75k-step experiment in Appendix L must be taken into account: F decreases from 0.0121 ± 0.0045 to 0.0088 ± 0.0031, consistent in direction but with substantial uncertainty, while return changes only from 24.6 to 25.0 (Table 13). The randomized and physical experiments support the full system, but do not separately test learned versus fixed ranking in those settings (Tables 1 and 5).
Questions for authors
- Could you provide complete formulas for the reward, progress feedback, mixed feedback, and adaptive gating, and specify which parameters are updated during physical fine-tuning and whether the scorer continues learning during evaluation?
- Which networks, training budgets, and action priors are used for each method in Table 1? How does top-depth estimate contact and frontier channels? Please provide the configurations and uncertainty for each control so that the matched comparison can be verified.
5 comments
Sincere Thanks for the Reviewer’s Constructive Comments:(Part 2/4): Weakness W1, items 6–7 and 2×2 experiment; Weakness W2
W1 (continued)
6. The randomized-campaign records (Sec. 4.2) specify 75,000 environment transitions per seed, 3 seeds, aggregated across nine environments. The encoders share feature dimension 128 and channel widths (32, 64, 96).
7. The top-depth affordance variant estimates near-table occupancy from valid top-surface depth with a 5.5 cm threshold, then constructs the same six channels: top, estimated support, fold residual, contact, frontier and edge. Frontier and edge use the same 3×3 morphology as in Sec. 3. This estimate replaces the bottom mask both in the policy observation and in prior target selection. We will document the depth-noise settings and any run-specific overrides.
We ran a 2×2 experiment crossing the policy observation with the action prior, with 5 seeds per condition and paired evaluation on the same 17 held-out states. Reward, SAC configuration, training steps, data split, and checkpoint selection are shared. Both observation types use the encoders of Appendix F, which share the channel widths (32, 64, 96), the four-head bidirectional cross-attention and pooling, and the 128-dimensional output. Only the input stems differ, as the inputs require: top and bottom stems for raw dual masks, and a six-channel global stem plus fold/frontier and contact/edge stems for affordance channels. In C and D the heuristic residual prior is identical and its scorer is frozen at .
| Condition | Policy observation | Action prior | Folded area F ↓ (m²) |
|---|---|---|---|
| A | Raw dual masks | Off | 0.0511 |
| B | Affordance channels | Off | 0.0201 |
| C | Raw dual masks | Heuristic residual prior, fixed scorer () | 0.0503 |
| D | Affordance channels | Heuristic residual prior, fixed scorer () | 0.0141 |
This supports the value of the geometric representation within an end-to-end learned policy.
W2
To directly address this concern, we added a controlled comparison between fixed and learned ranking under randomized initial states. All other components and settings are held constant. The fixed scorer uses manually specified, frozen ranking weights, whereas the reward-bandit scorer updates these weights using interaction feedback.
Both variants are trained with randomized initial states for 50k steps using 20 independent training seeds per method without the 100-rollout real-robot fine-tuning. Each trained model is evaluated on the same 100 held-out initial states. Physical evaluation includes three cloth types and 20 trials per method per type, with matched initial configurations, randomized execution order, and balanced use of models from multiple training seeds.
| Evaluation metric | Fixed scorer | Learned scorer | Improvement [95% CI] |
|---|---|---|---|
| Simulation: folded area F ↓ (m²) | 0.0148 | 0.0067 | [0.0069, 0.0091] |
| Physical: coverage gain ↑ | 0.10 | 0.13 | [0.01, 0.05] |
| Physical: success (NC + fold check) | 46.7% | 58.3% | [3.5%, 19.8%] |
We Appreciate the Opportunity to Clarify Based on Your Feedback: (Part 3/4): Question Q1 (complete formulas)
Q1
The following formulas give the complete specification, using the same notation as in W1.
Let , , and denote clipping to . , , , , denote projected area, planar spread, height standard deviation, folded area, and contact ratio, respectively.
The reward is
Its auxiliary terms are:
- : positive improvement over the historical best area, counted only when .
- : newly crossed thresholds {0.70, 0.80, 0.90, 1.00}, requiring , with comparison tolerance . : newly crossed gripper-distance thresholds {0.18, 0.12, 0.08} m, summed across arms.
- For arm , the grasp quality is , where is the gripper opening; the arm is active if . sums each arm's quality at its first active event, plus one at the first dual-active event. The reach weight is , multiplied by 0.2 when both arms are active; is the end-effector-to-cloth distance.
- “New progress” means a grounded improvement over a historical extremum exceeding in , in , in reduced , in reduced , or in , using metres and square metres.
- Let count consecutive active-grasp steps without new progress, resetting otherwise. Then , multiplied by 1.5 for a dual grasp. only during an active grasp without new progress, multiplied by 1.25 for a single grasp.
Progress, reward, and hybrid (mixed) feedback are
Non-finite component signals are set to zero. For the selected feedback (, , or the hybrid), the clipped advantage and EMA baseline are
These are updated only when a valid selection record exists and the scorer learning rate is positive.
For adaptive gating, let , , be the fold-residual, frontier, and contact mask sums divided by the top-mask sum:
Here is the latest pre-action advantage (the most recently stored ). The gate is zero for an empty top mask or a top-mask sum , and one when adaptation is disabled.
Physical fine-tuning: SAC updates the actor, twin critics, their visual encoders, and the entropy temperature (automatic entropy tuning, Appendix F); target critics receive Polyak updates. The learned scorer separately updates its six weights and feedback baseline.
The online scorer continues to update during evaluation, adapting its target ranking weights using interaction feedback within the current trial. At the beginning of each trial, its weights are reset to the same predefined initial values, and its feedback baseline and update counters are cleared. No scorer updates are carried over between trials.
Gratefully Responding to Your Constructive Feedback: (Part 1/4): Weakness W1, items 1–5 (reward, feedback, and gating formulas)
W1
Thank you for the detailed questions. Notation follows Sec. 3 and Table 24; new symbols are defined where they first appear. The complete formulas are given in our response to Q1, using the same notation.
1. Reward-component weights. Sec. 3 describes the reward only in words. The weights used in the Table 3 runs are: area / spread / flatness gains 300 / 120 / 120; best-area gain 800; coverage / proximity / grasp bonuses 35 / 1.5 / 4; reach 8; flatness bonus 1; action / stall / bad-pull penalties 0.002 / 10 / 120; success bonus 120. The alive term and the direct fold/contact reward terms have zero weight.
2. Exact progress-feedback formula. The grounded geometry progress gt of Sec. 3 is
gt=clip(Gt(10ΔA+5ΔSp−2ΔZ−10ΔF+0.5ΔC),−1,1)
where A,F,C,Z are area, folded area, contact ratio and height standard deviation as in Sec. 4, Sp is the cloth spread (written Sp to avoid a clash with the ranking index S), and Δx=xt−xt−1. The height gate is
Gt=exp(−max(zˉt−ztable−0.02,0)/0.05)
with zˉt the mean cloth height.
3. Mixing rule. Reward feedback is qt=tanh(Rt/2), as in Sec. 3. Hybrid feedback is
pt=clip(0.75gt+0.25qt,−1,1)
The geometry term gt is clipped before mixing. The progress-only and reward-only scorers use pt=gt and pt=qt.
4. Adaptive prior gating. In the executed action aexec=clip(aRL+λtaprior,−1,1) of Sec. 3, the prior weight is λt=0.35κt, where κt is the adaptive gate. Let mtfold, mtfr, mtcon be the fold-residual, frontier, and contact mask sums divided by the top-mask sum:
nt=clip(max(mtfold/0.04,mtfr/0.02),0,1),ht=clip((mtcon−0.97)/0.03,0,1)
κt=clip(min(nt(1−0.7ht(1−nt)),1+0.75vlast),0.15,1.25)
where vlast is the most recently stored scorer advantage (At in Sec. 3).
Because nt[1−0.7ht(1−nt)]≤1, the gate can only attenuate the prior: for any non-empty top mask, κt∈[0.15,1], i.e. λt∈[0.0525,0.35]. The advantage term can bind only when vlast<0. The upper clipping bound of 1.25 is part of the implementation but is never reached.
Two cases bypass this formula. For an empty top mask (top-mask sum ≤10−6), the fractions are undefined and the gate is set to κt=0, so no prior action is applied. With adaptation disabled, κt=1, i.e. λt=0.35 as in Appendix F.
5. Pseudocode and Table 3 conditions. We will replace the high-level Algorithm 1 (Appendix P.1) with explicit feedback, baseline-update, weight-update, gating and reset rules. Table 3 uses learning rates η = 0.10 / 0.20 / 0.40 for the progress / reward / hybrid scorers, each with the adaptive gate disabled or enabled (6 variants), plus the fixed heuristic residual prior: 7 variants × 3 seeds = 21 runs.