- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Factored Q Chunking, which is a variant of Q Chunking that allows the overall action to be factorized into fast and slow partitions being re-planned at different time scales. Each factor of the action is judged by a different critic, but the same actor. The authors show results on two evaluation suites BiGym and DexJoCo as well as ablations with QC and FQC varying partitions and other hyper parameters.
Critical strengths
(S-1) Critic x execution horizon is a nice ablation.
(S-2) Detailed appendix with very transparent statistics and per-task breakdowns. Released code is a plus.
Critical weaknesses
(W-1) I believe that the comparison between FQC at N=32 and QC at N=1 is unfair.
In particular this sentence "FQC improves on Q-chunking on both suites: 0.327 against 0.102 for QCcal on DexJoCo, and 0.480 against 0.302 on BiGym" (line 401). The fair comparison is QCcal+BoN vs. FQC+BoN which makes the gap smaller (0.327 vs. 0.231 (DexJoCo) and 0.480 vs. 0.440 BiGym). In the latter case, the uncertainties overlap meaning FQC is not significantly better.
Can you run FQC without BoN training (FQC with N=1) to compare directly to QC_cal at N=1? It would also be helpful to see N=4 and N=8 as well.
(W-2) The improved average performance of FQC on BiGym seems to be primarily attributed to one task, dsTrays where FQC achieves 0.784 vs. QC_Cal + BoN achieving 0.30 (which brings the average down). I would like to see further mentions of this point and closer ablations of QC_Cal + BoN on this task in particular (since it is the primary reason why FQC is winning on BiGym).
(W-3) In Table 1, it seems like FQC with a random partition is comparable in performance to FQC using slow-fast partitioning. On BiGym it is hard to see what factors matter since all the partition splits are within noise of each other. I would encourage the authors to run more seeds so the results can be statistically significant.
I am willing to raise my score if these concerns are addressed.
Questions for authors
(Q-1) It would be helpful to see a comparison to QC replanning more often at H=4. Can you add a baseline using QC_cal+BoN with H=4 on both suites?
(Q-2) QC_cm controls for parameter count but only at N=1. Can you run QC_cm + BoN?
Response to Reviewer p9ZB
Thank you for the review and the concrete suggestions. We ran all the experiments that you suggested, and they made the paper stronger. With the same number of candidates, FQC is higher than the single critic at every N on both suites, and the gap on DexJoCo is significant.
W-1. "The fair comparison is QC_cal+BoN vs. FQC+BoN. Can you run FQC with N = 1, and also N = 4 and N = 8?"
We agree that this is the fair comparison, and we ran it at every N. At N = 1 the single critic is QC_cal.
W-2. "The improved average on BiGym seems to be primarily attributed to dsTrays."
We looked closely at this task, and the gain is real. With ten seeds, FQC reaches 0.669 against 0.282 for QC_cal+BoN (+0.387 [+0.054, +0.720], Welch p = 0.025). It is not an effect of ranking either: FQC is higher at every N, and already at N = 1.
The four-member critic with ranking (QC_cm+BoN) reaches 0.579. This task does drive the BiGym average: without it the seven-task means are 0.436 for FQC and 0.460 for QC_cal+BoN. Section 5.4 now discusses this.
W-3. "FQC with a random partition is comparable to FQC using slow-fast partitioning. On BiGym it is hard to see what factors matter."
You are right about Table 1: with five seeds, the embodiment split, its swap and the random draw are level on BiGym (0.480, 0.482, 0.477). To test whether that draw is typical, we ran two more random partitions. Both are lower than the embodiment split on every BiGym seed (0.420 and 0.420 against 0.509; for the first, −0.089 [−0.156, −0.021]). On DexJoCo they reach 0.318 and 0.263 against 0.324. So the embodiment split is the best or tied for the best on both suites, and a random partition can be worse.
The factor that matters most on BiGym, and that is statistically significant with five seeds, is that the critics match the execution.
The interaction of the two-by-two ablation is +0.211, positive on all five seeds (p = 0.008).
Q-1. "Can you add a baseline using QC_cal+BoN with H = 4 on both suites?"
We added it with five seeds. FQC is higher on both suites.
Re-planning every coordinate more often does not help the single critic.
Q-2. "Can you run QC_cm + BoN?"
We ran it with the five seeds of the paper. FQC is higher on both suites.
Doubling the critic recovers less than half of FQC's gain on DexJoCo (+0.040 of +0.096) and none of it on BiGym. Capacity does not explain the result.
New: real-robot validation at equal N. On a Unitree G1 humanoid with an Inspire dexterous hand, FQC succeeds in 40 of 60 evaluation episodes, against 28 of 60 for QC_cal+BoN, both with N = 32. The general comment gives the details.
We hope that the new experiments and the revised text address your concerns. If so, we would be grateful if you would consider raising your score. We are glad to address any concern that remains.