1. It is difficult to separate the contribution of the learned policy from the control knowledge already built into the reward.
My main concern comes from the reward design in Appendix C.1. Equation (22) includes a penalty for deviating from a reference torque uk,jd, and this reference is computed from the attitude error and angular velocity using natural-frequency and damping-ratio parameters. In other words, the learning objective already contains a fairly explicit PD-like control target. The reference configuration also assigns a non-negligible weight to this term (wt=2).
I do not consider using control knowledge in the reward to be a problem in itself. In fact, this may be one reason why the method works well. However, it changes how I interpret the experimental results. Is DP-DDQN learning a substantially better control law, or is it mainly learning to approximate and adjust a reasonably good analytical controller?
The paper compares against PID and MPC, but it does not isolate the torque-reference penalty or evaluate the reference controller itself under the same conditions. Without such a comparison, I find it difficult to attribute the performance gain specifically to the proposed differential Q-learning formulation.
This also affects the novelty assessment. Incremental actions and dimension-wise Q-learning are already established ideas. The paper would be considerably more convincing if it showed which part of their combination produces the reported improvement.
2. The theoretical analysis and the actual training/evaluation setup leave two important gaps.
The channel-wise contraction result appears mathematically reasonable under the stated assumptions, but I am not convinced that it explains the behavior of the deployed controller. During training, a candidate action for one axis is evaluated while the other torque commands remain fixed. During execution, however, all three networks select and apply their increments simultaneously. The resulting joint transition is not the same transition used to construct the individual channel-wise Bellman operators.
Thus, the contraction result applies to the idealized channel-wise operators, not necessarily to the value functions learned by the neural networks or to the stability of the joint closed-loop policy. The distinction matters because the method is motivated partly by handling coupled rotational dynamics.
I also have concerns about how the learning-based comparisons should be interpreted. Appendix C.2 specifies 20 Bellman iterations, 50,000 sampled states per iteration for each axis, and evaluation of seven candidate actions per state. This suggests approximately 21 million single-step model-action evaluations across the three axes. By contrast, Appendix D reports approximately 500,000 training transitions for PPO and TD3. These are not directly equivalent computational units, but the difference is large enough that training cost and access to the dynamics model should be discussed explicitly.
As presented, the comparisons establish that the proposed training-and-control pipeline works well in the tested setting. They do not yet establish that it is a more efficient or generally preferable reinforcement learning method.