- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes TAP, a simple potential‑based reward‑shaping technique that repurposes target critic values as shaping potentials for DRL.
Critical strengths
- This paper proposes a new potential-based reward-shaping method.
- The paper validates its effectiveness through multiple experiments.
Critical weaknesses
The experiments are insufficient: they lack more visual-control benchmarks, do not compare against a sufficient number of baselines, and provide no hardware-efficiency analysis of the proposed method.
Questions for authors
- The authors note that both existing potential-based reward-shaping schemes increase complexity. It is recommended to conduct a complexity analysis, or a hardware-efficiency analysis (FLOPs, throughput, memory consumption, etc.), comparing the proposed method with baseline methods, including state-of-the-art potential-based reward-shaping methods.
- What would be the effect of directly using the online critic, rather than the target critic, as the potential function? Why must the target critic be used as the potential function?
- Two visual-control tasks are too few; it is recommended to increase the number to at least five visual-control tasks.
- The comparison between the original DRL algorithms and the Base-Visual algorithm is too limited, and the compared algorithms are all too old. It is recommended to additionally compare against five state-of-the-art DRL algorithms and several visual DRL algorithms.
- The baselines from related methods in the same field (same-type reward shaping / prior-knowledge transfer methods) are also too few and mostly outdated. It is recommended to compare against at least five state-of-the-art same-type reward-shaping / prior-knowledge transfer methods. In addition, reward shaping includes other types of methods besides potential-based ones; it is recommended to add several other types of reward-shaping methods as comparisons.
- If the prior knowledge is very poor, for example a source critic of very low quality, is the proposed method still effective?