- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
UniQ learns an offline policy that maximizes reward subject to a cost limit. It first assigns each state to a safe, boundary, or recovery region and freezes the reward–cost weights. One flow critic then fits the combined return, and one conditional flow policy learns from that value. Both are conditioned on accumulated cost, which is resampled during training. On nine MetaDrive tasks the reported normalized cost is below 1 on every task. The mean normalized reward is 0.59, against 0.40 for FISOR, the strongest baseline that is feasible on all nine tasks.
Critical strengths
The contribution is to learn both the combined critic and the policy as conditional flows. The critic fits the reward–cost return directly, and the policy is trained from that value under a frozen safety rule, conditioned on accumulated cost. On all nine MetaDrive tasks the normalized cost stays below 1, and the mean normalized reward is 0.59, about 48% above FISOR. The costs lie between 0.90 and 0.97, so the higher reward is obtained while still meeting the constraint.
Critical weaknesses
The objective depends on a boundary weight and on a second check that uses the cost already spent. Neither formula is given, so the return the flow is fitting cannot be reconstructed. The higher reward under the cost limit is also not tied to one piece of the method: the experiments never remove the flow critic, the frozen rule, or the resampling of accumulated cost. The method is complicated, and it depends on many hyperparameters.
Questions for authors
What is the formula for the boundary weight, and what is the check that uses the cost already spent?
If the flow critic is replaced by separate reward and cost critics, or the safety rule is allowed to keep adapting, or cost resampling is turned off, which change removes the gain?