- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper selects a budgeted concept subset for RL by preserving distinctions between states with different action values. It proposes DRS and its noise-aware extension, DRS-log, with analysis based on state abstraction. Experiments cover five RL environments, simulated concept interventions, and supervised concept selection on CUB.
Critical strengths
-
Useful formulation. Connecting concept selection to state abstraction provides a principled criterion for retaining decision-relevant information under a concept budget.
-
Encouraging results. Reduced concept sets retain strong performance across several environments, with additional evaluation under imperfect predictions and simulated interventions.
Critical weaknesses
-
The practical objective does not control the error used in the performance bound. Theorem 4.1 establishes minimax optimality with constraint (2d) and rho = 0. Practical DRS drops (2d), allowing its summed objective to trade a few large errors for many small improvements. The performance bound depends on the maximum error, which can remain large. Corollary 4.1 supplies no bound on the additional error from this relaxation, so it does not establish that practical DRS retains the idealized guarantee.
-
Theorem 4.2 omits a condition on reference-policy quality. The statement bounds estimation error relative to , but the proof uses , effectively treating the reference policy as optimal. For a near-optimal reference policy, a bound such as and a corresponding error term are needed. Without this link, accurate estimation of a suboptimal reference policy does not justify the stated performance guarantee. Appendix N provides empirical evidence but does not resolve this gap.
-
DRS-log's analysis stops short of the claimed performance motivation. The analysis bounds pairwise separation probabilities under independent concept errors, but does not connect the optimized surrogate to maximum abstraction error or policy return. Consequently, it leaves unclear when accounting for predictor noise through DRS-log should preserve or improve decision performance.
-
The interpretability benefit is underdeveloped. Selecting concepts using a reference policy and then training a new policy does not establish a faithful explanation of the reference policy. The new policy still uses a neural concept-to-action mapping. Reward and simulated correction results offer limited evidence of how this replacement helps a person understand, diagnose, or supervise decisions.
Questions for authors
-
Is Theorem 4.2 intended to assume ? If near-optimal reference policies are allowed, and how does the suboptimality enter the performance bound?
-
How are Appendix L's maximum abstraction errors computed? Are they evaluated on held-out states, and do they quantify the error introduced by relaxing (2d)?
-
What concrete inspection or diagnosis task benefits from replacing the reference policy with the new concept-based policy? Please illustrate with a specific state, concept values, and action or correction.
Rebuttal
Dear Reviewer PHJX,
Thank you for your review, and for viewing our paper as having a useful formulation with encouraging results. We provide answers to your questions below.
Thank you for your suggestion on Theorem 4.2. To deal with imperfections in the reference policy π^, we first define η:=s,amax∣Qπ∗(s,a)−Qπ^(s,a)∣. Using the triangle inequality theorem, we can apply the same proof as Theorem 4.2, to demonstrate that $V^{{\pi*}_{\mathbf{c}}}(s) - V^{\pi*_{\hat{\mathbf{c}}}}(s) \le \frac{2(\epsilon_{Q^{\pi}}(g_{\mathbf c})+ 4\delta + 4 \eta)}{(1-\gamma)^2}$ where π∗c^ denotes the optimal policy using the selected concepts. Here, c^ is selected using the approximate Q-values for the reference policy π^. The key intuition here is that a suboptimal policy is detrimental based on the degree that it reduces our knowledge of Q values.
Thank you for your comments, and we agree that the new policy might not necessarily be a faithful explanation of the original policy. Constructing a faithful explanation is an interesting problem, but would also require a different algorithm. Instead, our goal is to train a new, interpretable policy that can be used as a substitute for the original blackbox policy, while still preserving performance. This underlies the reason why we aim to maximize rewards while having an interpretable architecture, rather than imitating the actions exactly.
Thank you for the question on the abstraction errors from Appendix L. We agree that the theoretical guarantee applies to the original minimax formulation, and does not directly extend to the relaxed objective. Appendix L empirically quantifies the maximum abstraction error under this relaxation, evaluated on held-out states from separate rollouts. We will clarify this distinction and the evaluation procedure in the revised manuscript..
As a concrete example of the benefits of concept-based models in practice, consider a medical setting where an automatic glucose agent recommends insulin dosing. Interpretability is critical here because the doctor should be able to see how different factors might impact the glucose agent. Pure blackbox models such as LLMs would be inappropriate here because it would be difficult to audit them, and doctors would have little understanding or trust that the algorithm would correctly perform its duties. Concept-based models in this setting could improve explainability and allow doctors to understand why actions were taken by the policy.
In our scenario, the state space would be the raw medical information consisting of glucose history, blood-related data, time-of-day, information on the meals eaten, etc., and the set of concepts would be binary signals such as whether glucose has risen, and whether we have eaten a meal recently. For example, if glucose is rising and a meal was recently eaten, the policy might recommend increasing insulin delivery. If the meal concept were incorrectly predicted, a doctor could correct that concept and observe whether the recommended action changes. The key benefit of a concept-based policy is that it allows for human understanding and potential correction or debugging when errors are made.