- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
In robotics domains, it's quite common to assume that the world is composed of objects since that usually gives us good generalization. When the observation is object-factorized, the policy (or some other module) selects the objects of interest for the given state, and makes decisions based only on that set. Nevertheless, this assumption might be silently broken, or even though the method is designed to leverage object-factorization, it might not be doing so because (1) it might be overfit to selecting the same set of objects as in the training set, or (2) unselected objects' might be leaked through selection, or (3) it doesn't perform better than random object selection. This work proposes three tests to check each of these conditions to see if the "object-factorization helps" claim is true.
Critical strengths
The proposed tests could serve as a sanity check for methods that use the object-factorized formulation.
Critical weaknesses
It took my considerable effort to understand the paper. It is as if the paper is shared as an internal document with a colleague who works on the same topic and can understand what is meant by each phrase, rather than a manuscript with clear definitions intended for a general audience. I'm not saying this as a bad thing---it's just that the text is really hard to understand because of many unclear definitions and phrases. It could be because I'm not very familiar with the topic, but even then I could say that the motivation and narrative are not very clear from the abstract and introduction. Likewise the experiment settings and the results that accompany them.
The reasoning behind some of these tests is not well-motivated. Especially for the discovery condition, why exactly do we want to limit the method to train its selector from the task error only? Let's say there's indeed a labeled dataset that tells which of the objects should be selected. Why would this be a limitation, if the selector then can generalize?
Questions for authors
- What methods are we considering? Behavior-cloning policies, reinforcement learning, and in which setting? MDP, POMDP?
- Aren't selectors permutation-invariant by design? I can train a selector using the labels (which I'm not allowed to if I understand the discovery condition correctly) for each of the inputs without providing an order. Same for the cardinality. An object-level classifier could be trained on the annotation list which would not depend on the cardinality. In any case, these scenarios are not very clear.
3 comments
Thank you for your response.
Its subset therefore shows what the annotator decided, not what the model relies on, and cannot reveal, for example, a bias in the data that the annotator did not know (Sections 3.1 and 4.2).
Yes, but can't we just treat the annotator as part of the model? Correct me if I'm wrong. Let's say there's a selector model that is trained with labels. And it shows some generalization in the test set as well. Doesn't this show that the downstream model relies on it? Selector model is just another component of the big architecture. I'm not sure what I am missing here. I'm asking why do we want to specifically train the selector model with the downstream loss? You're claiming that it will memorize labels, but we don't know that---I don't think the test set performance will just drop to zero.
Thank you for the follow-up. We think the question separates two claims an explanation can make, and we agree with you on the first.
1. "The action depends only on the selected objects." If the annotator is treated as part of the model, a labeled selector with corrected removal supports this claim: its action depends only on the selected subset (Removal), and the subset is more informative than a random one (Consequence). Our labeled selector satisfies both conditions (Table 2), and it can be useful and generalize well (Section 3.4).
2. "These are the objects the model found relevant." This is the claim that object-selecting policies such as SIR make when they present the subset as revealing what a task requires, or a bias in the data. Discovery concerns only this claim. A selector trained on an annotation list learns to predict that list, and, as you say, it generalizes: it reaches an AUROC of at least 0.998 against the annotation on held-out frames. By "reproduces" we do not mean memorization or a drop in test performance. We mean that on new scenes it shows the objects the annotator would have marked. Its subset therefore reports a judgment that was already written down, not one the model formed. If the annotation misses an object the task needs, the labeled selector learns to drop it too, so the explanation cannot reveal the gap. A selector shaped only by the policy's loss can, because nothing tells it which objects to keep. Our label-free selectors, for example, keep different objects than the annotation (Section 4.2).
Why the downstream loss. Training the selector only through the policy's loss is not required for good performance or for a faithful explanation; it is required only if one wants to read the subset as what the model found. In our experiments, the labeled selector performs about as well as a policy that is simply given the annotated objects (Table 2), which illustrates that its selection adds little beyond the annotation itself.
We thank the reviewer for the careful reading and the constructive comments. We revised the paper mainly for clarity:
While revising, we found that the corrected removal path of the submission still leaked the positions of removed objects. Table 2 now reports the fully corrected path, which has no significant performance difference to the model with leaked positions. Therefore, the interpretation of the results is unchanged. If time permits, we would be grateful if you could read the revised Introduction and Section 3, where most changes are. We would really appreciate if the reviewer could now tell us if the paper is easier to follow and more understandable, as we want to make it available for anyone and not just our colleagues.
W1: Readability. Thank you for this direct feedback as it led to the largest changes. The Introduction now states the problem more directly and introduces the ILS Explanation framework explicitly. Section 3 defines the setting formally (a selector that picks a subset of the objects, and an action generator that acts on it) and then treats each condition with the same structure: A requirement, the ways it can fail, and its test. Figure 2 illustrates each condition, and Section 4 follows the same order.
W2: Why Discovery? D-C does not say that supervised selection is bad for performance or generalization. A labeled selector can select well and generalize (Section 3.4). It concerns only the explanation claim: Whether the selected subset shows what the model found. A selector trained on an annotation list learns to reproduce it (AUROC ≥ 0.998 on CALVIN), and in our experiments it performs about as well as a policy that is simply handed the annotated objects (Table 2). Its subset therefore shows what the annotator decided, not what the model relies on, and cannot reveal, for example, a bias in the data that the annotator did not know (Sections 3.1 and 4.2).
Q1: Setting. We consider policies trained by behavior cloning (imitation learning) and evaluated in closed loop. At each step, a set of objects is extracted from the benchmark's static cameras, so the policy observes the scene only partially through these objects. We do not consider reinforcement learning.
Q2: Permutation invariance and cardinality. Yes, the selectors are permutation-invariant over the input order. Our former "permutation test" did not permute the input. It reassigned the relevance labels to other objects to verify that the code of a label-free selector never reads them. We renamed it the label-shuffle test to avoid this confusion (Section 3.1, Appendix A.4). An object-level classifier trained on the annotation list is indeed independent of order and cardinality, but it still fails D-C, because it is trained on a relevance signal: its selection reproduces the annotation. The cardinality failure refers to something else: a budget taken from the annotation (e.g., the number of annotated objects per task) tells even a label-free selector how many objects to keep.