- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies two questions failure: whether the component receiving greater attribution also provides the larger repair gain, and whether any member of a fixed positive-scale family can recover a reference accuracy. The authors establish two post-failure conclusions that attribution and fitted performance alone leave unresolved. Matched component replacements quantify the accuracy cost of following attribution to the weaker tested repair.
Critical strengths
The component exchanges are concrete and interpretable. The positive-scale analysis also has a clear mathematical basis: candidate-score differences are affine in the scale, making exact winning-set calculations possible under the specified scoring and tie rules. The paper carefully distinguishes source-fitted rules from label-informed oracle capacities.
Critical weaknesses
The attribution and repair questions use different targets. Attribution explains the performance difference between included and excluded participants, while repair concerns only excluded participants. A change that helps both groups equally will cancel in the first comparison but still help the second. The radius gains—4.57 and 4.43 points—illustrate exactly this. Section 3.2 also says that attribution of the excluded-group loss agrees with the repair ranking. This makes the main finding less surprising than the framing suggests.
The OfficeHome result may depend heavily on the chosen map. The map retains only 44 of 512 dimensions. Showing that scaling its output cannot recover the original accuracy is valid, but how much does this tell us beyond the limits of that map? A comparison that preserves the remaining subspace would help.
The practical benefit of the capacity calculation is unclear. Source-task validation already chooses the unmodified representation on OfficeHome. On STAE, the positive result requires choosing a scale for each query using its label. Neither case shows how the analysis leads to a better repair decision without target labels.
Questions for authors
- After matching the attribution objective to excluded-role repair, what substantive attribution–repair discrepancy remains?
- Does the OfficeHome exclusion persist when the mapping preserves the complementary subspace or uses a regularized full-rank alternative?
- What concrete decision improves when exact capacity is available, compared with source-task validation and ordinary scale selection?
- Can the authors clarify which central quantities can be independently regenerated from per-query features and fitted maps, rather than checked from saved summaries or intervals?