- Rating
- 1
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This empirical paper asks how neurons in shared multi-task PPO policies are reused across tasks and whether the pattern reflects the geometry of task parameterizations. It computes task-conditioned neuron-importance profiles using gradient-times-activation at the penultimate layer, compares profiles through the Bhattacharyya coefficient, and performs task-targeted top-neuron ablations to test functional selectivity. Experiments use AntDir (eight goal directions), HalfCheetahVel (eight target speeds) and a seven-task learned subset of Meta-World MT10, with multiple seeds, two network widths where applicable, random-network controls, one-hot goal encodings and additional layer/pruning analyses. In AntDir, overlap and collateral ablation damage decrease with angular task distance, including under one-hot encoding. In HalfCheetahVel, overlap follows velocity distance but ablation damage is not target-selective. Meta-World shows overlap without robust task-pair ordering across seeds and exhibits asymmetric ablation effects. The paper appropriately distinguishes importance overlap from functional modularity.
Critical strengths
- A clear, controlled test of a precise representation question. The directional AntDir result is quantitatively strong: at width 64 under continuous encoding, mean overlap falls from 0.882 at 45° to 0.590 at 180° (Table 1), and an angular gradient remains when the continuous goal input is replaced by a one-hot task ID with comparable trained return (Sec. 4.1 and Appendix B). Random-initialization controls and first-layer analyses (Appendix F) help separate learned organization from trivial input similarity. Five seeds and width comparisons improve the credibility of the descriptive finding.
- Functional interventions and explicit negative cases prevent an overly simple modularity story. Targeted neuron ablations in AntDir preferentially damage nearby tasks (Table 2), while HalfCheetahVel shows little analogous selectivity despite overlap structure (Table 3). Meta-World shows asymmetric transfer of ablation damage, which symmetric overlap measures cannot capture (Table 5). The importance-guided pruning experiment (Appendix E) adds a limited but concrete use of the attribution profiles. This separation of similarity, causal vulnerability and directionality is informative.
Critical weaknesses
- An unresolved attribution-versus-state-distribution confound weakens the mechanistic interpretation. For the chosen penultimate layer with a linear action head, the Jacobian is the same readout-weight matrix for every task; the reported gradient-times-activation profiles therefore largely reweight the task-dependent magnitudes of hidden activations by fixed output weights (Sec. 3.1). Profiles are further averaged over separately collected rollouts for each task. Differences in visited states, action magnitude or occupancy may consequently induce the observed geometric structure without showing that reusable task-specific circuitry is organized by geometry. Causal neuron ablations provide useful support, but they do not by themselves identify which source creates the attribution pattern.
- The scope of robust, geometry-aligned functional specialization is narrow. A selective angular ablation gradient is established most clearly in AntDir, a small and highly structured eight-task family. In HalfCheetahVel, the overlap gradient is dissociated from selective functional damage (Sec. 4.2), while in the seven-task Meta-World subset the pairwise ordering reproduces only weakly between seeds (mean Spearman 0.21/0.30 at widths 64/400; Table 4). Three of ten MT10 tasks are excluded from the main analysis because the policies did not learn them (Sec. 4.3; Appendix H). These limitations reduce the case for a general claim about multi-task policies beyond this descriptive set of PPO MLPs. Figure/presentation concern (secondary): Figures 1 and 3 place many small numerical annotations inside heatmap cells and use dense rotated task labels; the multiple colored activation traces and bands in Fig. 2 are also difficult to distinguish at normal paper size. Because pairwise comparisons are central to the paper, the figures should be redesigned with larger labels, reduced in-cell clutter and more distinguishable encodings. This is not by itself evidence that the underlying measurements are wrong.
Questions for authors
- Matched-state attribution (assessment-critical). If all task importance profiles are recomputed on a shared pool of observations (or with a state-distribution-matched importance estimator), rather than on task-specific rollouts, does the AntDir angular overlap curve survive? A related activation-only baseline would help determine how much of the gradient-times-activation result is attributable to learned task-dependent computation versus occupancy and activation scale. This would materially affect the mechanistic interpretation.
- Generality and causal selectivity (assessment-critical). Can the authors provide a task-normalized measure of ablation selectivity that adjusts for baseline return and the magnitude of disruption, especially in HalfCheetahVel, where returns can become negative? Is there an additional continuous-goal task family with nontrivial state variation in which the AntDir pattern replicates? Either result would materially change the assessment of how broadly the proposed organizing principle holds.