- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper examines whether emergent social learning enables expert-modelling capabilities without explicit imitation objectives. It presents an information-theoretic framework and blind-corrected probing metrics to evaluate expert representations. Experiments in four environments (GoalCycle, Overcooked, Brax Ant, and Travelling Salesman) show a positive relationship between social learning and expert modelling, with targeted interventions enhancing both in several settings.
Critical strengths
The paper presents a well-motivated information-theoretic framework and blind-corrected probing metrics to distinguish expert-specific representations from shared environmental information. This systematic, interpretable approach advances the field by offering a clear causal account of emergent expertise, going beyond merely tracking improvements in social learning performance. The evaluation covers four distinct environments and uses controlled manipulations, correlation analyses, and targeted interventions to examine the relationship between social learning and expert modelling. The experimental breadth and transparent reporting of inconsistent findings enhance the credibility of the empirical contribution and support acceptance.
Critical weaknesses
The abstract could benefit from refinement to more clearly foreground the study’s novelty, particularly its theoretical and methodological contributions, which the main text articulates well, but the abstract's emphasis on empirical findings somewhat overshadows. Since the learner and expert generally share the same latent task, could the authors clarify how the observed expert-modelling capabilities differ from representations of task-relevant information inferred through expert observation, beyond what the blind-corrected probes and misaligned GoalCycle experiment already establish? Since earlier work, especially Bhoopchand et al. (2023) “Learning Few-Shot Imitation as Cultural Transmission”), has already examined internal representations of social information and goal states, could the authors more clearly explain how their study of emergent expert modelling goes beyond these analyses? The Halfway condition is evaluated using total episode returns; could the authors clarify how this metric distinguishes genuine in-context learning and retention from performance gains accumulated while the expert is still visible, and whether performance after the expert disappears was assessed separately? In Section 4, the information-theoretic analysis establishes a lower bound on expert dependency; however, could the authors clarify the assumptions under which improvements in social learning necessarily imply nonzero expert dependency, beyond the data-processing inequality established in Appendix F.1? In Section 5, social learning is measured across training checkpoints, while expert modelling is evaluated after training; could the authors clarify whether the reported correlations hold when both metrics are evaluated at corresponding training stages? The default Overcooked masking in Section 6 removes only the expert's symbol while retaining observations of its kitchen; could the authors clarify whether indirect information about the expert remains accessible in the Solo and blind conditions, and how this affects the reported expert-modelling results? In Section 7.3, the authors propose that interventions may improve exploration and representation learning more generally, and that alternative intrinsic rewards produce comparable benefits in Appendix G.6; could the authors clarify what evidence specifically supports expert modelling as the mechanism driving improvements in social learning? In Section 8, Brax shows no significant correlation between social learning and expert modelling, and that some interventions produce inconsistent results across environments; could the authors clarify how these findings support the broader conclusion that expert modelling and social learning are tightly coupled, and under what conditions this relationship is expected to hold?
Questions for authors
The abstract could benefit from refinement to more clearly foreground the study’s novelty, particularly its theoretical and methodological contributions, which the main text articulates well, but the abstract's emphasis on empirical findings somewhat overshadows. Since the learner and expert generally share the same latent task, could the authors clarify how the observed expert-modelling capabilities differ from representations of task-relevant information inferred through expert observation, beyond what the blind-corrected probes and misaligned GoalCycle experiment already establish? Since earlier work, especially Bhoopchand et al. (2023) “Learning Few-Shot Imitation as Cultural Transmission”), has already examined internal representations of social information and goal states, could the authors more clearly explain how their study of emergent expert modelling goes beyond these analyses? The Halfway condition is evaluated using total episode returns; could the authors clarify how this metric distinguishes genuine in-context learning and retention from performance gains accumulated while the expert is still visible, and whether performance after the expert disappears was assessed separately? In Section 4, the information-theoretic analysis establishes a lower bound on expert dependency; however, could the authors clarify the assumptions under which improvements in social learning necessarily imply nonzero expert dependency, beyond the data-processing inequality established in Appendix F.1? In Section 5, social learning is measured across training checkpoints, while expert modelling is evaluated after training; could the authors clarify whether the reported correlations hold when both metrics are evaluated at corresponding training stages? The default Overcooked masking in Section 6 removes only the expert's symbol while retaining observations of its kitchen; could the authors clarify whether indirect information about the expert remains accessible in the Solo and blind conditions, and how this affects the reported expert-modelling results? In Section 7.3, the authors propose that interventions may improve exploration and representation learning more generally, and that alternative intrinsic rewards produce comparable benefits in Appendix G.6; could the authors clarify what evidence specifically supports expert modelling as the mechanism driving improvements in social learning? In Section 8, Brax shows no significant correlation between social learning and expert modelling, and that some interventions produce inconsistent results across environments; could the authors clarify how these findings support the broader conclusion that expert modelling and social learning are tightly coupled, and under what conditions this relationship is expected to hold?
We thank the reviewer for the thoughtful review. We respond to the reviewer's concerns below. We've marked the changes in blue in the revised version of our manuscript.
ZSU7-W1 Abstract
We have revised the abstract to better emphasise the theoretical and methodological contributions. It now leads with the result that social learning requires encoding expert observations, then introduces the blind-corrected probes and the three hypotheses they test.
ZSU7-Q1 Expert vs task modelling
Task information alone would not let the learner predict the expert's momentary behaviour, such as its current action, which is what we primarily measure. For example, inferring the recipe in Overcooked requires modelling the expert's actions and positions, which is not reducible to the recipe itself. In the same way, ΔSubtask is positive even though only one subtask (choosing ingredients) reveals the recipe. Finally, when the task is given to both the learner and the blind baseline (H1), task information cancels out of ΔX-Pred, yet modelling stays above zero.
ZSU7-Q2 Relation to Bhoopchand et al.
We now discuss this in Related Work. Their work is complementary to ours. They probe whether the agent encodes the expert's presence and the agent's own goal entries. As they note, an auxiliary attention loss is at least partly responsible for the expert-presence representation.
By contrast, we probe unobserved expert properties (e.g., actions, hidden states, and subtasks), without an expert-directed objective, correct against an expert-masked baseline, and relate these representations to social learning across conditions and environments.
ZSU7-Q3 Retention
We initially adapted the protocol and metric from Bhoopchand et al. (2023). As you say, only the gain after the midpoint isolates retention. We now clarify this in Section 3.3. We also measure retention in App. E.1. The share of the second-half gain kept once the expert leaves is high in TSP, partial in GoalCycle 7×7 and near zero in Brax, and in Overcooked and GoalCycle 13×13 the seeds with the highest social learning retain much less than moderate ones.
ZSU7-Q4 Assumptions
We have added a paragraph to App. F.1 stating the assumptions explicitly: (a) the expert can influence the learner only through the expert-observation channel; (b) the learner's policy and critic are functions of its latent alone; and (c) the latent is updated recurrently. The learner's returns can then differ from Solo only if its actions depend on expert observations, which must pass through the latent encoding.
ZSU7-Q5 Matched training stages
Yes. With the same controls as Fig. 4 and both metrics averaged over training, every Fig. 4 correlation stays positive at similar strength, with all but one BCa interval excluding zero; Brax stays null. With both taken from the final evaluation, the GoalCycle correlations are unchanged and Overcooked is weaker. TSP's evaluation SLM is too noisy to rank seeds reliably.
ZSU7-Q6 Overcooked masking
We have expanded the Overcooked description in App. C. Solo and the blind baseline mask the expert's position, direction and inventory, so its kitchen remains visible (e.g. ingredients it has already placed in its pot). The recipe indicator is masked in both kitchens in every condition.
This makes both measures conservative. The blind baseline sees the same kitchen, so ΔX-Pred counts only information beyond it. Any cue the kitchen gives the Solo learner raises the Solo baseline, lowering SLM.
ZSU7-Q7 Mechanism
Prediction-based interventions could act through better exploration or representation learning in general, and we do not fully separate these from the effect on expert-modelling. Several results nevertheless point to expert-modelling being a key factor:
ZSU7-Q8 Brax and conditions
Intervention effects varied across environments. Across 13 of 14 comparisons in Figure 6, social learning and expert modelling either both improved or neither did. In Brax, both SLM and expert modelling declined when the expert became less informative (H1) and improved under an intervention (H3). The across-seed correlation in Brax remains inconclusive (H2) because Expert Modelling varied little across seeds, possibly because learners could directly observe the expert’s state. We expect the two measures to move together when the expert’s behaviour provides information the learner needs to solve the task.