- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper tests if a small change in a behavioral model's predictions after reward removal allows the inference that the behavior generating process relies little on rewards. Authors used two tasks with known generating policies (a 4-armed restless bandit task that combines reward learning and last choice kernel, and a spatial correlated bandit that combines reward generalization with a local choice tendency). The authors vary reward contribution across seven weights. The tested models include GRUs, transformers and fine tuned Llama 3.1 70B, Rescorla-Wagner model, per participant fitted generator and an oracle that uses ground truth generating parameters. Study separates retraining without rewards from donor replacements. Each model's change in choice probabilities is then compared with oracle's - this constitutes response fidelity. Networks trained without rewards still predict choices well. In donor replacement, accurate networks respond far less than the generator. Prediction and fidelity rankings can also dissociate. The authors conclude that ablation responses must be validated separately from predictive accuracy before drawing conclusions about behavior-generating processes.
Critical strengths
A clear and careful separation of two manipulations (retraining without an input and fixed model perturbation), and a well designed response fidelity measure are major strengths of this paper.
Critical weaknesses
I found the major weaknesses to be lack of novelty/surprising result - given that choices encode past rewards and are held fixed under replacement, the networks' low sensitivity is largely expected, making the paper's contribution more about quantification of this than discovery. Also the paper shows rankings can flip between tasks but doesn't provide an explanation for why that might happen, so there's no way to predict when an ablation might mislead.
Questions for authors
-
The oracle's epsilon greedy policy assigns a high probability to its best arm, so donor rewards make it confidently wrong which inflates delta NLL. Do the results change under a softer policy that's more typical for fitted human data?
-
Is there a reason why generators didn't use parameters estimated from real human choices?
-
Would it be possible to add larger spatial samples, as the RW-Llama comparison relies on only 50 task participants per weight?
-
Do you have a possible explanation for why Llama responds more faithfully than RW in the spatial task but less in the restless task (eg. different meaning of w, RW's misspecification, or something llm-specific)?
Dear Reviewers,
Thank you for your time and thoughtful reviews. Please read the authors’ response and share your follow-up comments promptly, indicating whether your concerns have been addressed. Please raise any additional questions early enough to give the authors sufficient time to respond.
Thank you.
Best,
AC