acceptodds
Desk rejected at ICLR 2027

Agents, Think Only When it Helps

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

LLM agent distillation transfers agentic capabilities to student models by training on complete teacher trajectories that typically interleave thoughts and actions. We question how critical thought supervision is for agents' ultimate action performance. To answer this, we disentangle their impacts by independently perturbing teacher thoughts and actions during distillation. On widely adopted agent benchmarks across model scales, altering or discarding thoughts has little effect on student performance. We find, however, that thought supervision becomes important when information about the thought is not disclosed in the corresponding action during training. By identifying such discrepant cases, our findings challenge the common practice of distilling entire teacher trajectories for LLM agents, and offer a new insight that thought supervision can be a discernible choice for efficiency depending on whether the information of thought is disclosed in the action. An anonymized version of the code is provided in the supplementary material for reproducibility.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agent distillation, Thought supervision

Related papers

Discussion (0)

Sign in to comment.