- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Search-augmented neural networks like AlphaZero learn from their search policies through MCTS. This paper aims to measure the extent to which specific tactical priors get absorbed into the network. They experiment on AlphaZero in two game settings: Go and Gomoku, introducing a hard coded tactical prior into AlphaZero's training-time search for both games. Their training includes unguided, soft guided, and forced guided settings, which vary the strength of the priors. After training, the authors test how much of the behavioural effect associated with each prior remains when search is removed, finding a partial effect.
Critical strengths
- the cross-phase experimental design (mostly) isolates the absorption of a specific tactical prior into the policy network, which is good
- It's nice to see a Go paper in a sea of chess papers; Go is an under-utilized domain
Critical weaknesses
- incredibly difficult to read, LLM like prose
- their Go agent is extremely weak, hard to know whether lessons generalize to strong game playing agents that people care about
- only one tactical prior per game
- obvious result, questionable relevance and impact to either game playing AI or the policy distillation literature
Questions for authors
- what does this paper establish beyond the fact that policy distillation w/search works?
- why not turn search all the way off for your prior-off condition, which still uses 32 MCTS rollouts?