- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Addressing the "strategy mismatch" problem, where different subtypes of temporal question answering (QA) require distinct reasoning procedures, this paper proposes a framework that incorporates graph-based construction, code generation, direct extraction, and hybrid reasoning. It trains a student policy through multi-step trajectory distillation (from a teacher model) and policy-conditioned reinforcement learning (RL).
Critical strengths
The paper identifies a clear, well-defined contribution: it highlights "strategy mismatch" as a core challenge, demonstrating through factorial experiments that different temporal sub-types favor specific reasoning procedures.
It unifies diverse reasoning procedures into a set of policy labels and distillable trajectories. By compressing multi-step teacher execution into direct generation by the student, the approach balances comprehensive reasoning capabilities with deployment efficiency.
Critical weaknesses
The approach relies heavily on predefined subtype mappings and teacher-based routing. Such fixed mappings underperform compared to a question-level "oracle," and the fact that strategy labels remain largely static after reinforcement learning suggests the student has failed to fully master dynamic, question-level strategy selection.
The comparison between the teacher-scaffolded approach and the distilled student model conflates changes in the model family with changes in the execution structure, making it impossible to isolate the causal effect of the distillation process. Furthermore, the failure of the "cold-start" control experiment leaves the value of structured strategy trajectories insufficiently demonstrated.
Key components, such as route-related weights, normalization factors, and reward subsets, are largely hand-tuned. The study lacks comprehensive ablation studies or sensitivity analyses for these reward components, and the evaluation of generated artifacts is limited to representations that can be reliably parsed.
Questions for authors
Please refer to the weaknesses.