- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper addresses the problem of large language models (LLMs) violating physical conservation laws during reasoning, even when producing correct final answers. To address this issue, the authors propose Conservation-Aware Fine-Tuning (CAFT), which employs a rule-based conservation-law verifier to detect physical violations in reasoning chains and constructs preference pairs using four strategies to optimize the model's reasoning process through Direct Preference Optimization.
Critical strengths
1.CAFT achieves substantial improvements in answer accuracy on the 1.5B model under sampled decoding, demonstrating its effectiveness within the evaluated setting. 2. The paper includes extensive ablation studies and controlled experiments, providing useful evidence for assessing the effectiveness and limitations of the proposed method.
Critical weaknesses
1.The proposed method relies heavily on a conservation-law verifier, which primarily detects predefined patterns of incorrect physical statements. Consequently, the problem addressed by this paper is largely limited to specific verbalized physics misconceptions, rather than general physical reasoning correctness.
Notably, both the data construction statistics and experimental results indicate that the base model already exhibits a relatively low rate of violations detected by this verifier. Nevertheless, approximately 64.5% of the DPO preference pairs are constructed by artificially injecting predefined violations. In other words, the authors devote substantial synthetic supervision to teaching the model to disprefer error patterns that are already infrequently observed in its natural outputs. Given that the overall framework mainly combines existing techniques, including rule-based verification, synthetic error injection, and standard DPO, while addressing a relatively narrow problem, I find the methodological novelty and potential impact of this work limited.
2.There is substantial overlap between the training supervision and the primary evaluation metric. Specifically, the authors first construct rejected responses by injecting predefined violation patterns, use the corresponding rule-based verifier to construct preference pairs, and then evaluate the fine-tuned model using the same verifier. Consequently, the observed reduction in Conservation Violation Rate (CVR) may partly reflect the model learning to avoid specific predefined expressions, rather than a genuine improvement in its physical reasoning capabilities.
Furthermore, the so-called out-of-distribution (OOD) evaluation still relies on the same eight problem templates, with only numerical parameters changed. The paper does not sufficiently clarify the actual distributional differences between the in-distribution (ID) and OOD test sets. Therefore, it remains unclear whether the OOD evaluation demonstrates meaningful generalization beyond the problem distributions already covered by the training data.
Questions for authors
1.As I understand it, one of the main motivations of this work is to address cases where an LLM produces a correct final answer but makes physical errors during its reasoning process. Could the authors evaluate CAFT on a widely used public physics reasoning benchmark and use independent LLM evaluators to assess the physical correctness of the generated reasoning? In particular, it would be valuable to specifically examine responses with correct final answers but physically incorrect reasoning steps.
2.What are the specific differences between the ID and OOD test sets?
3.Why does CAFT show substantial improvements under sampled decoding but worse performance under greedy decoding? Could the authors provide further analysis and explanation for this discrepancy?