- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ∆1-RSG, a framework that generates formally grounded reasoning supervision data from self-certifying theorem structures. It constructs five tasks: conclusion inference, support sufficiency, abstention, missing-support identification and support repair. Experiments on Qwen2-1.5B show that ∆1–RSG improves reasoning performance when evidence is insufficient.
Critical strengths
-
Compared with training only on samples with complete premises, training on RSG data improves the model's reasoning performance when evidence is insufficient.
-
The proposed data synthesis method is lightweight and does not require LLM API calls.
Critical weaknesses
-
The experiments are limited. The experiments do not adequately support the paper's claims. For example, the paper lacks comparisons with existing logical reasoning methods and ablation studies investigating the contributions of the five tasks.
-
The mathematical proof provides limited justification for the design of the five supervision tasks.
-
The main contribution is a new method for generating logical reasoning supervision data. Although Appendix E provides illustrative examples, actual samples from the generated dataset are missing. It is unclear whether the data consists of abstract logical statements (e.g., "red is warm", "warm is Alice") or meaningful natural language statements (e.g., "the sun rises in the east", "the Earth rotates"). This distinction is important for assessing generalization to real-world natural language reasoning.
Some Suggestions (Do Not Affect the Rating)
-
The Keywords below the abstract should be revised to comply with the template.
-
The paper addresses problems closely related to the following works, but does not discuss them:
- A Balanced Neuro-Symbolic Approach for Commonsense Abductive Logic, ICLR 2026.
- Open-World LLM Logical Reasoning, ICML 2026.
Questions for authors
-
Could the authors provide more detailed statistics on the training and test data distributions?
-
Since Missing-support Identification and Support Repair use identical supervision targets, why are they treated as two separate tasks?
-
Suppose the complete premise set is {A → B, B → C, C → D}, and C → D is removed. If the model instead adds A → D, how would this response be evaluated in the Missing-support Identification and Support Repair tasks?
-
How do proprietary LLMs, like GPT-5.6 and Claude Opus, perform on the five tasks?