- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper tests whether models can reconstruct learned policies as text and use that text to guide later actions. Its main strength is the careful separation of reconstruction, interpretation, execution, and invocation through controlled interventions. The results show strong operation following, but weak interpretation of new predicate values and limited reliability across branches. Overall, the paper provides a useful evaluation framework, but only preliminary evidence for decomposable learning.
Critical strengths
-
The paper clearly separates reconstruction, interpretation, execution, and invocation. This provides a useful framework for evaluating textual policy interfaces.
-
The controlled interventions identify both successful operation following and failed predicate interpretation. This supports careful and well-bounded conclusions.
Critical weaknesses
-
Training includes all single triggers and ordered pairs of distinct triggers. Testing uses six permutations of three distinct triggers. This provides limited evidence beyond the observed training structures. Testing a larger trigger set or greater held-out depth would strengthen the evaluation.
-
The novel predicate-value intervention shows −0.35 percentage-point matching-versus-off-target discrimination. This weakens the claim that the model interprets newly written predicate values. A direct-rule capability probe could identify the cause.
-
Pooled operation-swap following is 82.0% for LLaMA and 85.2% for Qwen. However, two rare LLaMA branches achieve only 4.5% and 2.0%. This shows strong variation across branches. The branch-specific results should be emphasized.
-
Preserving edits cause 16.1% decision churn. Thus, edits that preserve the prescribed decision for a record can still change the model’s decision. Further controls could examine this instability.
-
Only 71.5% of saved free runs reach the canonical editable prefix. All qualifying prefixes exactly match the gold prefix. Thus, editing after noncanonical reconstruction remains untested.
-
With informative policies, held-out depth-3 accuracy is 38.8% for starts 51–70 and 31.0% for starts 351–370. The paper should foreground these absolute results alongside the D–N advantage.
Questions for authors
Check my comments in Weaknesses.
3 comments
Thanks for the rebuttal. It is very hard to read and understand the response which is consistent with my earlier observation with the paper (hence a score of 1 for presentation).
Can you please simplify your response (and paper) a bit? Please add details. I am not against the usage of LLM for writing but please make sure that a human can read and comprehend it.
Plain-language response (part 2 of 2: points 4–6 and summary)
4. "Edits that should not change the decision still change it 16.1% of the time." In the workflow test we take one customer record, for example days=18, and edit the written threshold in two ways that move it the same distance from 18: days<=15 (the correct decision should change) or days<=21 (it should not). Comparing the two on the same record removes the effect of "any edit at all".
- The edits that should change the decision do so 64.2 points more often than those that should not.
- But the "should not change" edits still change 16.1% of decisions. Newly trained models show the same rate (16.0%), so this instability is real, and we will state it plainly.
- You asked for further controls. In the original test, the decision-changing edit usually moved the threshold further from its trained value (272 of 288 records). When we compare only edits matched on direction and size, the gap is still 49.9 points. One difference remains uncontrolled: how far the threshold moves from its trained value.
5. "Only 71.5% of the model's own runs could be edited, and all of those were already correct." Without our help, the model wrote a well-formed, editable workflow in 71.5% of runs, and in every one of those the text was exactly correct. So the original test never edited a run that had already gone wrong. We added that test with newly trained models (480 records each).
- When an earlier written check is wrong but still editable (46, 53 and 33 such runs for the three models), we correct it at the known spot. Correct final answers rise by 90.2 points on those runs.
- Across all records, including runs we could not edit, the gain is 8.3 points.
- We tell the model where the error is and what the right check is, so finding and fixing errors on its own remains untested.
6. "Low absolute accuracy." We will report the absolute numbers first, not only the difference from the control. On new three-step programs, D is correct 38.8% of the time with small starting numbers and 31.0% with larger ones; N is correct 9.5% and 7.7%.
- The help from the rule text depends on output format. When the model writes out its working, D beats N by 54.17 and 49.31 points; when it answers directly, the difference is only +4.31 and −2.78.
- A larger 8B model, trained on all pairs, reaches 78.5% on three-step programs. But swapping the order of the two branches inside a rule, which means the same thing and was never seen in training, costs it 30.4 points. This is another limit we will state.
Summary: what we claim and what we do not.
- The written rule does steer the model's next step for operations it already knows, in two model families, and in the workflow task an edited threshold redirects the decision.
- When the model recalls the wrong rule, giving it the right rule restores most of the accuracy.
- We do not claim that the model understands new condition values, composes pairs it never saw together without help, or is robust to rewording.
So we see the paper as a careful way to find where a written-rule interface works and where it breaks, not as proof that it works. In the final version, we will rewrite the paper in the same plain style as this reply, starting with the introduction and Section 5. If any part is still unclear, please tell us which one and we will explain it further.
Response to Reviewer ZHuQ
Plain-language response (part 1 of 2: setup and points 1–3)
Thank you for telling us the response was hard to read. Our earlier reply had to fit six answers into one 5,000-character comment, and it became too compressed. Here we restate it in plain language, with more detail on what each test did. There are no new results; every number is from our earlier reply or the paper.
The setup. The paper has two tasks.
Arithmetic task. We fine-tune a model on short programs. Each step is named by a code letter (X, Y or Z), and each letter stands for a two-branch rule, for example X = "if the number is odd, add 1; otherwise add 2". The test prompt gives only the letters, never the rules. Before each step the model writes the rule out from memory, then applies it:
t2=X rule={odd:+1 | even:+2} pre=56 branch=even op=+2 post=58We call this model D. As a control, model N is trained to write a meaningless text of the same length in the same place (
rule={q:v | w:zzz}). Comparing D with N tells us whether the content of the rule helps, rather than just having extra text.Workflow task. A separate model handles customer returns through four checks (e.g. "return window: days<=15"), and writes each check before applying it.
Our main tool is editing the written rule and seeing whether the model's next step follows the edit. If it does, the text is really being used.
1. "The test only recombines pairs seen in training." You are right: training included every ordered pair of letters, so a three-step test mostly chains familiar pairs. To test an unseen pair, we retrained with one pair (Y followed by X) removed.
For more letters and longer programs, we trained an 8B model with four letters on one- and two-step programs only. It solves 93.7% of three-step and 94.9% of four-step programs. However, every neighbouring pair was trained there, and it had more training data (1,600 vs 900 examples), so this shows longer chains of familiar pairs, not unseen pairs.
2. "The model does not understand new condition values." We agree; this is a limit of the paper. In the test, at a step where the number is 58, we rewrote Z's condition to "remainder after dividing by 5 equals 3" (true for 58) or "equals 2" (false for 58). If the model read the condition, it would take the first branch much more often in the "equals 3" case. It took it in 1 of 864 cases versus 4 of 864: no difference (−0.35 points).
As you suggested, we then tried a direct test of whether the model can compute such conditions at all. No model passed that test's own quality check (0 of 10, 0 of 6, 0 of 6 models), so we still cannot say whether the model fails to read the condition or fails to compute it. One finding helps: a condition the model already knows (from another letter) is followed when the whole rule is written out (balanced accuracy .963 and .977, where .5 means guessing), but not when it is mixed into the host letter's rule (.495 and .531).
3. "Rare branches." Swapping the two operations in a written rule (e.g. "+1" and "+2") changes the model's action 82.0% (LLaMA) and 85.2% (Qwen) of the time overall, but only 4.5% and 2.0% on LLaMA's two rare branches, which have 33 and 17 test cases per model. We will put these per-branch numbers in the main table. We also retrained with the rare branches shown more often. Swap following changed by −3 points and unedited accuracy on rare branches by +13 points; neither change is clearly different from zero, and accuracy on common branches fell by 11 points. So we no longer say rarity is the cause.
(continued in part 2)