- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies concept unlearning in multimodal reasoning models, where unwanted information may appear in the reasoning trace even when it is absent from the final answer. CoCoUN trains the model to switch into a short sequence of continuous hidden states before mentioning the target concept, then resume normal text generation. Its training objectives encourage this transition, suppress target-token probabilities during the hidden computation, and preserve a useful continuation. Experiments report improved full-trace unlearning rates across two models, alongside tests of retained ability and resistance to recovery attacks.
Critical strengths
-
Replacing a target span metion with a localized period of hidden computation is interesting.
-
The reported improvements on R1-OneVision are substantial compared with the tested baselines.
-
The component ablation provides evidence that the different training objectives work better together.
Critical weaknesses
The main contribution of the work is the gap between answer-level and full-trace unlearning, but R-MUSE has already identified reasoning-trace leakage in multimodal models and evaluated it directly. The author also cites and uses as a baseline. So the problem formulation should not be the main contribution. Other works, R^2MU and R-TOFU, also address the same issue in reasoning models and are not discussed. Therefore, the contribution should be reframed on the proposed mechanism, while the problem and the evaluation protocol are prior work.
On evaluation, Mulberry, MathVista accuracy increase from 26.9% to 40.2% after applying CoCoUN at the baseline, and several baselines also improve. The increase in reasoning accuracy needs further explanation. Does it reflect better problem-solving, or changes in answer formatting and completion within the generation limit? A method that is intended to make a model forget information should not make it better at reasoning. In Appendix B, the authors use a 512-token budget, so changes in length, completion or formatting could produce this gain. I would recommend that the authors report extraction failure and truncation rates, and results under a longer budget. Retention is also measured on one small MathVista subset, which does not show whether related concepts or general description are preserved. Also, CoCoUN does not beat the baseline in full-trace unlearning for Taylor Swift and Hello Kitty.
Questions for authors
-
How was R-MUSE adapted, and were its original steering components retained?
-
Which tokens or representations receive forgetting supervision in each baseline?
-
What are the answer-extraction failure rates, truncation rates, and output lengths on MathVista before and after adaptation?