- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper asks whether a generator’s own outputs can improve attribute control after source access ends, using requested attribute tuples as noisy labels for generated bodies. It trains a lightweight body-only verifier on these request–body pairs and freezes it to guide reinforcement fine-tuning or best-of-eight reranking.
Critical strengths
The paper attempts to address the attribute control problem from an output-verifier perspective, without requiring human annotation or LLM annotation.
Critical weaknesses
-
There is some gap between the description of the framework in Figure 1 and the method section; for example, the variable does not appear in the method. I suggest the authors revise the paper so that the method and the framework figure correspond more clearly.
-
I have concerns about the implementation of the verifier. In line 169, the authors state that TF-IDF is used, but the specific implementation details are not clearly presented.
-
I think the place where the output-verifier is most likely to work is that it uses clusters to smooth the noisy tags, together with a TF-IDF-based verifier. However, I remain concerned about the upper limit of these rather simple approaches: are they really sufficient to solve problems in general scenarios?
-
I think the experimental validation is insufficient. The authors appear to have conducted only a very limited baseline comparison, and the results do not look particularly strong. This raises doubts about the design of the method.
Questions for authors
See Weaknesses
6 comments
I thank the authors for their response. However, because the authors’ rebuttal comes late, I can only provide feedback near the end of the rebuttal period. In the rebuttal, the authors do provide many detailed explanations and additional experiments, and I acknowledge that some issues are addressed. Nevertheless, I still have two concerns. First, regarding the verifier, I feel that, due to noise, I still cannot fully accept this approach. Second, regarding the experiments, as I mention in Weakness 4, Tables 1 and 4 give me the impression that only recovery improves, while the other metrics do not look good. Therefore, I retain my score. That said, I acknowledge that my understanding may have some deviations, so if other reviewers or the chairs strongly support this paper, I would remain open to it.
Thank you for your response, Reviewer 65mr. We understand and sincerely apologize for the delay. Our reply took longer than anticipated for two reasons: we received the third review late, and running the additional experiments required extra time. We look forward to any further comments you may have.
Response to Reviewer 65mr
We thank the reviewer. We will clarify the three points raised below, and we added the requested baselines.
A1. Thank you. G₀ is the adapted generator, L4 in the primary experiment, not the frozen pretrained model. It supplies the verifier-fitting bodies, initializes RFT, and is the KL reference. We will define it at first use in Section 3 and use the same notation in the method and Figure 1.
A2. We will state the full configuration in Section 3.4. It is currently only in Appendix G.4. The verifier reads only the body (its first 600 characters). One TF-IDF vectorizer with at most 5000 unigram and bigram features is shared across attributes, followed by one class-balanced logistic-regression head per attribute (scikit-learn, L2 regularization with C = 1.0, lbfgs solver, at most 300 iterations). It is fit once on generated bodies paired with their requested tags and then frozen, and the reward is the fraction of requested attributes whose predicted value matches the request (Eq. 1). In the new label-source study, each seed uses 4000 fitting bodies and 992 diagnostic bodies.
A3. Clusters only decide which attribute tuples are requested. They do not smooth or correct the verifier's labels. The information flow is:
The verifier never receives the cluster or the requested tuple as input. We will draw this flow in Figure 1.
On the upper limit, the method is not tied to TF-IDF: any body-only classifier fit on the same pairs can supply the reward in Eq. 1. We agree that a lexical verifier has a ceiling: gains concentrate on attributes with lexical signatures, such as topic and formality, and urgency does not improve (0.385 → 0.363 in the new full-budget runs). Our judge-label ablation changes the labels but keeps the lexical verifier, so a stronger verifier remains future work.
A4. We added the controllable-generation baselines this point asks for: engineered and plan-then-write prompts on three models and a judge-labelled verifier (response to PF7S, Table R2), plus full-budget runs with a new selector (response to the Area Chair, Table R1). Under an independent judge, requested-tag RFT improves recovery over the adapted generator by +0.076 [+0.047, +0.102]. Strong prompting reaches higher raw adherence (0.623 for 27B plan-then-write versus 0.521), but with much worse domain fidelity, so we present the results as an adherence-fidelity trade-off rather than a ranking. We have not run matched comparisons with CTCL, DP-RFT, or guided decoding, and we will say so. DP-RFT's reward consults the private corpus at every update, which our setting rules out, and CTCL is an upstream generator design to which our control stage could be applied.
We hope these responses and the new results address your concerns.