- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose LIFT-Bench, an instruction following eval (with an accompanying training set). It is constructed through pulling seed topics from wikipedia, creating a report sketch, and then annotating it with constraints (both verifiable and LM-judged). Evaluation shows that models struggle on flat-style tasks especially, and further analyses shows that smaller models struggle more with higher numbers of constraints, along with constraint-type analyses. The authors also create a train set equivalent of LIFT-Bench, LIFT-Train, and show it generalizes better to other instruction following benchmarks than IFTrain, prior work.
Critical strengths
- The evaluation setup and analysis are interesting and impactful, in particular the document size, constraint type failure analyses are nice to see, and the distinguishing of Kimi K3 and GPT 5.5 in the analysis is interesting.
- I think the training results showing OOD performance does not drop is solid, and this work seems pretty clearly to be an improvement over IFTrain, its predecessor, as well as being a reasonably 'new' shape of IF benchmark by focusing more on report writing.
Critical weaknesses
- The LM judge quality is poorly validated. There is no details (even in the appendix) on how annotators were recruited, how many there were, or inter-annotator agreement. This is especially important as the judge is much weaker than many of the models tested in the evaluation, which themselves could be 'hacking' the judge. The study in Appendix C only has 78 invalid samples, and the judge accepts 51 of them.
- The benchmark is fairly saturated already for frontier models, with GPT 5.5 already almost at 100%. Benchmarks solvable by frontier models are still valid, but it would be more interesting and exciting if this work used its synthetic nature to really push to document sizes or constraint difficulties that frontier models struggle with - or spend more time on analysis of existing smaller models (e.g. focusing on training or on the GPT vs GPT-nano analysis) then proposing this benchmark.
- The reward hacking analysis is a little lacking: it would be nice to see more analysis (e.g. human annotations) on if the LM judge (which is deciding most of the score!) is being reward hacked in this scenario, and to what degree it is (cheaper/easier alternatives could be to use a set of model judges, and/or use much larger, newer, stronger models). Given LM judges are known to be biased, do the benchmark scores reflect real human scores even for the GPT 5.5 outputs? Over the course of RL training, does the LM judged quality diverge from human ratings? You could argue that hacking an LM judge is harder, and there is some evidence that this setup keeps the model from collapsing (e.g., from the length analysis), but it's tricky to say that it isn't reward hacking in different ways. It would also be nice to compare the reward hacking to other work, e.g. does training on the MOSAIC set generalize worse/better to OOD evals than LIFT-Train?
Right now, I am slightly negative due to the lack of details, but answering my questions and the concerns above could make me lift my score.
nit: Appendix H seems vacous, it references two tables already in the main text. I assume it would refer to Figure 8 but it doesn't state so in the text, and the floating nature of the figures makes it difficult to determine.
Questions for authors
- Is a wikipedia snapshot used for generation, or is wikipedia just directly queried?
- How were human annotators gathered, and how did you ensure they provided quality annotations?
- What do non-IF evaluations look like when training on LIFT-Train, and how does it perform when trained on in multi-task scenarios? These can be alternative ways to avoid reward hacking, e.g. Olmo 3 finds that mixing domains seems to prevent IF reward hacking to some extent (https://arxiv.org/abs/2512.13961).
3 comments
Hi, thanks for the response! The details are interesting and address some of my concerns. Considering the revised grader could change model rankings and potentially some of the analysis, I’d like to see that rerun before acceptance.
As for saturation, I still think 0.93 leaves limited headroom. The flat split is more convincing in having headroom, but then do we need the full pipeline just for the flat tasks? I think my original point stands that it would be interesting to push the structured tasks harder to see if there is more headroom to be had in the benchmark.
Finally, I think the wikipedia setup is still not fully reproducible - it would be good to get a promise of shared code/prompts, or just access dates/revision IDs for the article snapshots.
Overall, I’m keeping my current score, considering the issues around the LM judge changes.
Thank you for the follow-up, and for engaging with the details.
Regarding task saturation, we agree that the structured splits currently leave limited headroom for frontier models. Pushing them harder is possible within the same pipeline, since density and document size are generation parameters, and adding further constraint types is another direction; both need further experiments, which we intend to do, but didn’t have enough time to explore. On whether the full pipeline is needed: the structured tasks are far from solved for smaller models. These models still struggle on the structured splits and improve substantially when trained on LIFT-Train. We expect this to matter more once harder structured tasks open up the headroom, as you suggest: the same pipeline generates both the harder benchmark and its training data.
For rerunning with the revised grader, We agree. We are currently rescoring all models and splits with the revised judge setup, and the final version will report the updated results and model rankings, with any affected analysis revised accordingly.
Regarding Wikipedia reproducibility, our code, dataset and the Wikipedia topics we used are already released; the links are anonymized in the submission for double-blind review and will appear in the final version. The release includes the exact list of topic titles we sampled from. Since only titles are used, and no article text, this list is everything needed to reproduce the seeding step. The source list is also cited in the paper (Wikipedia: Vital Articles, Level 5, 2023; a historical snapshot of about 40,000 unique topics).
W1 / Q2. The LM judge is poorly validated; how were annotators gathered and their quality ensured?
Please see "Judge reliability" above.
Annotators. The meta-evaluation set was labelled by in-house annotators, who were given only a broad satisfaction criterion. The re-labeling showed this was too loose: annotators and judges were never given the grading standard. We corrected 56 of 305 labels (18%) and wrote the standard down. We will add these details and the standard to Appendix C.
False accepts. For the adopted judge, Appendix C reports 45 false accepts among the 78 invalid cases (scoped Gemma-4-31B). On the revised labels, out of 24 invalid labels it accepts 3 and scores 0.81 macro-F1.
W2. The benchmark is fairly saturated for frontier models.
We respectfully disagree. When considering the structured splits, once more constraints are added, then even frontier models are not saturated: GPT-5.5 drops from 0.97–0.98 on the few splits to 0.93 (short) and 0.91 (long) on numerous. Density and document size are generation parameters, so harder splits can be generated by raising them, with no new annotation.
In addition, flat splits exhibit much lower performances (0.71 on average for GPT-5.5).
W3. The reward-hacking analysis is lacking.
We agree that judge-based rewards need closer scrutiny.
Is the LM judge being reward hacked? We do not claim the policy is perfect. Our claim is that a judged content reward is much harder to exploit than the purely programmatic checkers common in prior instruction-following RL work, which reward surface compliance regardless of answer quality. Under checker-only rewards (IFTrain), median response length collapses (from 114 words to 28.5 by step 180 and to 18 by step 390); under LIFT-Train it stays (between 389 and 533.5 words). The share of short, high-reward completions reaches 28.5% at step 180 under IFTrain, and stays at 0% under LIFT-Train.
Stronger judges and judge ensembles. We followed this suggestion for the meta-evaluation: with the grading standard stated, frontier and open judges agree with the revised human labels at 0.83–0.88 macro-F1. An ensemble, or a fine-tuned rubric judge as in AdvancedIF, would further reduce exposure to any single judge's biases. We mention this as future work in the conclusion.
We did not collect human ratings of trained-model outputs over the course of RL training, as this would be very expensive. So we cannot say whether judged quality diverges from human ratings during training. We also did not train on MOSAIC, so we cannot compare its out-of-distribution generalization with LIFT-Train's. We will state these as limitations.
Q1. Is a Wikipedia snapshot used for generation, or is Wikipedia queried directly?
Wikipedia is a snapshot, and we only use it for titles. Topics are seeded from a historical snapshot of the Wikipedia Vital Articles list (level 5, about 40,000 topics, out of which we use 4,000). Article text is never used and Wikipedia is not queried during generation. The topic is consumed only as free text, so the list can be swapped for any topic source.
Q3. What do non-IF evaluations look like after training on LIFT-Train, and how does it perform in multi-task training?
We have not run non-IF evaluations or multi-task training. We agree that mixing domains is a complementary way to limit reward hacking, as Olmo 3 reports, alongside strengthening the reward signal as we do here. Mixing would also help preserve non-IF capabilities such as math and coding. We will add this to the discussion as potential future work.