W1. The construct-validity experiment may measure familiarity, not dated recall. The declarative margin in Section 5.3 may only show that the model has seen a price alongside the company, not that it knows which date the price belongs to. The paper’s own diagnostic on never-written facts points this way. If so, the readout gap concerns familiarity and does not transfer to the date-specific financial probes. The authors should report the margin on pairs where both prices were written. They should also confirm that the forced-choice readout still works on the synthetic companies after training.
W2. The financial application cannot answer its motivating question. Either Section 5.3 holds, in which case a forced-choice null says little about what the models hold, as the authors concede. Or, per W1, the readout’s validity for dated facts is untested. Either way, the Section 6 null does not tell us whether the models memorise market data. Applying the readout that recovers facts in Section 5.3 to real market data would be the natural fix.
W3. Limited novelty. Prior work on multiple-choice evaluation and knowledge probing has already established two things: base models often fail multiple-choice formats, and forced-choice questions can underreport what a model encodes. The contribution is therefore incremental. The claim that format competence is not monotone in model size is also unsupported, since the compared models differ in training data, not just size.
W4. Presentation. The paper is hard to follow for an ICLR audience. Key results are in the appendix, many numbers are reported in long prose passages instead of tables, and econometric terms are used in the main text without definition.