- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
VCrectify maintains a numerical perturbation predictor alongside an explicit knowledge-based reasoning state. Disagreement between them guides the acquisition of experimental evidence, and observed outcomes determine which state is updated. Numerical updates use replay; knowledge updates adjust the reliability of cited information. Experiments span several prediction and reasoning backbones, perturbation datasets, and transfer settings. Acquisition and update ablations test the workflow, while a blinded expert comparison assesses its rationales. The results support the combined prediction approach, although correction of individual knowledge items and retention over successive updates are less directly evaluated.
Critical strengths
-
The two-state design addresses a useful problem. New measurements need not correct a numerical predictor and an explicit knowledge state in the same way. Connecting disagreement-based acquisition with selective updates gives the framework a clear role in adapting virtual-cell models as experiments arrive.
-
The gains extend across several backbone combinations (Tables 1–2), with update-all, random-correction, and random-acquisition controls. The blinded rationale assessment also uses the same graph and experimental memory in both conditions (Appendix C.4). These comparisons make the evidence stronger than a collection of prediction scores or selected explanations alone.
Critical weaknesses
-
The sample-efficiency result needs stronger attribution. Random acquisition uses matched experimental budgets, and standalone baselines receive the full training split. However, numerical-uncertainty and diversity-based acquisition strategies are not compared. Total gradient steps and optimization schedules are also insufficiently specified for the sequential-versus-joint comparison. Thus, the benefits of the disagreement rule are not fully separated from selection strategy and training exposure.
-
Prediction correction is better established than knowledge correction. Equation 7 updates the reliability of all cited knowledge from the final prediction outcome, without identifying which item caused the error. Expert source checks support rationale quality, but Appendix C.4 compares final learned weights with unit weights. That comparison combines initial calibration and continual correction, so it does not isolate the latter. The authors disclaim causal validation; the remaining issue is attribution to individual knowledge items and the continual update stage.
-
The retention analysis mainly links the adaptation coefficient to final prediction error (PDF p. 9). It does not show whether earlier queries remain accurate as new experiments arrive. Replay is a reasonable mechanism, but the claimed retention benefit needs a direct assessment of forgetting along the update stream.
Questions for authors
-
What batch sizes, acquisition rounds, and total gradient steps are used in the full, random-acquisition, and joint-training variants? Is a matched comparison with numerical-uncertainty or diversity-based acquisition available? The experimental budgets are already reported as matched; the question is whether selection and optimization exposure explain the gains.
-
Can final reliabilities be compared with calibrated initial reliabilities held fixed, using the same final graph and experimental memory? The current unit-weight control includes both calibration and continual correction. For item-level examples, how do reliability changes relate to the source checks performed by the experts?
-
Can saved checkpoints be evaluated on a fixed set of earlier queries after each update round? This would show directly whether replay preserves previous performance during the existing update streams.