- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Consero augments a frozen tabular foundation model with explicit, potentially incomplete or erroneous domain knowledge. It represents feature relevance as present, absent, or unknown and separately encodes signed feature-pair hypotheses. A synthetically pretrained knowledge-data reader combines these declarations with support-set residual and interaction statistics. Its knowledge-conditioned output is contrasted with an empty-template output to correct a frozen LimiX predictor. At deployment, an LLM generates and iteratively revises candidate templates using training-side prediction feedback, while the reader and backbone remain fixed.
Critical strengths
-
Clean and modular knowledge interface. The template explicitly separates declared relevance, declared irrelevance, and missing information, while pair declarations select where the reader should examine residual-interaction evidence. The empty-template contrast also provides a natural identity fallback. This is a thoughtful and interpretable design for incorporating editable hypotheses around a frozen backbone.
-
Strong controlled synthetic diagnostics. The paper evaluates correct, partial, fallible, sign-reversed, node-flipped, and address-shuffled declarations while holding the reader and data fixed, and additionally studies training scale, knowledge coverage, and already-explained references. These experiments convincingly show that the implemented reader can exploit addressed evidence within the synthetic task family.
Critical weaknesses
-
The headline improvements compare Consero with frozen LimiX, but Tables 10–11 show that adapted LimiX already accounts for essentially all of them. Binary macro-F1 is 78.51 for both adapted LimiX and Consero; multiclass macro-F1 decreases from 63.00 to 62.93; and regression NRMSE changes only from 0.3934 to 0.3932. Removing revision or reconstruction produces essentially identical results. Thus, the experiments support “task-adapted LimiX improves over frozen LimiX,” but not the paper’s central claim that language-derived knowledge provides the improvement.
-
Semantic validity and statistical robustness remain unestablished. Semantic and anonymous proposals are nearly indistinguishable, and the generated real-task proposals contain no known-absent nodes, leaving a central part of the proposed interface untested on natural data. In the synthetic LimiX experiment, a “wrong” template retaining the correct pair addresses even outperforms the correct template for regression (0.39616 versus 0.40080 NRMSE), while sign reversal has little effect. This suggests that address selection and residual evidence may matter more than the claimed semantics. The paper also reports no paired task-level confidence intervals, repeated LLM runs, repeated selection splits, or simple address only baseline.
Questions for authors
-
Is Consero’s improvement over adapted LimiX statistically significant under a paired per-task analysis?
-
Are predictions on the feedback, policy, and replay partitions generated using strict out-of-fold exclusion of the scored example’s label?
-
Why does the wrong template achieve lower regression NRMSE than the correct template with official LimiX in Table 5?
3 comments
Dear Reviewer cFYq,
Thank you again for your valuable feedback, which has greatly helped us strengthen our content controls and protocol clarity. As the discussion period closes soon, we wanted to follow up briefly. We hope the fixed-address content intervention, correction-scale analysis, and no-calibration paired bootstrap results address your concerns. We would be grateful for any further feedback and are happy to clarify anything that remains unclear.
Thank you!
Thank you for the clarification. Your response has largely addressed my concerns, and I will raise my score accordingly.
Reply to Reviewer cFYq — Content sensitivity, paired protocol, and correction scale
Thank you for requesting stronger semantic controls, paired statistical analysis, and a precise account of label exclusion. We clarify that the positive capability claim rests on controlled content interventions and the no-calibration knowledge-pathway test, while matched adaptation comparisons answer a different question. We apologize for the confusing presentation of these distinct attribution questions and of the inherited training-side dependencies. We have revised the main-text component comparisons and downstream-protocol discussion, and include our experimental tables below.
cFYq-W2 / Q3: Content beyond address exposure
We fixed observations, support evidence, reader weights, relation addresses and known masks across 96 synthetic worlds per task using official LimiX predictions. Replacing type-mean content with correct declarations gives:
Content improves independent-query prediction while evidence access remains fixed. Our correction-scale analysis further shows that correct declarations are consistently better at intermediate scales:
At scale 0.50, correct C lowers NRMSE versus corrupted C by 0.007399 (1.828% relative). This indicates that full-strength injection can cause residual-distribution mismatch, while intermediate scaling preserves the expected superiority of correct declarations.
The official-backbone synthetic panel provides complementary capability evidence: supplied correct C improves binary/multiclass F1 by 2.708/3.721 pp and reduces NRMSE from 0.465565 to 0.400805, or 0.064760 (13.910%).
cFYq-W1 / Q1: Paired gain and protocol isolation
We clarify that the paper's knowledge-pathway claim is tested by the no-calibration comparison, not by an average gain over calibrated LimiX. We apologize that the earlier presentation conflated these questions. We have revised the claim and protocol discussion, and include the positive pathway test below:
The paired 95% task-bootstrap reduction interval is [0.000617, 0.002487] ([0.154%, 0.621%] relative). External test labels are never used; training-side calibration parameters are fit on training OOF folds and shared identically across the matched control and Consero. For all 50 regression tasks, recorded scored-role indices do not overlap the recorded backbone context, reader support or affine-fit support. A stricter repeated-split evaluation is discussed in the revised limitations.