- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how selecting particular context rows affects the damage caused by label corruption in tabular foundation models. It evaluates input-space and representation-space hubs, input anti-hubs, mean-distance selection, and query-coverage selection on TabPFNv2-12L, TabICLv2, and Mitra. The experiments show that selection effects depend on the attack budget, geometry, and evaluation metric. Representation-hub selection is practically equivalent to matched random selection in normalized loss at the specified 25-label setting. Input anti-hubs produce additional damage at modest budgets, including a 1.00 percentage-point additional accuracy drop at 5% corruption. Query coverage also provides an effective selection rule for some outcomes. The study includes three models, five splits, an eight-dataset main panel, broader hub experiments, and robustness checks for anti-hub selection.
Critical strengths
- The controls isolate a meaningful effect of row selection. Matching source-class counts and replacement labels avoids attributing differences in corruption volume or class composition to the selection rule. This is consequential because substantial total attack damage can coexist with little additional damage over matched random corruption. The paper makes that distinction explicit and evaluates it across budgets.
- The anti-hub result is supported by several targeted checks. At 5% corruption, anti-hubs add 1.00 percentage point of accuracy loss over matched random selection, with positive dataset-level accuracy effects on all eight main datasets. The authors also examine alternative tie choices, row permutations, and coverage-matched controls. These checks make the finding more convincing than a single comparison and challenge the intuitive expectation that highly connected rows should necessarily be the most damaging to corrupt.
Critical weaknesses
- The practical significance of the threat model needs clearer grounding. The manuscript specifies which information each selector uses, but provides limited discussion of a concrete deployment where an adversary can selectively rewrite context labels. Representation-based attacks additionally require access to internal representations, while query coverage requires access to query features. These assumptions may be reasonable in some settings, but they imply different attacker capabilities. A concrete example explaining how modified labels enter the inference context, and which selectors are feasible under that access, would make the security implications easier to assess.
- The generality of the main selection findings remains insufficiently established. The anti-hub and query-coverage results are evaluated primarily on eight datasets, while the broader 24-dataset experiments mainly investigate hubs. The geometric analysis fixes (k=10), and the representation-based selectors use one chosen layer per model. Sensitivity to neighborhood size, representation layer, and alternative multiclass replacement mappings is not established. Since the central conclusions concern geometry-dependent effects, these choices could affect the interpretation of both positive and near-zero findings. The existing tie and coverage checks are useful but address different sources of variation.
Questions for authors
- Robustness clarification: How sensitive are the anti-hub results to the neighborhood size (k), and how sensitive are the representation-hub conclusions to the chosen extraction layer? Were these settings selected before examining the main evaluation results?
- Evaluation clarification: How was the eight-dataset main panel selected, and was it held separate from selector development? Do the anti-hub findings persist on the remaining datasets in the broader panel?