- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes MinimaxMerge, a training- and data-free method for merging multiple task-specialized LLM adapters by directly optimizing a worst-case (minimax) objective over tasks. Specifically, it minimizes the maximum per-task reconstruction loss of reduced task vectors per merge unit (typically per layer), where each task’s surrogate loss is an exact quadratic defined by Gram inner products; the resulting convex min–max problem is solved via a tangent-cut (cutting-plane) LP with a certified primal–dual gap.
Critical strengths
1.Technical novelty and innovation. Introduces an explicit worst-case objective for coefficient selection in training-free merging, moving beyond aggregate/mean-focused objectives that can mask buy-out of the weakest task.
2.The experimental design is rigorous,following strict procedural standards and conducting detailed ablation studies.
Critical weaknesses
Technical limitations or concerns. The surrogate-to-accuracy link is unvalidated on the binding worst task (code): measured correlations are weak/wrong-signed, undermining the core premise that minimizing the surrogate protects real worst-task performance.
Experimental gaps or methodological issues. 1.Very limited task suite (T=3: GSM8K, ARC-C, MBPP/HumanEval) constrains generality; no larger multi-task rosters where minimax conservatism might matter most. 2.Many results are “not resolved” under the authors’ gate; adopting a decisive margin from unrelated work instead of a power analysis limits interpretability and masks effect sizes that might be practically relevant.
Clarity or presentation issues. 1.The narrative is verbose and leans heavily on preregistration language, which sometimes obscures central takeaways; key quantitative results could be surfaced more directly. 2.The current version of the paper significantly exceeds the page limit set by the conference.
Questions for authors
1.Can you provide results with a soft-robust objective (e.g., CVaR or log-sum-exp/temperatured max) to trade off conservatism and average performance, and does that alleviate the code-surrogate mismatch?
2.How sensitive are the results to allowing small negative coefficients (relaxing the simplex to an affine constraint with L2 or L1 regularization)? Could controlled extrapolation help the weakest task?
3.Can you report compute/time overhead relative to baselines (per-layer LP rounds, average number of cuts), and whether early stopping on the gap meaningfully changes accuracy?
4.Have you tried feature- or output-level surrogates (e.g., minimizing layer feature drift à la LOT, or logit-space reconstruction on a small unlabeled corpus) while keeping the rest of your pipeline identical?