- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces DepthRouter, which constructs depth-pruned variants of an LLM backbone by skipping FFN or attention sub-blocks and training LoRA adapters in neighboring retained layers through distillation. Some variants additionally incorporate fixed-capacity MoD, while a BERT-mini router selects one variant per query, with the selected index revealed to the servers. Experiments evaluate Llama-3.1-8B-Instruct with MPCache on four LongBench QA datasets and the TREC classification task.
Critical strengths
- The paper is well motivated by the high computational cost of MPC-based LLM inference.
Critical weaknesses
-
Limited novelty. As acknowledged by the authors, DepthRouter combines established techniques. The technical contribution of their adaptation to MPC is limited. Furthermore, the learned quality estimator does not improve the quality–cost frontier of fixed configurations, limiting the benefit of query-level selection.
-
Insufficient experimental validation. The main pruning baselines are evaluated without recovery, and comparisons against existing whole-layer pruning methods with LoRA healing are missing. Moreover, QA F1 is reported on the same 800 prompts used for recovery training and validation, leaving generalization to unseen queries unclear. MPC savings are extrapolated from per-operator profiles rather than validated through end-to-end secure execution, and the costs of the quality estimator and MoD selection operations are excluded.
-
Limited validation across backbones and model scales. All model-quality experiments use Llama-3.1-8B-Instruct. The absence of experiments on other model families or larger models leaves the generality of the proposed design and its scalability to larger LLMs unverified.
-
Imbalanced presentation and insufficient methodological detail. The Introduction and Background occupy substantial space, while important methodological details and controlled evaluations remain underdeveloped.
Questions for authors
See weakness